Due to the frequent changes in operating conditions, time-varying behaviors, including slow-varying dynamics and switching modes, commonly exist in industrial processes, resulting in different degrees of shifting in the process data distribution. When the data distribution shifts in a relatively wide range, conventional adaptive methods become ineffective since they are unable to distinguish normal shifts from real faults, leading to false alarms. In this study, an incremental variational Bayesian Gaussian mixture model (IncVBGMM) is proposed for developing a fine-scale adaptive monitoring scheme to efficiently accommodate the shifting data distribution caused by different degrees of time-varying behaviors. First, IncVBGMM with decremental optimization is proposed to adapt to the changing data distribution via the automatic complement of local models while reducing redundancy to optimize the mixture model. Then, a fine-scale adaptive monitoring scheme is built with physical interpretations to discern between normal shifts and real faults by joint analysis of the static and dynamic information. In addition, a novel monitoring statistic called the expectation of variational Bayesian inference distance (EVBID) is proposed, which can quantify the distance from samples to the variational monitoring model and indicate the fault effects. Case studies involving a real-world three-phase flow facility reveal that the proposed method can accurately differentiate various types of faults from normal shifts and effectively adapt to the time-varying dynamics.