Reinforced mixture learning

Abstract

In this article, we formulate the standard mixture learning problem as a Markov Decision Process (MDP). We theoretically show that the objective value of the MDP is equivalent to the log-likelihood of the observed data with a slightly different parameter space constrained by the policy. Different from some classic mixture learning methods such as Expectation-Maximization (EM) algorithm, the proposed reinforced algorithm requires no distribution assumptions and can handle the non-convex clustered data by constructing a model-free reward to evaluate the mixture assignment based on the spectral graph theory and Linear Discriminant Analysis (LDA). Extensive experiments on both synthetic and real examples demonstrate that the proposed method is comparable with the EM algorithm when the Gaussian mixture assumption is satisfied, and significantly outperforms it and other clustering methods in most scenarios when the model is misspecified. A Python implementation of our proposed method is available at https://github.com/leyuanheart/Reinforced-Mixture-Learning.

Keywords: Expectation–maximization; Mixture learning; Policy gradient; Reinforcement learning; Spectral embedding.