Manifold optimization-based analysis dictionary learning with an ℓ1∕2-norm regularizer

Abstract

Recently there has been increasing attention towards analysis dictionary learning. In analysis dictionary learning, it is an open problem to obtain the strong sparsity-promoting solutions efficiently while simultaneously avoiding the trivial solutions of the dictionary. In this paper, to obtain the strong sparsity-promoting solutions, we employ the ℓ_1∕2 norm as a regularizer. The very recent study on ℓ_1∕2 norm regularization theory in compressive sensing shows that its solutions can give sparser results than using the ℓ₁ norm. We transform a complex nonconvex optimization into a number of one-dimensional minimization problems. Then the closed-form solutions can be obtained efficiently. To avoid trivial solutions, we apply manifold optimization to update the dictionary directly on the manifold satisfying the orthonormality constraint, so that the dictionary can avoid the trivial solutions well while simultaneously capturing the intrinsic properties of the dictionary. The experiments with synthetic and real-world data verify that the proposed algorithm for analysis dictionary learning can not only obtain strong sparsity-promoting solutions efficiently, but also learn more accurate dictionary in terms of dictionary recovery and image processing than the state-of-the-art algorithms.

Keywords: norm regularizer; Analysis dictionary learning; Manifold optimization; Orthonormality constraint; Sparse model.