Linear embedding by joint Robust Discriminant Analysis and Inter-class Sparsity

Abstract

Linear Discriminant Analysis (LDA) and its variants are widely used as feature extraction methods. They have been used for different classification tasks. However, these methods have some limitations that need to be overcome. The main limitation is that the projection obtained by LDA does not provide a good interpretability for the features. In this paper, we propose a novel supervised method used for multi-class classification that simultaneously performs feature selection and extraction. The targeted projection transformation focuses on the most discriminant original features, and at the same time, makes sure that the transformed features (extracted features) belonging to each class have common sparsity. Our proposed method is called Robust Discriminant Analysis with Feature Selection and Inter-class Sparsity (RDA_FSIS). The corresponding model integrates two types of sparsity. The first type is obtained by imposing the ℓ_2,1 constraint on the projection matrix in order to perform feature selection. The second type of sparsity is obtained by imposing the inter-class sparsity constraint used for ensuring a common sparsity structure in each class. An orthogonal matrix is also introduced in our model in order to guarantee that the extracted features can retain the main variance of the original data and thus improve the robustness to noise. The proposed method retrieves the LDA transformation by taking into account the two types of sparsity. Various experiments are conducted on several image datasets including faces, objects and digits. The projected features are used for multi-class classification. Obtained results show that the proposed method outperforms other competing methods by learning a more compact and discriminative transformation.

Keywords: Feature extraction; Feature selection; Image classification; Inter-class sparsity; Linear discriminant analysis.