Feature reduction and selection of the best features for classification is a crucial stage for data analysis to reduce training time and overfitting. Commonly, feature reduction techniques rely on selecting features with high relevance to the outcome and minimum mutual information among one another. However, satisfying these criteria does not guarantee the selected features have a high classification power. In this study, we provide an algorithm to build predictive models for the desired outcome, while selecting the most useful features for high classification outcomes. The results of the proposed algorithm are compared with the outcomes of five popular feature reduction and selection techniques. A dataset from an obstructive sleep apnea study (data of 113 and 86 participants as training and blind testing datasets, respectively) was used to illustrate the algorithm's performance. The extracted features used in the algorithm were modeled using three-, four-, and five-feature combinations. The models with a high correlation to the outcome and low overlap percentages among specific subgroups of the data were selected and examined. Then, a set of the best selected models were averaged to provide better classification accuracy. The accuracy of the proposed algorithm has been 25% higher than that when using five popular feature reduction/selection techniques. Furthermore, the proposed algorithm is about 20 times faster than the five popular techniques.
Keywords: Biological signal; Classification accuracy; Correlation; Feature reduction and selection; Feature redundancy; Information maximization; Linear modeling; Mutual information.
© 2021. International Federation for Medical and Biological Engineering.