An efficient model selection for linear discriminant function-based recursive feature elimination

Abstract

Model selection is an important issue in support vector machine-based recursive feature elimination (SVM-RFE). However, performing model selection on a linear SVM-RFE is difficult because the generalization error of SVM-RFE is hard to estimate. This paper proposes an approximation method to evaluate the generalization error of a linear SVM-RFE, and designs a new criterion to tune the penalty parameter C. As the computational cost of the proposed algorithm is expensive, several alpha seeding approaches are proposed to reduce the computational complexity. We show that the performance of the proposed algorithm exceeds that of the compared algorithms on bioinformatics datasets, and empirically demonstrate the computational time saving achieved by alpha seeding approaches.

Keywords: Alpha seeding; Model selection; Recursive feature elimination; Support vector machine.