Effective hybrid feature selection using different bootstrap enhances cancers classification performance

Noura Mohammed Abdelwahed; Gh S El-Tawel; M A Makhlouf

doi:10.1186/s13040-022-00304-y

Effective hybrid feature selection using different bootstrap enhances cancers classification performance

BioData Min. 2022 Sep 30;15(1):24. doi: 10.1186/s13040-022-00304-y.

Authors

Noura Mohammed Abdelwahed¹, Gh S El-Tawel², M A Makhlouf³

Affiliations

¹ Department of Information Systems, Faculty of Computers and Informatics, Suez Canal University, Ismailia, Egypt. malekmalek20131988@gmail.com.
² Department of Computer Science, Faculty of Computers and Informatics, Suez Canal University, Ismailia, Egypt.
³ Department of Information Systems, Faculty of Computers and Informatics, Suez Canal University, Ismailia, Egypt.

Abstract

Background: Machine learning can be used to predict the different onset of human cancers. Highly dimensional data have enormous, complicated problems. One of these is an excessive number of genes plus over-fitting, fitting time, and classification accuracy. Recursive Feature Elimination (RFE) is a wrapper method for selecting the best subset of features that cause the best accuracy. Despite the high performance of RFE, time computation and over-fitting are two disadvantages of this algorithm. Random forest for selection (RFS) proves its effectiveness in selecting the effective features and improving the over-fitting problem.

Method: This paper proposed a method, namely, positions first bootstrap step (PFBS) random forest selection recursive feature elimination (RFS-RFE) and its abbreviation is PFBS- RFS-RFE to enhance cancer classification performance. It used a bootstrap with many positions included in the outer first bootstrap step (OFBS), inner first bootstrap step (IFBS), and outer/ inner first bootstrap step (O/IFBS). In the first position, OFBS is applied as a resampling method (bootstrap) with replacement before selection step. The RFS is applied with bootstrap = false i.e., the whole datasets are used to build each tree. The importance features are hybrid with RFE to select the most relevant subset of features. In the second position, IFBS is applied as a resampling method (bootstrap) with replacement during applied RFS. The importance features are hybrid with RFE. In the third position, O/IFBS is applied as a hybrid of first and second positions. RFE used logistic regression (LR) as an estimator. The proposed methods are incorporated with four classifiers to solve the feature selection problems and modify the performance of RFE, in which five datasets with different size are used to assess the performance of the PFBS-RFS-RFE.

Results: The results showed that the O/IFBS-RFS-RFE achieved the best performance compared with previous work and enhanced the accuracy, variance and ROC area for RNA gene and dermatology erythemato-squamous diseases datasets to become 99.994%, 0.0000004, 1.000 and 100.000%, 0.0 and 1.000, respectively.

Conclusion: High dimensional datasets and RFE algorithm face many troubles in cancers classification performance. PFBS-RFS-RFE is proposed to fix these troubles with different positions. The importance features which extracted from RFS are used with RFE to obtain the effective features.

Keywords: High dimensional data; Learning algorithms; Machine Learning; Over-fitting; Random Forest feature importance; Recursive feature elimination and its disadvantages.