Dimensionality reduction of the data set representation for the construction of the quantitative structure-activity relationship classification models is an important research subject for the interpretability of the models and the computational cost efficiency of the classification algorithms. Feature selection techniques are appropriate as only a short number of relevant features should be used in the classification process because irrelevant and redundant features should be discarded, the same as the noninterpretable ones. In this paper, we propose an embedded feature selection technique for the construction of classification models using the rivality index neighborhood (RINH) algorithm. This technique uses a filter selection in the preprocessing stage considering the selectivity of the features as a selection criterion and a wrapper technique in the processing stage based on the improvement of the accuracy and reliability of the models generated using the RINH algorithm with LTN and GTN functions. The results obtained using the RINH algorithm with and without the selection of features and compared with those results obtained using 14 machine learning algorithms have demonstrated that the feature selection technique proposed in this paper is capable of clearly building more accurate and reliable models, reducing the data dimensionality around 90%, and generating high robust and interpretable models.