Can machine learning predict pharmacotherapy outcomes? An application study in osteoporosis

Abstract

Background and objective: The specific aim of this study is to develop machine learning models as a clinical approach for personalized treatment of osteoporosis. The model performance on outcome prediction was compared between four machine learning algorithms.

Methods: Retrospective, electronic clinical data for patients with suspected or confirmed osteoporosis treated at Wan Fang Hospital between 2011 to 2018 were used as inputs for building the following predictive machine learning models,i.e., artificial neural network (ANN), random forest (RF), support vector machine (SVM) and logistic regression (LR) models. The predicted outcome was defined as an increase/decrease in T-score after treatment. A genetic algorithm was employed to select relevant variables as input features for each model; the leave-one-out method was applied for model building and internal validation. The model with best performance was selected by a separate set of testing. Area under the receiver operating characteristic curve, accuracy, precision, sensitivity and F1 score were calculated to evaluate model performance. Main analysis for all the patients with subclinical or confirmed osteoporosis and subgroup analysis for the patients with confirmed osteoporosis (T score < -2.5) were carried out in this study.

Results: A genetic algorithm was employed to select 12 to 18 features from all 33 variables for the four models. No difference was found in accuracy (ANN, 71.7%; LR, 70.0%; RF, 75.0%; SVM, 66.7%), precision (ANN, 80.0%; LR, 59.3%; RF, 70.0%; SVM, 63.6%), and AUC (ANN, 0.709; LR, 0.731; RF, 0.719; SVM, 0.702) among the ANN, LR, RF and SVM models. Main analysis in performance revealed significant recall in the LR model, as compared to ANN and SVM model; while subgroup revealed significant recall in ANN model, compared to LR and SVM model.

Conclusions: Machine learning-based models hold potential in forecasting the outcomes of treatment for osteoporosis via early initiation of first-line therapy for patients with subclinical disease; or a switch to second-line treatment for patients with a high risk of impending treatment failure. This convenient approach can assist clinicians in adjusting treatment tailored to individual patient for prevention of disease progression or ineffective therapy.

Keywords: Artificial neural network; Genetic algorithm; Logistic regression; Machine learning; Osteoporosis; Random forest; Support vector machine.