Comparison of Conventional Logistic Regression and Machine Learning Methods for Predicting Delayed Cerebral Ischemia After Aneurysmal Subarachnoid Hemorrhage: A Multicentric Observational Cohort Study

Ping Hu; Yuntao Li; Yangfan Liu; Geng Guo; Xu Gao; Zhongzhou Su; Long Wang; Gang Deng; Shuang Yang; Yangzhi Qi; Yang Xu; Liguo Ye; Qian Sun; Xiaohu Nie; Yanqi Sun; Mingchang Li; Hongbo Zhang; Qianxue Chen

doi:10.3389/fnagi.2022.857521

Comparison of Conventional Logistic Regression and Machine Learning Methods for Predicting Delayed Cerebral Ischemia After Aneurysmal Subarachnoid Hemorrhage: A Multicentric Observational Cohort Study

Front Aging Neurosci. 2022 Jun 17:14:857521. doi: 10.3389/fnagi.2022.857521. eCollection 2022.

Authors

Ping Hu^{1

2}, Yuntao Li¹, Yangfan Liu², Geng Guo³, Xu Gao⁴, Zhongzhou Su⁵, Long Wang¹, Gang Deng¹, Shuang Yang^{6

7}, Yangzhi Qi¹, Yang Xu¹, Liguo Ye¹, Qian Sun¹, Xiaohu Nie⁵, Yanqi Sun³, Mingchang Li¹, Hongbo Zhang⁸, Qianxue Chen¹

Affiliations

¹ Department of Neurosurgery, Renmin Hospital of Wuhan University, Wuhan, China.
² Department of Neurosurgery, Affiliated Hospital of Panzhihua University, Panzhihua, China.
³ Department of Neurosurgery, First Hospital of Shanxi Medical University, Taiyuan, China.
⁴ Department of Neurosurgery, General Hospital of Northern Theater Command, Shenyang, China.
⁵ Department of Neurosurgery, Huzhou Central Hospital, Huzhou, China.
⁶ School of Physics and Technology, Wuhan University, Wuhan, China.
⁷ School of Electronic Information and Automation, Guilin University of Aerospace Technology, Guilin, China.
⁸ Department of Neurosurgery, The Second Affiliated Hospital of Nanchang University, Nanchang, China.

Abstract

Background: Timely and accurate prediction of delayed cerebral ischemia is critical for improving the prognosis of patients with aneurysmal subarachnoid hemorrhage. Machine learning (ML) algorithms are increasingly regarded as having a higher prediction power than conventional logistic regression (LR). This study aims to construct LR and ML models and compare their prediction power on delayed cerebral ischemia (DCI) after aneurysmal subarachnoid hemorrhage (aSAH).

Methods: This was a multicenter, retrospective, observational cohort study that enrolled patients with aneurysmal subarachnoid hemorrhage from five hospitals in China. A total of 404 aSAH patients were prospectively enrolled. We randomly divided the patients into training (N = 303) and validation cohorts (N = 101) according to a ratio of 75-25%. One LR and six popular ML algorithms were used to construct models. The area under the receiver operating characteristic curve (AUC), accuracy, balanced accuracy, confusion matrix, sensitivity, specificity, calibration curve, and Hosmer-Lemeshow test were used to assess and compare the model performance. Finally, we calculated each feature of importance.

Results: A total of 112 (27.7%) patients developed DCI. Our results showed that conventional LR with an AUC value of 0.824 (95%CI: 0.73-0.91) in the validation cohort outperformed k-nearest neighbor, decision tree, support vector machine, and extreme gradient boosting model with the AUCs of 0.792 (95%CI: 0.68-0.9, P = 0.46), 0.675 (95%CI: 0.56-0.79, P < 0.01), 0.677 (95%CI: 0.57-0.77, P < 0.01), and 0.78 (95%CI: 0.68-0.87, P = 0.50). However, random forest (RF) and artificial neural network model with the same AUC (0.858, 95%CI: 0.78-0.93, P = 0.26) were better than the LR. The accuracy and the balanced accuracy of the RF were 20.8% and 11% higher than the latter, and the RF also showed good calibration in the validation cohort (Hosmer-Lemeshow: P = 0.203). We found that the CT value of subarachnoid hemorrhage, WBC count, neutrophil count, CT value of cerebral edema, and monocyte count were the five most important features for DCI prediction in the RF model. We then developed an online prediction tool (https://dynamic-nomogram.shinyapps.io/DynNomapp-DCI/) based on important features to calculate DCI risk precisely.

Conclusions: In this multicenter study, we found that several ML methods, particularly RF, outperformed conventional LR. Furthermore, an online prediction tool based on the RF model was developed to identify patients at high risk for DCI after SAH and facilitate timely interventions.

Clinical trial registration: http://www.chictr.org.cn, Unique identifier: ChiCTR2100044448.

Keywords: delayed cerebral ischemia; inflammatory response; logistic regression; machine learning; prediction model; subarachnoid hemorrhage.