Machine learning using institution-specific multi-modal electronic health records improves mortality risk prediction for cardiac surgery patients

Aaron J Weiss; Arjun S Yadaw; David L Meretzky; Matthew A Levin; David H Adams; Ken McCardle; Gaurav Pandey; Ravi Iyengar

doi:10.1016/j.xjon.2023.03.010

Machine learning using institution-specific multi-modal electronic health records improves mortality risk prediction for cardiac surgery patients

JTCVS Open. 2023 Apr 5:14:214-251. doi: 10.1016/j.xjon.2023.03.010. eCollection 2023 Jun.

Authors

Aaron J Weiss^{1

2

3}, Arjun S Yadaw³, David L Meretzky³, Matthew A Levin⁴, David H Adams¹, Ken McCardle⁵, Gaurav Pandey⁶, Ravi Iyengar³

Affiliations

¹ Department of Cardiovascular Surgery, Icahn School of Medicine at Mount Sinai, New York, NY.
² Department of Thoracic and Cardiovascular Surgery, Heart, Vascular & Thoracic Institute, Cleveland Clinic, Cleveland, Ohio.
³ Department of Pharmacological Sciences and Institute for Systems Biomedicine, Icahn School of Medicine at Mount Sinai, New York, NY.
⁴ Division of Cardiothoracic Anesthesia, Department of Anesthesiology and Critical Care, Icahn School of Medicine at Mount Sinai, New York, NY.
⁵ Department of Clinical Operations, Icahn School of Medicine at Mount Sinai, New York, NY.
⁶ Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY.

Abstract

Background: The Society of Thoracic Surgeons risk scores are widely used to assess risk of morbidity and mortality in specific cardiac surgeries but may not perform optimally in all patients. In a cohort of patients undergoing cardiac surgery, we developed a data-driven, institution-specific machine learning-based model inferred from multi-modal electronic health records and compared the performance with the Society of Thoracic Surgeons models.

Methods: All adult patients undergoing cardiac surgery between 2011 and 2016 were included. Routine electronic health record administrative, demographic, clinical, hemodynamic, laboratory, pharmacological, and procedural data features were extracted. The outcome was postoperative mortality. The database was randomly split into training (development) and test (evaluation) cohorts. Models developed using 4 classification algorithms were compared using 6 evaluation metrics. The performance of the final model was compared with the Society of Thoracic Surgeons models for 7 index surgical procedures.

Results: A total of 6392 patients were included and described by 4016 features. Overall mortality was 3.0% (n = 193). The XGBoost algorithm using only features with no missing data (336 features) yielded the best-performing predictor. When applied to the test set, the predictor performed well (F-measure = 0.775; precision = 0.756; recall = 0.795; accuracy = 0.986; area under the receiver operating characteristic curve = 0.978; area under the precision-recall curve = 0.804). eXtreme Gradient Boosting consistently demonstrated improved performance over the Society of Thoracic Surgeons models when evaluated on index procedures within the test set.

Conclusions: Machine learning models using institution-specific multi-modal electronic health records may improve performance in predicting mortality for individual patients undergoing cardiac surgery compared with the standard-of-care, population-derived Society of Thoracic Surgeons models. Institution-specific models may provide insights complementary to population-derived risk predictions to aid patient-level decision making.

Keywords: cardiac surgery; clinical outcomes; electronic health records; machine learning; prediction modeling; risk prediction.

Grants and funding

R01 HG011407/HG/NHGRI NIH HHS/United States