Simplified Pediatric Index of Mortality 3 Score by Explainable Machine Learning Algorithm

Orkun Baloglu; Matthew Nagy; Chidiebere Ezetendu; Samir Q Latifi; Aziz Nazha

doi:10.1097/CCE.0000000000000561

Simplified Pediatric Index of Mortality 3 Score by Explainable Machine Learning Algorithm

Crit Care Explor. 2021 Oct 19;3(10):e0561. doi: 10.1097/CCE.0000000000000561. eCollection 2021 Oct.

Authors

Orkun Baloglu^{1

2}, Matthew Nagy³, Chidiebere Ezetendu¹, Samir Q Latifi^{1

2}, Aziz Nazha^{4

5}

Affiliations

¹ Pediatric Critical Care Medicine, Cleveland Clinic Children's, Cleveland, OH.
² Cleveland Clinic Children`s Center for Artificial Intelligence (C4AI), Cleveland, OH.
³ Lerner College of Medicine of Case Western Reserve University, Cleveland, OH.
⁴ Hematology and Medical Oncology, Cleveland Clinic, Cleveland, OH.
⁵ Cleveland Clinic, Center for Clinical Artificial Intelligence, Cleveland, OH.

Abstract

Pediatric Index of Mortality 3 is a validated tool including 11 variables for the assessment of mortality risk in PICU patients. With the recent advances in explainable machine learning algorithms, we aimed to assess feasibility of application of these machine learning models to simplify the Pediatric Index of Mortality 3 scoring system in order to decrease time and labor required for data collection and entry for Pediatric Index of Mortality 3.

Design: Single-center, retrospective cohort study. Data from the Virtual Pediatric Systems for patients admitted to Cleveland Clinic Children`s PICU between January 2008 and December 2019 was obtained. Light Gradient Boosting Machine Regressor (a gradient boosting decision tree algorithm) was used for building the machine learning models. Variable importance was analyzed by SHapley Additive exPlanations. All of the 11 Pediatric Index of Mortality 3 variables were used as input variables in the machine learning models to predict Pediatric Index of Mortality 3 risk of mortality as the outcome variable. Mean absolute error, root mean squared error, and R-squared were calculated for each of the 11 machine learning models as model performance parameters.

Setting: Quaternary children's hospital.

Patients: PICU patients.

Interventions: None.

Measurements and main results: Five-thousand sixty-eight patients were analyzed. The machine learning models were able to maintain similar predictive error until the number of input variables decreased to four. The machine learning model with five input variables (mechanical ventilation in the first hour of PICU admission, very-high-risk diagnosis, surgical recovery from a noncardiac procedure, low-risk diagnosis, and base excess) produced lowest mean root mean squared error of 1.49 (95% CI, 1.05-1.93) and highest R-squared of 0.73 (95% CI, 0.6-0.86) with mean absolute error of 0.43 (95% CI, 0.35-0.5) among all the 11 machine learning models.

Conclusions: Explainable machine learning methods were feasible in simplifying the Pediatric Index of Mortality 3 scoring system with similar risk of mortality predictions compared to the original Pediatric Index of Mortality 3 model tested in a single-center dataset.

Keywords: Pediatric Index of Mortality; critical care; data science; machine learning; mortality; pediatrics.