Predicting sepsis using deep learning across international sites: a retrospective development and validation study

Michael Moor; Nicolas Bennett; Drago Plečko; Max Horn; Bastian Rieck; Nicolai Meinshausen; Peter Bühlmann; Karsten Borgwardt

doi:10.1016/j.eclinm.2023.102124

Predicting sepsis using deep learning across international sites: a retrospective development and validation study

EClinicalMedicine. 2023 Aug 11:62:102124. doi: 10.1016/j.eclinm.2023.102124. eCollection 2023 Aug.

Authors

Michael Moor^{1

2

3}, Nicolas Bennett⁴, Drago Plečko⁴, Max Horn^{1

2}, Bastian Rieck^{1

2}, Nicolai Meinshausen⁴, Peter Bühlmann⁴, Karsten Borgwardt^{1

2}

Affiliations

¹ Department of Biosystems Science and Engineering, ETH Zurich, Basel 4058, Switzerland.
² SIB Swiss Institute of Bioinformatics, Switzerland.
³ Department of Computer Science, Stanford University, Stanford, CA, USA.
⁴ Seminar for Statistics, Department of Mathematics, ETH Zurich, Switzerland.

Abstract

Background: When sepsis is detected, organ damage may have progressed to irreversible stages, leading to poor prognosis. The use of machine learning for predicting sepsis early has shown promise, however international validations are missing.

Methods: This was a retrospective, observational, multi-centre cohort study. We developed and externally validated a deep learning system for the prediction of sepsis in the intensive care unit (ICU). Our analysis represents the first international, multi-centre in-ICU cohort study for sepsis prediction using deep learning to our knowledge. Our dataset contains 136,478 unique ICU admissions, representing a refined and harmonised subset of four large ICU databases comprising data collected from ICUs in the US, the Netherlands, and Switzerland between 2001 and 2016. Using the international consensus definition Sepsis-3, we derived hourly-resolved sepsis annotations, amounting to 25,694 (18.8%) patient stays with sepsis. We compared our approach to clinical baselines as well as machine learning baselines and performed an extensive internal and external statistical validation within and across databases, reporting area under the receiver-operating-characteristic curve (AUC).

Findings: Averaged over sites, our model was able to predict sepsis with an AUC of 0.846 (95% confidence interval [CI], 0.841-0.852) on a held-out validation cohort internal to each site, and an AUC of 0.761 (95% CI, 0.746-0.770) when validating externally across sites. Given access to a small fine-tuning set (10% per site), the transfer to target sites was improved to an AUC of 0.807 (95% CI, 0.801-0.813). Our model raised 1.4 false alerts per true alert and detected 80% of the septic patients 3.7 h (95% CI, 3.0-4.3) prior to the onset of sepsis, opening a vital window for intervention.

Interpretation: By monitoring clinical and laboratory measurements in a retrospective simulation of a real-time prediction scenario, a deep learning system for the detection of sepsis generalised to previously unseen ICU cohorts, internationally.

Funding: This study was funded by the Personalized Health and Related Technologies (PHRT) strategic focus area of the ETH domain.

Keywords: Deep learning; Early prediction; Early warning; ICU; Intensive care; Machine learning; Sepsis.