Importance: The Sepsis Prediction Model (SPM) is a proprietary decision support tool created by Epic Systems; it generates a predicting sepsis score (PSS). The model has not undergone validation against existing sepsis prediction tools, such as Systemic Inflammatory Response Syndrome (SIRS), Sequential Organ Failure Assessment (SOFA), or quick Sepsis-Related Organ Failure Asessement (qSOFA).
Objective: To assess the validity and timeliness of the SPM compared with SIRS, qSOFA, and SOFA.
Design, setting, and participants: This retrospective cohort study included all adults admitted to 5 acute care hospitals in a single US health system between June 5, 2019, and December 31, 2020. Data analysis was conducted from March 2021 to February 2023.
Main outcomes and measures: A sepsis event was defined as receipt of 4 or more days of antimicrobials, blood cultures collected within ±48 hours of initial antimicrobial, and at least 1 organ dysfunction as defined by the organ dysfunction criteria optimized for the electronic health record (eSOFA). Time zero was defined as 15 minutes prior to qualifying antimicrobial or blood culture order.
Results: Of 60 507 total admissions, 1663 (2.7%) met sepsis criteria, with 1324 electronic health record-confirmed sepsis (699 [52.8%] male patients; 298 [22.5%] Black patients; 46 [3.5%] Hispanic/Latinx patients; 945 [71.4%] White patients), 339 COVID-19 sepsis (183 [54.0%] male patients; 98 [28.9%] Black patients; 36 [10.6%] Hispanic/Latinx patients; and 189 [55.8%] White patients), and 58 844 (97.3%; 26 632 [45.2%] male patients; 12 698 [21.6%] Black patients; 3367 [5.7%] Hispanic/Latinx patients; 40 491 White patients) did not meet sepsis criteria. The median (IQR) age was 63 (51 to 73) years for electronic health record-confirmed sepsis, 69 (60 to 77) years for COVID-19 sepsis, and 60 (42 to 72) years for nonsepsis admissions. Within the vendor recommended threshold PSS range of 5 to 8, PSS of 8 or greater had the highest balanced accuracy for classifying a sepsis admission at 0.79 (95% CI, 0.78 to 0.80). Change in SOFA score of 2 or more had the highest sensitivity, at 0.97 (95% CI, 0.97 to 0.98). At a PSS of 8 or greater, median (IQR) time to score positivity from time zero was 68.00 (6.75 to 605.75) minutes. For SIRS, qSOFA, and SOFA, median (IQR) time to score positivity was 7.00 (-105.00 to 08.00) minutes, 74.00 (-22.25 to 599.25) minutes, and 28.00 (-108.50 to 134.00) minutes, respectively.
Conclusions and relevance: In this cohort study of hospital admissions, balanced accuracy of the SPM outperformed other models at higher threshold PSS; however, application of the SPM in a clinical setting was limited by poor timeliness as a sepsis screening tool as compared to SIRS and SOFA.