Prospective validation of a near real-time EHR-integrated automated SOFA score calculator

Christopher Aakre; Pablo Moreno Franco; Micaela Ferreyra; Jaben Kitson; Man Li; Vitaly Herasevich

doi:10.1016/j.ijmedinf.2017.04.001

Prospective validation of a near real-time EHR-integrated automated SOFA score calculator

Int J Med Inform. 2017 Jul:103:1-6. doi: 10.1016/j.ijmedinf.2017.04.001. Epub 2017 Apr 3.

Authors

Christopher Aakre¹, Pablo Moreno Franco², Micaela Ferreyra³, Jaben Kitson⁴, Man Li⁴, Vitaly Herasevich⁵

Affiliations

¹ Division of General Internal Medicine, Department of Internal Medicine, Mayo Clinic, 200 First St SW, Rochester, MN 55905, USA. Electronic address: aakre.christopher@mayo.edu.
² Department of Critical Care, Mayo Clinic,4500 San Pablo Rd S, Jacksonville, FL 32224, USA; Department of Transplant, Mayo Clinic, 4500 San Pablo Rd S, Jacksonville, FL 32224, USA.
³ Department of Transplant, Mayo Clinic, 4500 San Pablo Rd S, Jacksonville, FL 32224, USA.
⁴ Department of Information Technology, Mayo Clinic, 200 First St SW, Rochester, MN 55905, USA; Multidisciplinary Epidemiology and Translation Research in Intensive Care (METRIC), Mayo Clinic, 200 First St SW, Rochester, MN 55905, USA.
⁵ Multidisciplinary Epidemiology and Translation Research in Intensive Care (METRIC), Mayo Clinic, 200 First St SW, Rochester, MN 55905, USA; Department of Anesthesiology, Mayo Clinic, 200 First St SW, Rochester, MN 55905, USA.

PMID: 28550994
DOI: 10.1016/j.ijmedinf.2017.04.001

Abstract

Objectives: We created an algorithm for automated Sequential Organ Failure Assessment (SOFA) score calculation within the Electronic Health Record (EHR) to facilitate detection of sepsis based on the Third International Consensus Definitions for Sepsis and Septic Shock (SEPSIS-3) clinical definition. We evaluated the accuracy of near real-time and daily automated SOFA score calculation compared with manual score calculation.

Methods: Automated SOFA scoring computer programs were developed using available EHR data sources and integrated into a critical care focused patient care dashboard at Mayo Clinic in Rochester, Minnesota. We prospectively compared the accuracy of automated versus manual calculation for a sample of patients admitted to the medical intensive care unit at Mayo Clinic Hospitals in Rochester, Minnesota and Jacksonville, Florida. Agreement was calculated with Cohen's kappa statistic. Reason for discrepancy was tabulated during manual review.

Results: Random spot check comparisons were performed 134 times on 27 unique patients, and daily SOFA score comparisons were performed for 215 patients over a total of 1206 patient days. Agreement between automatically scored and manually scored SOFA components for both random spot checks (696 pairs, κ=0.89) and daily calculation (5972 pairs, κ=0.89) was high. The most common discrepancies were in the respiratory component (inaccurate fraction of inspired oxygen retrieval; 200/1206) and creatinine (normal creatinine in patients with no urine output on dialysis; 128/1094). 147 patients were at risk of developing sepsis after intensive care unit admission, 10 later developed sepsis confirmed by chart review. All were identified before onset of sepsis with the ΔSOFA≥2 point criterion and 46 patients were false-positives.

Conclusions: Near real-time automated SOFA scoring was found to have strong agreement with manual score calculation and may be useful for the detection of sepsis utilizing the new SEPSIS-3 definition.

Keywords: Automation; Clinical decision support; Computer-assisted diagnosis; Early diagnosis; Sepsis.

Publication types

Validation Study

MeSH terms

Aged
Algorithms*
Consensus
Critical Care
Electronic Health Records*
Female
Humans
Intensive Care Units
Male
Middle Aged
Minnesota / epidemiology
Organ Dysfunction Scores*
Prospective Studies
Sepsis / diagnosis*
Sepsis / epidemiology

Grants and funding

UL1 TR000135/TR/NCATS NIH HHS/United States