External validation of semi-automated surveillance algorithms for deep surgical site infections after colorectal surgery in an independent country

Abstract

Background: Automated surveillance methods that re-use electronic health record data are considered an attractive alternative to traditional manual surveillance. However, surveillance algorithms need to be thoroughly validated before being implemented in a clinical setting. With semi-automated surveillance patients are classified as low or high probability of having developed infection, and only high probability patients subsequently undergo manual record review. The aim of this study was to externally validate two existing semi-automated surveillance algorithms for deep SSI after colorectal surgery, developed on Spanish and Dutch data, in a Swedish setting.

Methods: The algorithms were validated in 225 randomly selected surgeries from Karolinska University Hospital from the period January 1, 2015 until August 31, 2020. Both algorithms were based on (re)admission and discharge data, mortality, reoperations, radiology orders, and antibiotic prescriptions, while one additionally used microbiology cultures. SSI was based on ECDC definitions. Sensitivity, specificity, positive predictive value, negative predictive value, and workload reduction were assessed compared to manual surveillance.

Results: Both algorithms performed well, yet the algorithm not relying on microbiological culture data had highest sensitivity (97.6, 95%CI: 87.4-99.6), which was comparable to previously published results. The latter algorithm aligned best with clinical practice and would lead to 57% records less to review.

Conclusions: The results highlight the importance of thorough validation before implementation in other clinical settings than in which algorithms were originally developed: the algorithm excluding microbiology cultures had highest sensitivity in this new setting and has the potential to support large-scale semi-automated surveillance of SSI after colorectal surgery.

Keywords: Algorithms; Automated surveillance; Colorectal surgery; Healthcare-associated infections; Surgical site infections; Validation.