Objective: Central line-associated bloodstream infection (BSI) rates are a key quality metric for comparing hospital quality and safety. Traditional BSI surveillance may be limited by interrater variability. We assessed whether a computer-automated method of central line-associated BSI detection can improve the validity of surveillance.
Design: Retrospective cohort study.
Setting: Eight medical and surgical intensive care units (ICUs) in 4 academic medical centers.
Methods: Traditional surveillance (by hospital staff) and computer algorithm surveillance were each compared against a retrospective audit review using a random sample of blood culture episodes during the period 2004-2007 from which an organism was recovered. Episode-level agreement with audit review was measured with κ statistics, and differences were assessed using the test of equal κ coefficients. Linear regression was used to assess the relationship between surveillance performance (κ) and surveillance-reported BSI rates (BSIs per 1,000 central line-days).
Results: We evaluated 664 blood culture episodes. Agreement with audit review was significantly lower for traditional surveillance (κ [95% confidence interval (CI) = 0.44 [0.37-0.51]) than computer algorithm surveillance (κ [95% CI] = 0.58; P = .001). Agreement between traditional surveillance and audit review was heterogeneous across ICUs (P = .01); furthermore, traditional surveillance performed worse among ICUs reporting lower (better) BSI rates (P = .001). In contrast, computer algorithm performance was consistent across ICUs and across the range of computer-reported central line-associated BSI rates. Conclusions: Compared with traditional surveillance of bloodstream infections, computer automated surveillance improves accuracy and reliability, making interfacility performance comparisons more valid.