Objective: To characterize agreement in the ascertainment of surgical site infections (SSIs) between the National Surgical Quality Improvement Program (NSQIP), National Healthcare Safety Network (NHSN), and administrative data.
Background: The NSQIP, NHSN, and administrative data are the primary systems used to monitor and report SSIs for the purpose of quality control and benchmarking of hospitals and surgeons. These systems have different methods for identifying SSIs.
Methods: We queried the NHSN, NSQIP, and administrative data systems for patients who had an operation at 1 of 4 hospitals within a single health system between January 2013 and September 2015. The detection of an SSI during a postoperative hospitalization was the outcome of analysis. Any SSI detected by one (or more) of these systems was analyzed by 2 reviewers to determine the presence of discrete elements of documentation constituting evidence of SSI. Concordance between the 3 systems (NHSN, NSQIP, and administrative data) was analyzed using Cohen's kappa.
Results: After application of appropriate exclusion criteria, a cohort of 9447 inpatient operations was analyzed. In total, 130 SSIs were detected by 1 or more of the 3 systems, with reported SSI rates of 0.5% (NHSN), 0.7% (administrative data), and 1.0% (NSQIP). Of these 130 SSIs, only 17 SSIs were reported by all 3 systems. The concordance between these 3 systems was moderate (kappa values NSQIP-NHSN = 0.50 [0.40-0.60], administrative-NHSN = 0.36 [0.24-0.47], and administrative-NSQIP = 0.47 [0.38-0.57]). Chart review found that reasons for discordance were related to issues of different criteria as well as inaccuracies.
Conclusion: There is significant discordance in the determination of SSIs reported by the NHSN, NSQIP, and administrative data. The differences and limitations of each of these systems have to be recognized, especially when using these data for quality reports and pay for performance.