Background: If there are systematic differences in the types of patients captured in registries, then differences in outcomes in centers might be related not to differences in the practice of care, but differences in registry inclusion criteria. We set out to evaluate the effect of variable case ascertainment of dead on arrivals on external benchmarking of risk-adjusted mortality using a form of sensitivity analysis.
Methods: We used data from the National Trauma Data Bank to look for indirect evidence of systematic differences in case ascertainment. We evaluated whether there was any relationship between fewer than expected early (< or = 24 hours) deaths and overall risk-adjusted mortality. Fewer than expected early deaths were estimated through the W statistic and through an adjusted ratio of early to late (E/L) deaths. E/L ratios were assessed due to the potential correlation between performance and absolute number of early deaths as assessed by the W statistic.
Results: We estimate that as many as 47% of all deaths might be missing due to problems with case ascertainment. Centers with unexpectedly few early deaths (W statistic) were consistently high performing centers with a lower than expected overall mortality. More importantly, there was no relationship between the E/L death ratio and overall risk-adjusted mortality.
Conclusions: Variable case ascertainment of dead on arrivals does not affect the ability to assess performance. Given that our approach has several assumptions, it is critically important that external validation of trauma registries be performed. If centers are to be judged through the quality of their data, then it is incumbent to first assure that data quality meets expectations.