Do variations in hospital admission rates bias comparisons of standardized hospital mortality rates? A population-based cohort study

Abstract

Background: Standardized mortality rates are routinely used as measures of hospital performance and quality. Such metrics may, however, be biased if hospital admission thresholds differ and patient severity is not fully measured.

Aim: To examine whether comparisons of hospital mortality rates suffer from selection bias due to variations in hospital admission rates, using the example of variations by day of the week.

Data: 12,900,687 emergency department attendances and 3,418,446 unplanned admissions to all acute non-specialist hospitals of the National Health Service in England between 1 April 2013 and 28 February 2014.

Methods: Population-based retrospective cohort study. Mortality within 30 days of attendance is modelled as a function of weekend or weekday attendance and hospital-level predictors of admission rates using patient-level risk-adjusted probit and bivariate Heckman selection models. Robustness is supported by the use of different hospital-level predictors.

Results: When examining only the admitted population, patients admitted to hospital at weekends have a 0.206 percentage point higher risk of death within 30 days compared to patients admitted during the week. However, patients attending emergency departments at weekends have a 1.390 percentage point lower probability of being admitted to hospital. Once this selection bias is accounted for, the weekend effect in mortality is reduced by two-thirds to a 0.068 percentage point increase in the risk of death.

Conclusions: Comparisons of standardized hospital mortality rates following unplanned admissions can be biased by variations in emergency department admission rates, leading to incorrect conclusions about quality. The use of mortality as a performance measure could therefore lead to misleading comparisons if admission rates vary and illness severity is not fully controlled for. Accounting for sample selection bias and dependence between admission and mortality rates is vital if accurate comparisons of hospital performance are to be made.

Keywords: Hospital performance; Hospital quality; Mortality; Risk-adjustment; Selection bias; Standardized hospital mortality.