Objectives: When two outcome measures, such as mortality and complication rates, are intended to measure the same underlying quantity (in this case hospital quality of care), one expects they will be highly correlated. In addition, as data quality improves, one expects the correlation will increase. The authors show that these expectations are, in a significant way, mistaken.
Methods: The authors study two outcomes (hospital mortality and complication rates after surgery) using three predictive models that vary in adjustment for severity of illness.
Results: Two hospital rankings, based on each of the two outcomes, are well correlated when not adjusted for severity. However, as clinical data are added to the models, the correlation tends to disappear. The authors explain this based on assumptions regarding the relative size of the partial correlations between mortality, complication rate, and severity covariates.
Conclusions: Before claims of construct validity can be made, investigators must show that correlations between outcomes purporting to measure quality of care are sustained after adequate correction for severity. Most importantly, it should be recognized that inadequately controlled confounding variables may lead to a spurious high correlation between an accepted and a new outcome measure, and a false sense of adequate construct validity.