Extended-age Out-of-sample Validation of Risk Stratification Index 3.0 Models Using Commercial All-payer Claims

Abstract

Background: The authors previously reported a broad suite of individualized Risk Stratification Index 3.0 (Health Data Analytics Institute, Inc., USA) models for various meaningful outcomes in patients admitted to a hospital for medical or surgical reasons. The models used International Classification of Diseases, Tenth Revision, trajectories and were restricted to information available at hospital admission, including coding history in the previous year. The models were developed and validated in Medicare patients, mostly age 65 yr or older. The authors sought to determine how well their models predict utilization outcomes and adverse events in younger and healthier populations.

Methods: The authors' analysis was based on All Payer Claims for surgical and medical hospital admissions from Utah and Oregon. Endpoints included unplanned hospital admissions, in-hospital mortality, acute kidney injury, sepsis, pneumonia, respiratory failure, and a composite of major cardiac complications. They prospectively applied previously developed Risk Stratification Index 3.0 models to the younger and healthier 2017 Utah and Oregon state populations and compared the results to their previous out-of-sample Medicare validation analysis.

Results: In the Utah dataset, there were 55,109 All Payer Claims admissions across 40,710 patients. In the Oregon dataset, there were 21,213 admissions from 16,951 patients. Model performance on the two state datasets was similar or better than in Medicare patients, with an average area under the curve of 0.83 (0.71 to 0.91). Model calibration was reasonable with an R2 of 0.93 (0.84 to 0.97) for Utah and 0.85 (0.71 to 0.91) for Oregon. The mean sensitivity for the highest 5% risk population was 28% (17 to 44) for Utah and 37% (20 to 56) for Oregon.

Conclusions: Predictive analytical modeling based on administrative claims history provides individualized risk profiles at hospital admission that may help guide patient management. Similar predictive performance in Medicare and in younger and healthier populations indicates that Risk Stratification Index 3.0 models are valid across a broad range of adult hospital admissions.