Prospective Evaluation of a Machine-Learning Prediction Model for Missed Radiology Appointments

Abstract

The term "no-show" refers to scheduled appointments that a patient misses, or for which she arrives too late to utilize medical resources. Accurately predicting no-shows creates opportunities to intervene, ensuring that patients receive needed medical resources. A machine-learning (ML) model can accurately identify individuals at high no-show risk, to facilitate strategic and targeted interventions. We used 4,546,104 non-same-day scheduled appointments in our medical system from 1/1/2017 through 1/1/2020 for training data, including 631,386 no-shows. We applied eight ML techniques, which yielded cross-validation AUCs of 0.77-0.93. We then prospectively tested the best performing model, Gradient Boosted Regression Trees, over a 6-week period at a single outpatient location. We observed 123 no-shows. The model accurately identified likely no-show patients retrospectively (AUC 0.93) and prospectively (AUC 0.73, p < 0.0005). Individuals in the highest-risk category were three times more likely to no-show than the average of all other patients. No-show prediction modeling based on machine learning has the potential to identify patients for targeted interventions to improve their access to medical resources, reduce waste in the medical system and improve overall operational efficiency. Caution is advised, due to the potential for bias to decrease the quality of service for patients based on race, zip code, and gender.