Predicting ophthalmic clinic non-attendance using machine learning: Development and validation of models using nationwide data

Abstract

Background: Ophthalmic clinic non-attendance in New Zealand is associated with poorer health outcomes, marked inequities and costs NZD$30 million per annum. Initiatives to improve attendance typically involve expensive and ineffective brute-force strategies. The aim was to develop machine learning models to accurately predict ophthalmic clinic non-attendance.

Methods: This multicentre, retrospective observational study developed and validated predictive models of clinic non-attendance. Attendance data for 3.1 million appointments from all New Zealand government-funded ophthalmology clinics from 2009 to 2018 were aggregated for analysis. Repeated ten-fold cross validation was used to train and optimise XGBoost and logistic regression models on several demographic and clinic-related variables. Models developed using the entire training set were compared with those restricted to regional subsets of the data.

Results: In the testing data set from 2019, there were 407 574 appointments (median [range] age, 66 [0-105] years; 210 365 [51.6%] female) with a non-attendance rate of 5.7% (n = 23 309 missed appointments), XGBoost models trained on each region's data achieved the highest mean AUROC of 0.764 (SD 0.058) and mean AUPRC of 0.157 (SD 0.072). XGBoost performed better than logistic regression (mean AUROC = 0.756, p = 0.002). Training individual XGBoost models for each region led to better performance than training a single model on the complete nationwide dataset (mean AUROC = 0.754, p = 0.04).

Conclusion: Machine learning algorithms can predict ophthalmic clinic non-attendance with relatively basic demographic and clinic data. These findings suggest further research examining implementation of such algorithms in scheduling systems or public health interventions may be useful.

Keywords: epidemiology; equity; machine learning.