Historical visit attendance as predictor of treatment interruption in South African HIV patients: Extension of a validated machine learning model

Rachel T Esra; Jacques Carstens; Janne Estill; Ricky Stoch; Sue Le Roux; Tonderai Mabuto; Michael Eisenstein; Olivia Keiser; Mhari Maskew; Matthew P Fox; Lucien De Voux; Kieran Sharpey-Schafer

doi:10.1371/journal.pgph.0002105

Historical visit attendance as predictor of treatment interruption in South African HIV patients: Extension of a validated machine learning model

PLOS Glob Public Health. 2023 Jul 19;3(7):e0002105. doi: 10.1371/journal.pgph.0002105. eCollection 2023.

Authors

Rachel T Esra^{1

2}, Jacques Carstens³, Janne Estill¹, Ricky Stoch⁴, Sue Le Roux⁵, Tonderai Mabuto⁵, Michael Eisenstein⁵, Olivia Keiser¹, Mhari Maskew⁶, Matthew P Fox^{6

7}, Lucien De Voux³, Kieran Sharpey-Schafer³

Affiliations

¹ Institute of Global Health, University of Geneva, Geneva, Switzerland.
² Imperial College of London, London, United Kingdom.
³ Palindrome Data, Cape Town, South Africa.
⁴ Studio Fundi Ltd, London, United Kingdom.
⁵ The Aurum Institute, Johannesburg, South Africa.
⁶ Health Economics and Epidemiology Research Office, Department of Internal Medicine, School of Clinical Medicine, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa.
⁷ Departments of Epidemiology and Global Health, Boston University School of Public Health, Boston, Massachusetts, United States of America.

Abstract

Retention of antiretroviral (ART) patients is a priority for achieving HIV epidemic control in South Africa. While machine-learning methods are being increasingly utilised to identify high risk populations for suboptimal HIV service utilisation, they are limited in terms of explaining relationships between predictors. To further understand these relationships, we implemented machine learning methods optimised for predictive power and traditional statistical methods. We used routinely collected electronic medical record (EMR) data to evaluate longitudinal predictors of lost-to-follow up (LTFU) and temporal interruptions in treatment (IIT) in the first two years of treatment for ART patients in the Gauteng and North West provinces of South Africa. Of the 191,162 ART patients and 1,833,248 visits analysed, 49% experienced at least one IIT and 85% of those returned for a subsequent clinical visit. Patients iteratively transition in and out of treatment indicating that ART retention in South Africa is likely underestimated. Historical visit attendance is shown to be predictive of IIT using machine learning, log binomial regression and survival analyses. Using a previously developed categorical boosting (CatBoost) algorithm, we demonstrate that historical visit attendance alone is able to predict almost half of next missed visits. With the addition of baseline demographic and clinical features, this model is able to predict up to 60% of next missed ART visits with a sensitivity of 61.9% (95% CI: 61.5-62.3%), specificity of 66.5% (95% CI: 66.4-66.7%), and positive predictive value of 19.7% (95% CI: 19.5-19.9%). While the full usage of this model is relevant for settings where infrastructure exists to extract EMR data and run computations in real-time, historical visits attendance alone can be used to identify those at risk of disengaging from HIV care in the absence of other behavioural or observable risk factors.

Copyright: © 2023 Esra et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Grants and funding

The Aurum Institute is funded by the PEPFAR programme under grant number GGH001981: Programmatic Implementation and Technical Assistance for HIV/AIDS & TB Programs in Priority Districts of South Africa. The contents are the responsibility of the authors and do not necessarily reflect the views of PEPFAR, USAID or the United States Government. Jacques Carstens received funding in the form of salary from the commercial company Palindrome Data. Palindrome Data is partially funded by Janssen Pharmaceutica (Pty) Ltd, part of the Janssen Pharmaceutical Companies of Johnson & Johnson. The contents are the responsibility of the authors and do not necessarily reflect the views of Janssen Pharmaceutica (Pty) Ltd. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The corresponding author had full access to all the dataset in the study and had final responsibility for the decision to submit for publication. The specific roles of these authors are articulated in the ‘author contributions’ section.