Interpretable machine learning to predict adverse perinatal outcomes: examining marginal predictive value of risk factors during pregnancy

Abstract

Background: The timely identification of nulliparas at high risk of adverse fetal and neonatal outcomes during pregnancy is crucial for initiating clinical interventions to prevent perinatal complications. Although machine learning methods have been applied to predict preterm birth and other pregnancy complications, many models do not provide explanations of their predictions, limiting the clinical use of the model.

Objective: This study aimed to develop interpretable prediction models for a composite adverse perinatal outcome (stillbirth, neonatal death, estimated Combined Apgar score of <10, or preterm birth) at different points in time during the pregnancy and to evaluate the marginal predictive value of individual predictors in the context of a machine learning model.

Study design: This was a secondary analysis of the Nulliparous Pregnancy Outcomes Study: Monitoring Mothers-to-be data, a prospective cohort study in which 10,038 nulliparous pregnant individuals with singleton pregnancies were enrolled. Here, interpretable prediction models were developed using L1-regularized logistic regression for adverse perinatal outcomes using data available at 3 study visits during the pregnancy (visit 1: 6 0/7 to 13 6/7 weeks of gestation; visit 2: 16 0/7 to 21 6/7 weeks of gestation; visit 3: 22 0/7 to 29 6/7 weeks of gestation). We identified the important predictors for each model using SHapley Additive exPlanations, a model-agnostic method of computing explanations of model predictions, and evaluated the marginal predictive value of each predictor using the DeLong test.

Results: Our interpretable machine learning model had an area under the receiver operating characteristic curves of 0.617 (95% confidence interval, 0.595-0.639; all predictor variables at visit 1), 0.652 (95% confidence interval, 0.631-0.673; all predictor variables at visit 2), and 0.673 (95% confidence interval, 0.651-0.694; all predictor variables at visit 3). For all visits, the placental biomarker inhibin A was a valuable predictor, as including inhibin A resulted in better performance in predicting adverse perinatal outcomes (P<.001, all visits). At visit 1, endoglin was also a valuable predictor (P<.001). At visit 2, free beta human chorionic gonadotropin (P=.001) and uterine artery pulsatility index (P=.023) were also valuable predictors. At visit 3, cervical length was also a valuable predictor (P<.001).

Conclusion: Despite various advances in predictive modeling in obstetrics, the accurate prediction of adverse perinatal outcomes remains difficult. Interpretable machine learning can help clinicians understand how predictions are made, but barriers exist to the widespread clinical adoption of machine learning models for adverse perinatal outcomes. A better understanding of the evolution of risk factors for adverse perinatal outcomes throughout pregnancy is necessary for the development of effective interventions.

Keywords: machine learning; pregnancy complications; risk prediction.