Using deep learning and natural language processing models to detect child physical abuse

Niti Shahi; Ashwani K Shahi; Ryan Phillips; Gabrielle Shirek; Daniel M Lindberg; Steven L Moulton

doi:10.1016/j.jpedsurg.2021.03.007

Using deep learning and natural language processing models to detect child physical abuse

J Pediatr Surg. 2021 Dec;56(12):2326-2332. doi: 10.1016/j.jpedsurg.2021.03.007. Epub 2021 Mar 18.

Authors

Niti Shahi¹, Ashwani K Shahi², Ryan Phillips³, Gabrielle Shirek⁴, Daniel M Lindberg⁵, Steven L Moulton³

Affiliations

¹ Division of Pediatric Surgery, Children's Hospital Colorado, Aurora, CO, USA; Department of Surgery, University of Colorado School of Medicine, Aurora, CO, USA; Department of Surgery, University of Massachusetts School of Medicine, Worcester, MA, USA. Electronic address: niti.shahi@umassmemorial.org.
² Division of Pediatric Surgery, Children's Hospital Colorado, Aurora, CO, USA; Department of Surgery, University of Colorado School of Medicine, Aurora, CO, USA; Department of Surgery, University of Massachusetts School of Medicine, Worcester, MA, USA; Department of Emergency Medicine, University of Colorado School of Medicine, Aurora, CO, USA; Kempe Center for the Prevention and Treatment of Child Abuse and Neglect, University of Colorado School of Medicine, Aurora, CO, USA.
³ Division of Pediatric Surgery, Children's Hospital Colorado, Aurora, CO, USA; Department of Surgery, University of Colorado School of Medicine, Aurora, CO, USA.
⁴ Division of Pediatric Surgery, Children's Hospital Colorado, Aurora, CO, USA.
⁵ Department of Emergency Medicine, University of Colorado School of Medicine, Aurora, CO, USA; Kempe Center for the Prevention and Treatment of Child Abuse and Neglect, University of Colorado School of Medicine, Aurora, CO, USA.

PMID: 33838900
DOI: 10.1016/j.jpedsurg.2021.03.007

Abstract

Background: The recognition of child physical abuse can be challenging and often requires a multidisciplinary assessment. Deep learning models, based on clinical characteristics, laboratory studies, and imaging findings, were developed to facilitate unbiased identification of children who may have been abused.

Methods: Level 1 pediatric trauma center registry data from 1/1/2010-1/31/2020 were queried for abused children and matched participants with non-abusive trauma. Observations were de-identified and divided into training and validation sets. Model 1 used patient demographics (age, gender, and insurance type) and clinical characteristics (vital signs, shock index pediatric age-adjusted, Glasgow Coma Score, lactate, base deficit, and international normalized ratio). Model 2 used the same features as Model 1, but with the text of the radiology reports of head computed tomography, brain MRIs, and skeletal surveys. Google's latest BERT Natural Language Processing (NLP) model, which was pre-trained on a large corpus, was used for fine-tuning Model 2. Accuracy, sensitivity, specificity, F1 scores, and positive predictive values were used to assess performance.

Results: Of 1,312 patients, 737 (56.2%) were abused. Model 1 had an accuracy of 86.3%, sensitivity of 87.2%, specificity of 85.1%, F1 score of 0.86, and positive predictive value (PPV) of 88.7% for the validation set with an area under the receiver Operating Curve (ROC AUC) of 0.86. NLP based Model 2 had an accuracy of 93.4%, sensitivity 92.5%, specificity of 94.6%, F1 score of 0.93, and PPV of 95.9% for the validation set, with a ROC AUC of 0.94. Most features had weak individual correlations with abuse (r < 0.3).

Conclusions: Deep learning models accurately distinguished child physical abuse from non-abuse, and NLP further improved the accuracy of the models. Such models could be developed to run in real-time in the electronic medical record and alert clinicians when certain criteria are met, which would prompt them to pursue the diagnosis of abuse.

Level of evidence: III STUDY TYPE: Diagnostic.

Keywords: Abuse; Artificial intelligence; Big data; Deep learning; Machine learning.

MeSH terms

Child
Deep Learning*
Electronic Health Records
Humans
Natural Language Processing*
Physical Abuse
Radiography