Development and Validation of an Artificial Intelligence Electrocardiogram Recommendation System in the Emergency Department

Dung-Jang Tsai; Shih-Hung Tsai; Hui-Hsun Chiang; Chia-Cheng Lee; Sy-Jou Chen

doi:10.3390/jpm12050700

Development and Validation of an Artificial Intelligence Electrocardiogram Recommendation System in the Emergency Department

J Pers Med. 2022 Apr 27;12(5):700. doi: 10.3390/jpm12050700.

Authors

Dung-Jang Tsai¹, Shih-Hung Tsai², Hui-Hsun Chiang³, Chia-Cheng Lee⁴, Sy-Jou Chen²

Affiliations

¹ Institute of Life Sciences, School of Public Health, National Defense Medical Center, Taipei 11499, Taiwan.
² Department of Emergency Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei 11499, Taiwan.
³ School of Nursing, National Defense Medical Center, Taipei 11499, Taiwan.
⁴ Planning and Management Office, Tri-Service General Hospital, National Defense Medical Center, Taipei 11490, Taiwan.

Abstract

The machine learning-assisted electrocardiogram (ECG) is increasingly recognized for its unprecedented capabilities in diagnosing and predicting cardiovascular diseases. Identifying the need for ECG examination early in emergency department (ED) triage is key to timely artificial intelligence-assisted analysis. We used machine learning to develop and validate a clinical decision support tool to predict ED triage patients' need for ECG. Data from 301,658 ED visits from August 2017 to November 2020 in a tertiary hospital were divided into a development cohort, validation cohort, and two test cohorts that included admissions before and during the COVID-19 pandemic. Models were developed using logistic regression, decision tree, random forest, and XGBoost methods. Their areas under the receiver operating characteristic curves (AUCs), positive predictive values (PPVs), and negative predictive values (NPVs) were compared and validated. In the validation cohort, the AUCs were 0.887 for the XGBoost model, 0.885 for the logistic regression model, 0.878 for the random forest model, and 0.845 for the decision tree model. The XGBoost model was selected for subsequent application. In test cohort 1, the AUC was 0.891, with sensitivity of 0.812, specificity of 0.814, PPV of 0.708 and NPV of 0.886. In test cohort 2, the AUC was 0.885, with sensitivity of 0.816, specificity of 0.812, PPV of 0.659, and NPV of 0.908. In the cumulative incidence analysis, patients not receiving an ECG yet positively predicted by the model had significantly higher probability of receiving the examination within 48 h compared with those negatively predicted by the model. A machine learning model based on triage datasets was developed to predict ECG acquisition with high accuracy. The ECG recommendation can effectively predict whether patients presenting at ED triage will require an ECG, prompting subsequent analysis and decision-making in the ED.

Keywords: artificial intelligence; decision support system; electrocardiogram; emergency department; machine learning; triage.

Abstract

Grants and funding