Predictive values of AI-based triage model in suboptimal CT pulmonary angiography

Shadi Ebrahimian; Subba R Digumarthy; Fatemeh Homayounieh; Bernardo C Bizzo; Keith J Dreyer; Mannudeep K Kalra

doi:10.1016/j.clinimag.2022.03.011

Predictive values of AI-based triage model in suboptimal CT pulmonary angiography

Clin Imaging. 2022 Jun:86:25-30. doi: 10.1016/j.clinimag.2022.03.011. Epub 2022 Mar 16.

Authors

Shadi Ebrahimian¹, Subba R Digumarthy², Fatemeh Homayounieh³, Bernardo C Bizzo⁴, Keith J Dreyer⁵, Mannudeep K Kalra⁶

Affiliations

¹ Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02114, USA. Electronic address: sebrahimian@mgh.harvard.edu.
² Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02114, USA. Electronic address: sdigumarthy@mgh.harvard.edu.
³ Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02114, USA.
⁴ Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02114, USA; Data Science Office, Mass General Brigham, Boston, MA 02114, USA. Electronic address: bbizzo@mgh.harvard.edu.
⁵ Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02114, USA; Data Science Office, Mass General Brigham, Boston, MA 02114, USA. Electronic address: kdreyer@partners.org.
⁶ Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02114, USA. Electronic address: mkalra@mgh.harvard.edu.

PMID: 35316621
DOI: 10.1016/j.clinimag.2022.03.011

Abstract

Purpose: We evaluated and compared performance of an acute pulmonary embolism (PE) triaging artificial intelligence (PE-AI) model in suboptimal and optimal CT pulmonary angiography (CTPA).

Methods: In an IRB approved, retrospective study we identified 104 consecutive, suboptimal CTPA which were deemed as suboptimal for PE evaluation in radiology reports due to motion, artifacts and/or inadequate contrast enhancement. We enriched this dataset, with additional 226 optimal CTPA (over same timeframe as suboptimal CTPA) with and without PE. Two thoracic radiologists (ground truth) independently reviewed all 330 CTPA for adequacy (to assess PE down to distal segmental level), reason for suboptimal CTPA (artifacts or poor contrast enhancement), as well as for presence and location of PE. CT values (HU) were measured in the main pulmonary artery. Same attributes were assessed in 80 patients who had repeat or follow-up CTPA following suboptimal CTPA. All CTPA were processed with the PE-AI (Aidoc).

Results: Among 104 suboptimal CTPA (mean age ± standard deviation 56 ± 15 years), 18/104 (17%) were misclassified as suboptimal for PE evaluation in their radiology reports but relabeled as optimal on ground truth evaluation. Of 226 optimal CTPA, 47 (21%) were reclassified as suboptimal CTPA. PEs were present in 97/330 CTPA. PE-AI had similar performance on suboptimal CTPA (sensitivity 100%; specificity 89%; AUC 0.89, 95% CI 0.80-0.98) and optimal CTPA (sensitivity 96%; specificity 92%; AUC 0.87, 95% CI 0.81-0.93).

Conclusion: Suboptimal CTPA examinations do not impair the performance of PE-AI triage model; AI retains clinically meaningful sensitivity and high specificity regardless of diagnostic quality.

Keywords: Artificial intelligence; CT; Pulmonary embolism.

MeSH terms

Angiography
Artificial Intelligence
Computed Tomography Angiography
Contrast Media
Humans
Pulmonary Embolism* / diagnostic imaging
Retrospective Studies
Triage*

Substances

Contrast Media