Real world validation of an AI-based CT hemorrhage detection tool

Dongang Wang; Ruilin Jin; Chun-Chien Shieh; Adrian Y Ng; Hiep Pham; Tej Dugal; Michael Barnett; Luis Winoto; Chenyu Wang; Yael Barnett

doi:10.3389/fneur.2023.1177723

Real world validation of an AI-based CT hemorrhage detection tool

Front Neurol. 2023 Aug 3:14:1177723. doi: 10.3389/fneur.2023.1177723. eCollection 2023.

Authors

Dongang Wang^{1

2}, Ruilin Jin³, Chun-Chien Shieh¹, Adrian Y Ng⁴, Hiep Pham³, Tej Dugal¹, Michael Barnett^{1

2}, Luis Winoto⁴, Chenyu Wang^{1

2}, Yael Barnett^{1

3}

Affiliations

¹ Sydney Neuroimaging Analysis Centre, Camperdown, NSW, Australia.
² Brain and Mind Centre, University of Sydney, Sydney, NSW, Australia.
³ Department of Medical Imaging, St. Vincent's Hospital, Sydney, NSW, Australia.
⁴ Emergency Department, St. Vincent's Hospital, Sydney, NSW, Australia.

Abstract

Introduction: Intracranial hemorrhage (ICH) is a potentially life-threatening medical event that requires expedited diagnosis with computed tomography (CT). Automated medical imaging triaging tools can rapidly bring scans containing critical abnormalities, such as ICH, to the attention of radiologists and clinicians. Here, we retrospectively investigated the real-world performance of VeriScout^™, an artificial intelligence-based CT hemorrhage detection and triage tool.

Methods: Ground truth for the presence or absence of ICH was iteratively determined by expert consensus in an unselected dataset of 527 consecutively acquired non-contrast head CT scans, which were sub-grouped according to the presence of artefact, post-operative features and referral source. The performance of VeriScout^™ was compared with the ground truths for all groups.

Results: VeriScout^™ detected hemorrhage with a sensitivity of 0.92 (CI 0.84-0.96) and a specificity of 0.96 (CI 0.94-0.98) in the global dataset, exceeding the sensitivity of general radiologists (0.88) with only a minor relative decrement in specificity (0.98). Crucially, the AI tool detected 13/14 cases of subarachnoid hemorrhage, a potentially fatal condition that is often missed in emergency department settings. There was no decrement in the performance of VeriScout^™ in scans containing artefact or postoperative change. Using an integrated informatics platform, VeriScout^™ was deployed into the existing radiology workflow. Detected hemorrhage cases were flagged in the hospital radiology information system (RIS) and relevant, annotated, preview images made available in the picture archiving and communications system (PACS) within 10 min.

Conclusion: AI-based radiology worklist prioritization for critical abnormalities, such as ICH, may enhance patient care without adding to radiologist or clinician burden.

Keywords: AI-based tools; clinical validation; deep learning – artificial intelligence; intracranial hemorrhage; post-operative; subarachanoid hemorrhage.