Artificial Intelligence (AI) versus POCUS Expert: A Validation Study of Three Automatic AI-Based, Real-Time, Hemodynamic Echocardiographic Assessment Tools

J Clin Med. 2023 Feb 8;12(4):1352. doi: 10.3390/jcm12041352.

Abstract

Background: Point Of Care Ultra-Sound (POCUS) is an operator dependent modality. POCUS examinations usually include 'Eyeballing' the inspected anatomical structure without conducting accurate measurements due to complexity and insufficient time. Automatic real time measuring tools can make accurate measurements fast and simple and dramatically increase examination reliability while saving the operator much time and effort. In this study we aim to assess three automatic tools which are integrated into the Venue™ device by GE: the automatic ejection fraction, velocity time integral, and inferior vena cava tools in comparison to the gold standard-an examination by a POCUS expert.

Methods: A separate study was conducted for each of the three automatic tools. In each study, cardiac views were acquired by a POCUS expert. Relevant measurements were taken by both an auto tool and a POCUS expert who was blinded to the auto tool's measurement. The agreement between the POCUS expert and the auto tool was measured for both the measurements and the image quality using a Cohen's Kappa test.

Results: All three tools have shown good agreement with the POCUS expert for high quality views: auto LVEF (0.498; p < 0.001), auto IVC (0.536; p = 0.009), and the auto VTI (0.655; p = 0.024). Auto VTI has also shown a good agreement for medium quality clips (0.914; p < 0.001). Image quality agreement was significant for the auto EF and auto IVC tools.

Conclusions: The Venue™ show a high agreement with a POCUS expert for high quality views. This shows that auto tools can provide reliable real time assistance in performing accurate measurements, but do not reduce the need of a good image acquisition technique.

Keywords: Artificial Intelligence; Ben Gurion University of the Negev; Echocardiography; Ejection Fraction (EF); IVC collapsibility index (IVC CI); Inferior Vena Cava measurement; Intensive Care Unit (ICU); Point Of Care UltraSound (POCUS); Real Time US tool; Soroka University Medical Center; Velocity Time Integral (VTI).

Grants and funding

This study was funded by General Electric. The sponsor was not involved in the study design, data interpretation and processing, data analysis, drafting of the manuscript or in the decision to publish the results.