Objective Task-Based Evaluation of Artificial Intelligence-Based Medical Imaging Methods:: Framework, Strategies, and Role of the Physician

Abhinav K Jha; Kyle J Myers; Nancy A Obuchowski; Ziping Liu; Md Ashequr Rahman; Babak Saboury; Arman Rahmim; Barry A Siegel

doi:10.1016/j.cpet.2021.06.013

Objective Task-Based Evaluation of Artificial Intelligence-Based Medical Imaging Methods:: Framework, Strategies, and Role of the Physician

PET Clin. 2021 Oct;16(4):493-511. doi: 10.1016/j.cpet.2021.06.013.

Authors

Abhinav K Jha¹, Kyle J Myers², Nancy A Obuchowski³, Ziping Liu⁴, Md Ashequr Rahman⁴, Babak Saboury⁵, Arman Rahmim⁶, Barry A Siegel⁷

Affiliations

¹ Department of Biomedical Engineering, Mallinckrodt Institute of Radioly, Alvin J. Siteman Cancer Center, Washington University in St. Louis, 510 S Kingshighway Boulevard, St Louis, MO 63110, USA. Electronic address: a.jha@wustl.edu.
² Division of Imaging, Diagnostics, and Software Reliability, Office of Science and Engineering Laboratories, Center for Devices and Radiological Health, Food and Drug Administration (FDA), Silver Spring, MD, USA.
³ Quantitative Health Sciences, Cleveland Clinic, Cleveland, OH, USA.
⁴ Department of Biomedical Engineering, Washington University in St. Louis, 1 Brookings Drive, St Louis, MO 63130, USA.
⁵ Department of Radiology and Imaging Sciences, Clinical Center, National Institutes of Health, 9000 Rockville Pike, Bethesda, MD 20892, USA.
⁶ Department of Radiology, Department of Physics, University of British Columbia, BC Cancer, BC Cancer Research Institute, 675 West 10th Avenue, Office 6-112, Vancouver, British Columbia V5Z 1L3, Canada.
⁷ Division of Nuclear Medicine, Mallinckrodt Institute of Radiology, Alvin J. Siteman Cancer Center, Washington University School of Medicine, 510 S Kingshighway Boulevard #956, St Louis, MO 63110, USA.

PMID: 34537127
DOI: 10.1016/j.cpet.2021.06.013

Abstract

Artificial intelligence-based methods are showing promise in medical imaging applications. There is substantial interest in clinical translation of these methods, requiring that they be evaluated rigorously. We lay out a framework for objective task-based evaluation of artificial intelligence methods. We provide a list of available tools to conduct this evaluation. We outline the important role of physicians in conducting these evaluation studies. The examples in this article are proposed in the context of PET scans with a focus on evaluating neural network-based methods. However, the framework is also applicable to evaluate other medical imaging modalities and other types of artificial intelligence methods.

Keywords: Artificial intelligence (AI); Machine learning; Medical imaging; Positron emission tomography (PET); Role of physician; Task-based evaluation.

Publication types

Review

MeSH terms

Artificial Intelligence*
Humans
Physicians*
Positron-Emission Tomography