Conditional generative learning for medical image imputation

Ragheb Raad; Deep Ray; Bino Varghese; Darryl Hwang; Inderbir Gill; Vinay Duddalwar; Assad A Oberai

doi:10.1038/s41598-023-50566-7

Conditional generative learning for medical image imputation

Sci Rep. 2024 Jan 2;14(1):171. doi: 10.1038/s41598-023-50566-7.

Authors

Ragheb Raad¹, Deep Ray², Bino Varghese³, Darryl Hwang³, Inderbir Gill⁴, Vinay Duddalwar³, Assad A Oberai⁵

Affiliations

¹ Aerospace and Mechanical Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, 90089, USA.
² Department of Mathematics, University of Maryland, College Park, MD, 20742, USA.
³ Radiology, Keck School of Medicine, University of Southern California, Los Angeles, CA, 90033, USA.
⁴ Urology, Keck School of Medicine, University of Southern California, Los Angeles, CA, 90033, USA.
⁵ Aerospace and Mechanical Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, 90089, USA. aoberai@usc.edu.

Abstract

Image imputation refers to the task of generating a type of medical image given images of another type. This task becomes challenging when the difference between the available images, and the image to be imputed is large. In this manuscript, one such application is considered. It is derived from the dynamic contrast enhanced computed tomography (CECT) imaging of the kidneys: given an incomplete sequence of three CECT images, we are required to impute the missing image. This task is posed as one of probabilistic inference and a generative algorithm to generate samples of the imputed image, conditioned on the available images, is developed, trained, and tested. The output of this algorithm is the "best guess" of the imputed image, and a pixel-wise image of variance in the imputation. It is demonstrated that this best guess is more accurate than those generated by other, deterministic deep-learning based algorithms, including ones which utilize additional information and more complex loss terms. It is also shown that the pixel-wise variance image, which quantifies the confidence in the reconstruction, can be used to determine whether the result of the imputation meets a specified accuracy threshold and is therefore appropriate for a downstream task.

MeSH terms

Algorithms*
Image Processing, Computer-Assisted / methods
Mental Processes
Tomography, X-Ray Computed*

Grants and funding

W911NF2010050/United States Department of Defense | United States Army | U.S. Army Research, Development and Engineering Command | Army Research Office (ARO)