Systematic Analysis of Common Factors Impacting Deep Learning Model Generalizability in Liver Segmentation

Brandon Konkel; Jacob Macdonald; Kyle Lafata; Islam H Zaki; Erol Bozdogan; Mohammad Chaudhry; Yuqi Wang; Gemini Janas; Walter F Wiggins; Mustafa R Bashir

doi:10.1148/ryai.220080

Systematic Analysis of Common Factors Impacting Deep Learning Model Generalizability in Liver Segmentation

Radiol Artif Intell. 2023 Feb 22;5(3):e220080. doi: 10.1148/ryai.220080. eCollection 2023 May.

Authors

Affiliation

¹ From the Department of Radiology (B.K., J.M., K.L., I.H.Z., E.B., M.C., G.J., W.F.W., M.R.B.), Department of Radiation Oncology (K.L.), and Department of Medicine, Division of Gastroenterology (M.R.B.), Duke University School of Medicine, Duke University Medical Center, Box 3808, Durham, NC 27710; Department of Electrical & Computer Engineering, Duke University Pratt School of Engineering, Durham, NC (K.L., Y.W.); Department of Radiology, Faculty of Medicine, Benha University, Benha, Egypt (I.H.Z.); Department of Radiology, College of Medicine-Tucson, University of Arizona, Tucson, AZ (E.B.); and Department of Radiology, Rutgers Health-Newark Beth Israel Medical Center, Newark, NJ (M.C.).

Abstract

Purpose: To investigate the effect of training data type on generalizability of deep learning liver segmentation models.

Materials and methods: This Health Insurance Portability and Accountability Act-compliant retrospective study included 860 MRI and CT abdominal scans obtained between February 2013 and March 2018 and 210 volumes from public datasets. Five single-source models were trained on 100 scans each of T1-weighted fat-suppressed portal venous (dynportal), T1-weighted fat-suppressed precontrast (dynpre), proton density opposed-phase (opposed), single-shot fast spin-echo (ssfse), and T1-weighted non-fat-suppressed (t1nfs) sequence types. A sixth multisource (DeepAll) model was trained on 100 scans consisting of 20 randomly selected scans from each of the five source domains. All models were tested against 18 target domains from unseen vendors, MRI types, and modality (CT). The Dice-Sørensen coefficient (DSC) was used to quantify similarity between manual and model segmentations.

Results: Single-source model performance did not degrade significantly against unseen vendor data. Models trained on T1-weighted dynamic data generally performed well on other T1-weighted dynamic data (DSC = 0.848 ± 0.183 [SD]). The opposed model generalized moderately well to all unseen MRI types (DSC = 0.703 ± 0.229). The ssfse model failed to generalize well to any other MRI type (DSC = 0.089 ± 0.153). Dynamic and opposed models generalized moderately well to CT data (DSC = 0.744 ± 0.206), whereas other single-source models performed poorly (DSC = 0.181 ± 0.192). The DeepAll model generalized well across vendor, modality, and MRI type and against externally sourced data.

Conclusion: Domain shift in liver segmentation appears to be tied to variations in soft-tissue contrast and can be effectively bridged with diversification of soft-tissue representation in training data.Keywords: Convolutional Neural Network (CNN), Deep Learning Algorithms, Machine Learning Algorithms, Supervised Learning, CT, MRI, Liver Segmentation Supplemental material is available for this article. © RSNA, 2023.

Keywords: CT; Convolutional Neural Network (CNN); Deep Learning Algorithms; Liver Segmentation; MRI; Machine Learning Algorithms; Supervised Learning.