Influence of Data Distribution on Federated Learning Performance in Tumor Segmentation

Guibo Luo; Tianyu Liu; Jinghui Lu; Xin Chen; Lequan Yu; Jian Wu; Danny Z Chen; Wenli Cai

doi:10.1148/ryai.220082

Influence of Data Distribution on Federated Learning Performance in Tumor Segmentation

Radiol Artif Intell. 2023 Apr 26;5(3):e220082. doi: 10.1148/ryai.220082. eCollection 2023 May.

Authors

Guibo Luo¹, Tianyu Liu¹, Jinghui Lu¹, Xin Chen¹, Lequan Yu¹, Jian Wu¹, Danny Z Chen¹, Wenli Cai¹

Affiliation

¹ From the Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 25 New Chardon St, 400C, Boston, MA 02114 (G.L., T.L., J.L., W.C.); Intel Corporation, Santa Clara, Calif (X.C.); Department of Statistics and Actuarial Science, The University of Hong Kong, Hong Kong (L.Y.); College of Computer Science and Technology, Zhejiang University, Hangzhou, China (J.W.); and Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, Ind (D.Z.C.).

Abstract

Purpose: To investigate the correlation between differences in data distributions and federated deep learning (Fed-DL) algorithm performance in tumor segmentation on CT and MR images.

Materials and methods: Two Fed-DL datasets were retrospectively collected (from November 2020 to December 2021): one dataset of liver tumor CT images (Federated Imaging in Liver Tumor Segmentation [or, FILTS]; three sites, 692 scans) and one publicly available dataset of brain tumor MR images (Federated Tumor Segmentation [or, FeTS]; 23 sites, 1251 scans). Scans from both datasets were grouped according to site, tumor type, tumor size, dataset size, and tumor intensity. To quantify differences in data distributions, the following four distance metrics were calculated: earth mover's distance (EMD), Bhattacharyya distance (BD), χ² distance (CSD), and Kolmogorov-Smirnov distance (KSD). Both federated and centralized nnU-Net models were trained by using the same grouped datasets. Fed-DL model performance was evaluated by using the ratio of Dice coefficients, θ, between federated and centralized models trained and tested on the same 80:20 split datasets.

Results: The Dice coefficient ratio (θ) between federated and centralized models was strongly negatively correlated with the distances between data distributions, with correlation coefficients of -0.920 for EMD, -0.893 for BD, and -0.899 for CSD. However, KSD was weakly correlated with θ, with a correlation coefficient of -0.479.

Conclusion: Performance of Fed-DL models in tumor segmentation on CT and MRI datasets was strongly negatively correlated with the distances between data distributions.Keywords: CT, Abdomen/GI, Liver, Comparative Studies, MR Imaging, Brain/Brain Stem, Convolutional Neural Network (CNN), Federated Deep Learning, Tumor Segmentation, Data Distribution Supplemental material is available for this article. © RSNA, 2023See also the commentary by Kwak and Bai in this issue.

Keywords: Abdomen/GI; Brain/Brain Stem; CT; Comparative Studies; Convolutional Neural Network (CNN); Data Distribution; Federated Deep Learning; Liver; MR Imaging; Tumor Segmentation.