Apriori prediction of chemotherapy response in locally advanced breast cancer patients using CT imaging and deep learning: transformer versus transfer learning

Amir Moslemi; Laurentius Oscar Osapoetra; Archya Dasgupta; David Alberico; Maureen Trudeau; Sonal Gandhi; Andrea Eisen; Frances Wright; Nicole Look-Hong; Belinda Curpen; Michael C Kolios; Gregory J Czarnota

doi:10.3389/fonc.2024.1359148

Apriori prediction of chemotherapy response in locally advanced breast cancer patients using CT imaging and deep learning: transformer versus transfer learning

Front Oncol. 2024 May 2:14:1359148. doi: 10.3389/fonc.2024.1359148. eCollection 2024.

Authors

Amir Moslemi¹, Laurentius Oscar Osapoetra¹, Archya Dasgupta¹, David Alberico¹, Maureen Trudeau^{2

3}, Sonal Gandhi^{2

3}, Andrea Eisen^{2

3}, Frances Wright^{4

5}, Nicole Look-Hong^{4

5}, Belinda Curpen^{6

7}, Michael C Kolios⁸, Gregory J Czarnota^{1

8

9

10

11}

Affiliations

¹ Physical Sciences, Sunnybrook Research Institute, Toronto, ON, Canada.
² Department of Medical Oncology, Department of Medicine, Sunnybrook Health Sciences Centre, Toronto, ON, Canada.
³ Department of Medicine, University of Toronto, Toronto, ON, Canada.
⁴ Department of Surgical Oncology, Department of Surgery, Sunnybrook Health Sciences Centre, Toronto, ON, Canada.
⁵ Department of Surgery, University of Toronto, Toronto, ON, Canada.
⁶ Department of Medical Imaging, Sunnybrook Health Sciences Centre, Toronto, ON, Canada.
⁷ Department of Medical Imaging, University of Toronto, Toronto, ON, Canada.
⁸ Department of Physics, Toronto Metropolitan University, Toronto, ON, Canada.
⁹ Department of Radiation Oncology, University of Toronto, Toronto, ON, Canada.
¹⁰ Department of Radiation Oncology, Sunnybrook Health Sciences Centre, Toronto, ON, Canada.
¹¹ Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada.

Abstract

Objective: Neoadjuvant chemotherapy (NAC) is a key element of treatment for locally advanced breast cancer (LABC). Predicting the response to NAC for patients with Locally Advanced Breast Cancer (LABC) before treatment initiation could be beneficial to optimize therapy, ensuring the administration of effective treatments. The objective of the work here was to develop a predictive model to predict tumor response to NAC for LABC using deep learning networks and computed tomography (CT).

Materials and methods: Several deep learning approaches were investigated including ViT transformer and VGG16, VGG19, ResNet-50, Res-Net-101, Res-Net-152, InceptionV3 and Xception transfer learning networks. These deep learning networks were applied on CT images to assess the response to NAC. Performance was evaluated based on balanced_accuracy, accuracy, sensitivity and specificity classification metrics. A ViT transformer was applied to utilize the attention mechanism in order to increase the weight of important part image which leads to better discrimination between classes.

Results: Amongst the 117 LABC patients studied, 82 (70%) had clinical-pathological response and 35 (30%) had no response to NAC. The ViT transformer obtained the best performance range (accuracy = 71 ± 3% to accuracy = 77 ± 4%, specificity = 86 ± 6% to specificity = 76 ± 3%, sensitivity = 56 ± 4% to sensitivity = 52 ± 4%, and balanced_accuracy=69 ± 3% to balanced_accuracy=69 ± 3%) depending on the split ratio of train-data and test-data. Xception network obtained the second best results (accuracy = 72 ± 4% to accuracy = 65 ± 4, specificity = 81 ± 6% to specificity = 73 ± 3%, sensitivity = 55 ± 4% to sensitivity = 52 ± 5%, and balanced_accuracy = 66 ± 5% to balanced_accuracy = 60 ± 4%). The worst results were obtained using VGG-16 transfer learning network.

Conclusion: Deep learning networks in conjunction with CT imaging are able to predict the tumor response to NAC for patients with LABC prior to start. A ViT transformer could obtain the best performance, which demonstrated the importance of attention mechanism.

Keywords: LABC; ViT transformer; deep learning; neoadjuvant chemotherapy; response prediction and CT imaging.

Grants and funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. Natural Sciences and Engineering Research Council of Canada (NSERC) as well as Terry Fox Research Institute (TFRI)/Lotte & Hecht Memorial Foundation (project #1115). The funding agencies had no role in the study design, study methodology, study results, or in the preparation of the manuscript.