Using Multi-Scale Convolutional Neural Network Based on Multi-Instance Learning to Predict the Efficacy of Neoadjuvant Chemoradiotherapy for Rectal Cancer

Dehai Zhang; Yongchun Duan; Jing Guo; Yaowei Wang; Yun Yang; Zhenhui Li; Kelong Wang; Lin Wu; Minghao Yu

doi:10.1109/JTEHM.2022.3156851

Using Multi-Scale Convolutional Neural Network Based on Multi-Instance Learning to Predict the Efficacy of Neoadjuvant Chemoradiotherapy for Rectal Cancer

IEEE J Transl Eng Health Med. 2022 Mar 3:10:4300108. doi: 10.1109/JTEHM.2022.3156851. eCollection 2022.

Authors

Dehai Zhang¹, Yongchun Duan¹, Jing Guo², Yaowei Wang¹, Yun Yang³, Zhenhui Li⁴, Kelong Wang¹, Lin Wu⁴, Minghao Yu¹

Affiliations

¹ School of SoftwareYunnan University Kunming 650106 China.
² School of Information Science and EngineeringYunnan University Kunming 650106 China.
³ Key Laboratory in Software Engineering of Yunnan Province, School of SoftwareYunnan University Kunming 650504 China.
⁴ Third Affiliated Hospital of Kunming Medical University, Yunnan Cancer Hospital, Yunnan Cancer Center Kunming 650118 China.

Abstract

Background: At present, radical total mesorectal excision after neoadjuvant chemoradiotherapy is crucial for locally advanced rectal cancer. Therefore, the use of histopathological images analysis technology to predict the efficacy of neoadjuvant chemoradiotherapy for rectal cancer is of great significance for the subsequent treatment of patients.

Methods: In this study, we propose a new pathological images analysis method based on multi-instance learning to predict the efficacy of neoadjuvant chemoradiotherapy for rectal cancer. Specifically, we proposed a gated attention normalization mechanism based on the multilayer perceptron, which accelerates the convergence of stochastic gradient descent optimization and can speed up the training process. We also proposed a bilinear attention multi-scale feature fusion mechanism, which organically fuses the global features of the larger receptive fields and the detailed features of the smaller receptive fields and alleviates the problem of pathological images context information loss caused by block sampling. At the same time, we also designed a weighted loss function to alleviate the problem of imbalance between cancerous instances and normal instances.

Results: We evaluated our method on a locally advanced rectal cancer dataset containing 150 whole slide images. In addition, to verify our method's generalization performance, we also tested on two publicly available datasets, Camelyon16 and MSKCC. The results show that the AUC values of our method on the Camelyon16 and MSKCC datasets reach 0.9337 and 0.9091, respectively.

Conclusion: Our method has outstanding performance and advantages in predicting the efficacy of neoadjuvant chemoradiotherapy for rectal cancer. Clinical and Translational Impact Statement -This study aims to predict the efficacy of neoadjuvant chemoradiotherapy for rectal cancer to assist clinicians quickly diagnose and formulate personalized treatment plans for patients.

Keywords: Pervasive computing; internet of things; neoadjuvant chemoradiotherapy; pathological images; rectal cancer.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Chemoradiotherapy
Humans
Neoadjuvant Therapy* / methods
Neural Networks, Computer
Rectal Neoplasms* / therapy
Retrospective Studies
Treatment Outcome

Grants and funding

This work was supported in part by the Chinese Natural Science Foundation under Grant 61876166 and Grant 61663046, in part by the Yunnan Provincial Major Science and Technology Special Plan Project under Grant 202002AD080001, in part by the Yunnan Basic Research Program for Distinguished Young Youths Project under Grant 202101AV070003, in part by the Yunnan Provincial Major Science and Technology Special Plan Projects: Digitization Research and Application Demonstration of Yunnan Characteristic Industry under Grant 202002AD080001, and in part by the Open Foundation of Key Laboratory in Software Engineering of Yunnan Province under Grant 2020SE304.