Deep learning model improves radiologists' performance in detection and classification of breast lesions

Yingshi Sun; Yuhong Qu; Dong Wang; Yi Li; Lin Ye; Jingbo Du; Bing Xu; Baoqing Li; Xiaoting Li; Kexin Zhang; Yanjie Shi; Ruijia Sun; Yichuan Wang; Rong Long; Dengbo Chen; Haijiao Li; Liwei Wang; Min Cao

doi:10.21147/j.issn.1000-9604.2021.06.05

Deep learning model improves radiologists' performance in detection and classification of breast lesions

Chin J Cancer Res. 2021 Dec 31;33(6):682-693. doi: 10.21147/j.issn.1000-9604.2021.06.05.

Authors

Yingshi Sun¹, Yuhong Qu^{1

2}, Dong Wang³, Yi Li⁴, Lin Ye⁵, Jingbo Du⁶, Bing Xu⁷, Baoqing Li⁸, Xiaoting Li¹, Kexin Zhang³, Yanjie Shi¹, Ruijia Sun¹, Yichuan Wang⁹, Rong Long¹, Dengbo Chen⁹, Haijiao Li¹, Liwei Wang^{3

9}, Min Cao¹

Affiliations

¹ Key Laboratory of Carcinogenesis and Translational Research (Ministry of Education/Beijing), Department of Radiology, Peking University Cancer Hospital & Institute, Beijing 100142, China.
² Department of Radiology, Beijing Chaoyang Hospital, Capital Medical University, Beijing 100020, China.
³ Key Laboratory of Machine Perception, MOE, School of EECS, Peking University, Beijing 100871, China.
⁴ Department of Radiology, Shunyi Women's & Children's Hospital of Beijing Children's Hospital, Beijing 101399, China.
⁵ Department of Radiology, Beijing Chaoyang Maternal and Child Health Center, Beijing 122099, China.
⁶ Department of Radiology, Beijing Daxing District People's Hospital, Beijing 102699, China.
⁷ Department of Radiology, Shunyi District Hospital, Beijing 101312, China.
⁸ Department of Medical Imaging, Beijing Shijingshan Hospital, Beijing 100040, China.
⁹ Center for Data Science, Peking University, Beijing 100871, China.

PMID: 35125812
PMCID: PMC8742176
DOI: 10.21147/j.issn.1000-9604.2021.06.05

Abstract

Objective: Computer-aided diagnosis using deep learning algorithms has been initially applied in the field of mammography, but there is no large-scale clinical application.

Methods: This study proposed to develop and verify an artificial intelligence model based on mammography. Firstly, mammograms retrospectively collected from six centers were randomized to a training dataset and a validation dataset for establishing the model. Secondly, the model was tested by comparing 12 radiologists' performance with and without it. Finally, prospectively enrolled women with mammograms from six centers were diagnosed by radiologists with the model. The detection and diagnostic capabilities were evaluated using the free-response receiver operating characteristic (FROC) curve and ROC curve.

Results: The sensitivity of model for detecting lesions after matching was 0.908 for false positive rate of 0.25 in unilateral images. The area under ROC curve (AUC) to distinguish the benign lesions from malignant lesions was 0.855 [95% confidence interval (95% CI): 0.830, 0.880]. The performance of 12 radiologists with the model was higher than that of radiologists alone (AUC: 0.852 vs. 0.805, P=0.005). The mean reading time of with the model was shorter than that of reading alone (80.18 s vs. 62.28 s, P=0.032). In prospective application, the sensitivity of detection reached 0.887 at false positive rate of 0.25; the AUC of radiologists with the model was 0.983 (95% CI: 0.978, 0.988), with sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of 94.36%, 98.07%, 87.76%, and 99.09%, respectively.

Conclusions: The artificial intelligence model exhibits high accuracy for detecting and diagnosing breast lesions, improves diagnostic accuracy and saves time.

Keywords: Breast cancer; artificial intelligence; deep learning; mammography.