Value of Artificial Intelligence in Improving the Accuracy of Diagnosing TI-RADS Category 4 Nodules

Min Lai; Bojian Feng; Jincao Yao; Yifan Wang; Qianmeng Pan; Yuhang Chen; Chen Chen; Na Feng; Fang Shi; Yuan Tian; Lu Gao; Dong Xu

doi:10.1016/j.ultrasmedbio.2023.08.008

Value of Artificial Intelligence in Improving the Accuracy of Diagnosing TI-RADS Category 4 Nodules

Ultrasound Med Biol. 2023 Nov;49(11):2413-2421. doi: 10.1016/j.ultrasmedbio.2023.08.008. Epub 2023 Aug 30.

Authors

Min Lai¹, Bojian Feng², Jincao Yao³, Yifan Wang⁴, Qianmeng Pan⁵, Yuhang Chen⁶, Chen Chen⁷, Na Feng⁸, Fang Shi⁹, Yuan Tian⁹, Lu Gao⁹, Dong Xu¹⁰

Affiliations

¹ Second Clinical Medical College of Zhejiang Chinese Medical University, Hangzhou, China; Taizhou Cancer Hospital, Taizhou, China; Key Laboratory of Minimally Invasive Interventional Therapy and Big Data Artificial Intelligence in Medicine of Taizhou, Taizhou, China.
² Department of Diagnostic Ultrasound Imaging & Interventional Therapy, Zhejiang Cancer Hospital, Hangzhou, China; Key Laboratory of Minimally Invasive Interventional Therapy and Big Data Artificial Intelligence in Medicine of Taizhou, Taizhou, China; Key Laboratory of Head & Neck Cancer Translational Research of Zhejiang Province, Hangzhou, China.
³ Department of Diagnostic Ultrasound Imaging & Interventional Therapy, Zhejiang Cancer Hospital, Hangzhou, China; Key Laboratory of Head & Neck Cancer Translational Research of Zhejiang Province, Hangzhou, China.
⁴ Department of Diagnostic Ultrasound Imaging & Interventional Therapy, Zhejiang Cancer Hospital, Hangzhou, China; Key Laboratory of Minimally Invasive Interventional Therapy and Big Data Artificial Intelligence in Medicine of Taizhou, Taizhou, China.
⁵ Taizhou Cancer Hospital, Taizhou, China; Key Laboratory of Minimally Invasive Interventional Therapy and Big Data Artificial Intelligence in Medicine of Taizhou, Taizhou, China.
⁶ Zhejiang Gongshang University, Hangzhou, China.
⁷ Taizhou Cancer Hospital, Taizhou, China; Key Laboratory of Minimally Invasive Interventional Therapy and Big Data Artificial Intelligence in Medicine of Taizhou, Taizhou, China; Graduate School, Wannan Medical College, Wuhu, China.
⁸ Department of Diagnostic Ultrasound Imaging & Interventional Therapy, Zhejiang Cancer Hospital, Hangzhou, China.
⁹ Capacity Building and Continuing Education Center of National Health Commission, Beijing, China.
¹⁰ Department of Diagnostic Ultrasound Imaging & Interventional Therapy, Zhejiang Cancer Hospital, Hangzhou, China; Taizhou Cancer Hospital, Taizhou, China; Key Laboratory of Minimally Invasive Interventional Therapy and Big Data Artificial Intelligence in Medicine of Taizhou, Taizhou, China; Key Laboratory of Head & Neck Cancer Translational Research of Zhejiang Province, Hangzhou, China. Electronic address: xudong@zjcc.org.cn.

PMID: 37652837
DOI: 10.1016/j.ultrasmedbio.2023.08.008

Abstract

Objective: Considerable heterogeneity is observed in the malignancy rates of thyroid nodules classified as category 4 according to the Thyroid Imaging Reporting and Data System (TI-RADS). This study was aimed at comparing the diagnostic performance of artificial intelligence algorithms and radiologists with different experience levels in distinguishing benign and malignant TI-RADS 4 (TR4) nodules.

Methods: Between January 2019 and September 2022, 1117 TR4 nodules with well-defined pathological findings were collected for this retrospective study. An independent external data set of 125 TR4 nodules was incorporated for testing purposes. Traditional feature-based machine learning (ML) models, deep convolutional neural networks (DCNN) models and a fusion model that integrated the prediction outcomes from all models were used to classify benign and malignant TR4 nodules. A fivefold cross-validation approach was employed, and the diagnostic performance of each model and radiologists was compared.

Results: In the external test data set, the area under the receiver operating characteristic curve (AUROC) of the three DCNN-based secondary transfer learning models-InceptionV3, DenseNet121 and ResNet50-were 0.852, 0.837 and 0.856, respectively. These values were higher than those of the three traditional ML models-logistic regression, multilayer perceptron and random forest-at 0.782, 0.790, and 0.767, respectively, and higher than that of an experienced radiologist (0.815). The fusion diagnostic model we developed, with an AUROC of 0.880, was found to outperform the experienced radiologist in diagnosing TR4 nodules.

Conclusion: The integration of artificial intelligence algorithms into medical imaging studies could improve the accuracy of identifying high-risk TR4 nodules pre-operatively and have significant clinical application potential.

Keywords: Artificial intelligence; Thyroid Imaging Reporting and Data System; Thyroid nodules; Ultrasound.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Algorithms
Artificial Intelligence*
Humans
Neural Networks, Computer
Retrospective Studies
Thyroid Nodule* / diagnostic imaging