Deep Learning-Based Detect-Then-Track Pipeline for Treatment Outcome Assessments in Immunotherapy-Treated Liver Cancer

Jie Zhou; Yujia Xia; Xiaolei Xun; Zhangsheng Yu

doi:10.1007/s10278-024-01132-8

Deep Learning-Based Detect-Then-Track Pipeline for Treatment Outcome Assessments in Immunotherapy-Treated Liver Cancer

J Imaging Inform Med. 2024 May 13. doi: 10.1007/s10278-024-01132-8. Online ahead of print.

Authors

Jie Zhou^#^{1

2}, Yujia Xia^#^{3

2}, Xiaolei Xun⁴, Zhangsheng Yu^{5

6

7

8}

Affiliations

¹ Department of Statistics, School of Mathematical Sciences, Shanghai Jiao Tong University, Shanghai, China.
² SJTU-Yale Joint Center for Biostatistics and Data Science, School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, China.
³ Department of Bioinformatics and Biostatistics, School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, China.
⁴ Global Statistics and Data Science, BeiGene, Shanghai, China.
⁵ Department of Statistics, School of Mathematical Sciences, Shanghai Jiao Tong University, Shanghai, China. yuzhangsheng@sjtu.edu.cn.
⁶ Department of Bioinformatics and Biostatistics, School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, China. yuzhangsheng@sjtu.edu.cn.
⁷ SJTU-Yale Joint Center for Biostatistics and Data Science, School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, China. yuzhangsheng@sjtu.edu.cn.
⁸ Clinical Research Institute, Shanghai Jiao Tong University School of Medicine, Shanghai, China. yuzhangsheng@sjtu.edu.cn.

^# Contributed equally.

PMID: 38740661
DOI: 10.1007/s10278-024-01132-8

Abstract

Accurate treatment outcome assessment is crucial in clinical trials. However, due to the image-reading subjectivity, there exist discrepancies among different radiologists. The situation is common in liver cancer due to the complexity of abdominal scans and the heterogeneity of radiological imaging manifestations in liver subtypes. Therefore, we developed a deep learning-based detect-then-track pipeline that can automatically identify liver lesions from 3D CT scans then longitudinally track target lesions, thereby providing the evaluation of RECIST treatment outcomes in liver cancer. We constructed and validated the pipeline on 173 multi-national patients (344 venous-phase CT scans) consisting of a public dataset and two in-house cohorts of 28 centers. The proposed pipeline achieved a mean average precision of 0.806 and 0.726 of lesion detection on the validation and test sets. The model's diameter measurement reliability and consistency are significantly higher than that of clinicians (p = 1.6 × 10^-4). The pipeline can make precise lesion tracking with accuracies of 85.7% and 90.8% then finally yield the RECIST accuracies of 82.1% and 81.4% on the validation and test sets. Our proposed pipeline can provide precise and convenient RECIST outcome assessments and has the potential to aid clinicians with more efficient therapeutic decisions.

Keywords: Deep learning; Lesion detection; Liver cancer; Longitudinal CTs; Oncologic response.

Abstract

Grants and funding