Deep learning-based algorithm improved radiologists' performance in bone metastases detection on CT

Abstract

Objectives: To develop and evaluate a deep learning-based algorithm (DLA) for automatic detection of bone metastases on CT.

Methods: This retrospective study included CT scans acquired at a single institution between 2009 and 2019. Positive scans with bone metastases and negative scans without bone metastasis were collected to train the DLA. Another 50 positive and 50 negative scans were collected separately from the training dataset and were divided into validation and test datasets at a 2:3 ratio. The clinical efficacy of the DLA was evaluated in an observer study with board-certified radiologists. Jackknife alternative free-response receiver operating characteristic analysis was used to evaluate observer performance.

Results: A total of 269 positive scans including 1375 bone metastases and 463 negative scans were collected for the training dataset. The number of lesions identified in the validation and test datasets was 49 and 75, respectively. The DLA achieved a sensitivity of 89.8% (44 of 49) with 0.775 false positives per case for the validation dataset and 82.7% (62 of 75) with 0.617 false positives per case for the test dataset. With the DLA, the overall performance of nine radiologists with reference to the weighted alternative free-response receiver operating characteristic figure of merit improved from 0.746 to 0.899 (p < .001). Furthermore, the mean interpretation time per case decreased from 168 to 85 s (p = .004).

Conclusion: With the aid of the algorithm, the overall performance of radiologists in bone metastases detection improved, and the interpretation time decreased at the same time.

Key points: • A deep learning-based algorithm for automatic detection of bone metastases on CT was developed. • In the observer study, overall performance of radiologists in bone metastases detection improved significantly with the aid of the algorithm. • Radiologists' interpretation time decreased at the same time.

Keywords: Bone diseases; Deep learning; Multidetector computed tomography; Neoplasm metastasis; Radiographic image interpretation computer-assisted.