Objective: To evaluate the performance and robustness of a deep learning-based automatic fresh rib fracture detection and positioning system (FRF-DPS).
Methods: CT scans of 18,172 participants admitted to eight hospitals from June 2009 to March 2019 were retrospectively collected. Patients were divided into development set (14,241), multicenter internal test set (1612), and external test set (2319). In internal test set, sensitivity, false positives (FPs) and specificity were used to assess fresh rib fracture detection performance at the lesion- and examination-levels. In external test set, the performance of detecting fresh rib fractures by radiologist and FRF-DPS were evaluated at lesion, rib, and examination levels. Additionally, the accuracy of FRF-DPS in rib positioning was investigated by the ground-truth labeling.
Results: In multicenter internal test set, FRF-DPS showed excellent performance at the lesion- (sensitivity: 0.933 [95%CI, 0.916-0.949], FPs: 0.50 [95%CI, 0.397-0.583]) and examination-level. In external test set, the sensitivity and FPs at the lesion-level of FRF-DPS (0.909 [95%CI, 0.883-0.926], p < 0.001; 0.379 [95%CI, 0.303-0.422], p = 0.001) were better than the radiologist (0.789 [95%CI, 0.766-0.807]; 0.496 [95%CI, 0.383-0.571]), so were the rib- and patient-levels. In subgroup analysis of CT parameters, FRF-DPS were robust (0.894-0.927). Finally, FRF-DPS(0.997 [95%CI, 0.992-1.000], p < 0.001) is more accurate than radiologist (0.981 [95%CI, 0.969-0.996]) in rib positioning and takes 20 times less time.
Conclusion: FRF-DPS achieved high detection rate of fresh rib fractures with low FP values, and precise positioning of ribs, thus can be used in clinical practice to improve the detection rate and work efficiency.
Advances in knowledge: We developed the FRF-DPS system which can detect fresh rib fractures and rib position, and evaluated by a large amount of multicenter data.