Computer-Aided Diagnosis of Spinal Tuberculosis From CT Images Based on Deep Learning With Multimodal Feature Fusion

Zhaotong Li; Fengliang Wu; Fengze Hong; Xiaoyan Gai; Wenli Cao; Zeru Zhang; Timin Yang; Jiu Wang; Song Gao; Chao Peng

doi:10.3389/fmicb.2022.823324

Computer-Aided Diagnosis of Spinal Tuberculosis From CT Images Based on Deep Learning With Multimodal Feature Fusion

Front Microbiol. 2022 Feb 23:13:823324. doi: 10.3389/fmicb.2022.823324. eCollection 2022.

Authors

Zhaotong Li^{1

2}, Fengliang Wu^{3

4}, Fengze Hong⁵, Xiaoyan Gai⁶, Wenli Cao⁷, Zeru Zhang^{1

2}, Timin Yang⁴, Jiu Wang⁴, Song Gao¹, Chao Peng⁴

Affiliations

¹ Institute of Medical Technology, Peking University Health Science Center, Beijing, China.
² School of Health Humanities, Peking University, Beijing, China.
³ Beijing Key Laboratory of Spinal Disease Research, Engineering Research Center of Bone and Joint Precision Medicine, Department of Orthopedics, Peking University Third Hospital, Beijing, China.
⁴ Department of Orthopedic, People's Hospital of Tibet Autonomous Region, Lhasa, China.
⁵ Medical College, Tibet University, Lhasa, China.
⁶ Department of Respiratory and Critical Care Medicine, Peking University Third Hospital, Beijing, China.
⁷ Tuberculosis Department, Beijing Geriatric Hospital, Beijing, China.

Abstract

Background: Spinal tuberculosis (TB) has the highest incidence in remote plateau areas, particularly in Tibet, China, due to inadequate local healthcare services, which not only facilitates the transmission of TB bacteria but also increases the burden on grassroots hospitals. Computer-aided diagnosis (CAD) is urgently required to improve the efficiency of clinical diagnosis of TB using computed tomography (CT) images. However, classical machine learning with handcrafted features generally has low accuracy, and deep learning with self-extracting features relies heavily on the size of medical datasets. Therefore, CAD, which effectively fuses multimodal features, is an alternative solution for spinal TB detection.

Methods: A new deep learning method is proposed that fuses four elaborate image features, specifically three handcrafted features and one convolutional neural network (CNN) feature. Spinal TB CT images were collected from 197 patients with spinal TB, from 2013 to 2020, in the People's Hospital of Tibet Autonomous Region, China; 3,000 effective lumbar spine CT images were randomly screened to our dataset, from which two sets of 1,500 images each were classified as tuberculosis (positive) and health (negative). In addition, virtual data augmentation is proposed to enlarge the handcrafted features of the TB dataset. Essentially, the proposed multimodal feature fusion CNN consists of four main sections: matching network, backbone (ResNet-18/50, VGG-11/16, DenseNet-121/161), fallen network, and gated information fusion network. Detailed performance analyses were conducted based on the multimodal features, proposed augmentation, model stability, and model-focused heatmap.

Results: Experimental results showed that the proposed model with VGG-11 and virtual data augmentation exhibited optimal performance in terms of accuracy, specificity, sensitivity, and area under curve. In addition, an inverse relationship existed between the model size and test accuracy. The model-focused heatmap also shifted from the irrelevant region to the bone destruction caused by TB.

Conclusion: The proposed augmentation effectively simulated the real data distribution in the feature space. More importantly, all the evaluation metrics and analyses demonstrated that the proposed deep learning model exhibits efficient feature fusion for multimodal features. Our study provides a profound insight into the preliminary auxiliary diagnosis of spinal TB from CT images applicable to the Tibetan area.

Keywords: computed tomography; computer-aided diagnosis; deep learning; feature fusion; spinal tuberculosis.