Purpose: Comparing the efficacy of a deep-learning model in classifying the etiology of pneumonia on pediatric chest X-rays (CXRs) with that of human readers.
Methods: We built a clinical-pediatric CXR set containing 4035 patients to exploit a deep-learning model called Resnet-50 for differentiating viral from bacterial pneumonia. The dataset was split into training (80%) and validation (20%). Model performance was assessed by receiver operating characteristic curve and area under the curve (AUC) on the first test set of 400 CXRs collected from different studies. For the second test set composed of 100 independent examinations obtained from the daily clinical practice at our institution, the kappa coefficient was selected to measure the interrater agreement in a pairwise fashion for the reference standard, all reviewers, and the model. Gradient-weighted class activation mapping was used to visualize the significant areas contributing to the model prediction.
Results: On the first test set, the best-performing classifier achieved an AUC of 0.919 (p < .001), with a sensitivity of 79.0% and specificity of 88.9%. On the second test set, the classifier achieved performance similar to that of human experts, which resulted in a sensitivity of 74.3% and specificity of 90.8%, positive and negative likelihood ratios of 8.1 and 0.3, respectively. Contingence tables and kappa values further revealed that expert reviewers and model reached substantial agreements on differentiating the etiology of pediatric pneumonia.
Conclusions: This study demonstrated that the model performed similarly as human reviewers and recognized the regions of pathology on CXRs.
Keywords: deep-learning; image classification; pediatric chest X-rays; pneumonia etiology diagnosis.
© 2020 Wiley Periodicals LLC.