Diagnostic Test Accuracy of Deep Learning Detection of COVID-19: A Systematic Review and Meta-Analysis

Temitope Emmanuel Komolafe; Yuzhu Cao; Benedictor Alexander Nguchu; Patrice Monkam; Ebenezer Obaloluwa Olaniyi; Haotian Sun; Jian Zheng; Xiaodong Yang

doi:10.1016/j.acra.2021.08.008

Diagnostic Test Accuracy of Deep Learning Detection of COVID-19: A Systematic Review and Meta-Analysis

Acad Radiol. 2021 Nov;28(11):1507-1523. doi: 10.1016/j.acra.2021.08.008. Epub 2021 Sep 17.

Authors

Temitope Emmanuel Komolafe¹, Yuzhu Cao¹, Benedictor Alexander Nguchu¹, Patrice Monkam¹, Ebenezer Obaloluwa Olaniyi¹, Haotian Sun¹, Jian Zheng¹, Xiaodong Yang²

Affiliations

¹ School of Biomedical Engineering (Suzhou) (T.E.K.,Y.C., H.S.), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Department of Medical Imaging (T.E.K.,Y.C., H.S., J.Z., X.Y.), Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China; Hefei National Lab for Physical Sciences at the Microscale and Centres for Biomedical Engineering (B.A.N.), University of Science and Technology of China, Hefei, 230026, China; EasySignal Group, Department of Automation (P.M.), Tsinghua University, Beijing 100084, China; Department of Biomedical Engineering (E.O.O.), Shenzhen University, Shenzhen, 518060, China; Jinhua Laboratory (X.Y.), Foshan, 528000, China.
² School of Biomedical Engineering (Suzhou) (T.E.K.,Y.C., H.S.), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Department of Medical Imaging (T.E.K.,Y.C., H.S., J.Z., X.Y.), Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China; Hefei National Lab for Physical Sciences at the Microscale and Centres for Biomedical Engineering (B.A.N.), University of Science and Technology of China, Hefei, 230026, China; EasySignal Group, Department of Automation (P.M.), Tsinghua University, Beijing 100084, China; Department of Biomedical Engineering (E.O.O.), Shenzhen University, Shenzhen, 518060, China; Jinhua Laboratory (X.Y.), Foshan, 528000, China. Electronic address: xiaodong.yang@sibet.ac.cn.

Abstract

Rationale and objective: To perform a meta-analysis to compare the diagnostic test accuracy (DTA) of deep learning (DL) in detecting coronavirus disease 2019 (COVID-19), and to investigate how network architecture and type of datasets affect DL performance.

Materials and methods: We searched PubMed, Web of Science and Inspec from January 1, 2020, to December 3, 2020, for retrospective and prospective studies on deep learning detection with at least reported sensitivity and specificity. Pooled DTA was obtained using random-effect models. Sub-group analysis between studies was also carried out for data source and network architectures.

Results: The pooled sensitivity and specificity were 91% (95% confidence interval [CI]: 88%, 93%; I² = 69%) and 92% (95% CI: 88%, 94%; I² = 88%), respectively for 19 studies. The pooled AUC and diagnostic odds ratio (DOR) were 0.95 (95% CI: 0.88, 0.92) and 112.5 (95% CI: 57.7, 219.3; I² = 90%) respectively. The overall accuracy, recall, F1-score, LR⁺ and LR^- are 89.5%, 89.5%, 89.7%, 23.13 and 0.13. Sub-group analysis shows that the sensitivity and DOR significantly vary with the type of network architectures and sources of data with low heterogeneity are (I² = 0%) and (I² = 18%) for ResNet architecture and single-source datasets, respectively.

Conclusion: The diagnosis of COVID-19 via deep learning has achieved incredible performance, and the source of datasets, as well as network architectures, strongly affect DL performance.

Keywords: COVID-19; Chest computed tomography; Deep learning; Diagnostic test accuracy; Meta-analysis.

Publication types

Meta-Analysis
Research Support, Non-U.S. Gov't
Systematic Review

MeSH terms

COVID-19*
Deep Learning*
Diagnostic Tests, Routine
Humans
Prospective Studies
Retrospective Studies
SARS-CoV-2