Machine learning with multimodal data for COVID-19

Weijie Chen; Rui C Sá; Yuntong Bai; Sandy Napel; Olivier Gevaert; Diane S Lauderdale; Maryellen L Giger

doi:10.1016/j.heliyon.2023.e17934

Machine learning with multimodal data for COVID-19

Heliyon. 2023 Jul 5;9(7):e17934. doi: 10.1016/j.heliyon.2023.e17934. eCollection 2023 Jul.

Authors

Weijie Chen^{1

2}, Rui C Sá^{1

3}, Yuntong Bai^{1

2}, Sandy Napel^{1

4}, Olivier Gevaert^{1

5}, Diane S Lauderdale^{1

6}, Maryellen L Giger^{1

7}

Affiliations

¹ Medical Imaging and Data Resource Center (MIDRC), USA.
² Division of Imaging, Diagnostics, and Software Reliability, Office of Science and Engineering Laboratories, Center for Devices and Radiological Health, USA.
³ Department of Medicine, University of California, San Diego, USA.
⁴ Department of Radiology, Stanford University, USA.
⁵ Department of Medicine and Department of Biomedical Data Science, Stanford University, USA.
⁶ Department of Public Health Sciences, University of Chicago, USA.
⁷ Department of Radiology, University of Chicago, USA.

Abstract

In response to the unprecedented global healthcare crisis of the COVID-19 pandemic, the scientific community has joined forces to tackle the challenges and prepare for future pandemics. Multiple modalities of data have been investigated to understand the nature of COVID-19. In this paper, MIDRC investigators present an overview of the state-of-the-art development of multimodal machine learning for COVID-19 and model assessment considerations for future studies. We begin with a discussion of the lessons learned from radiogenomic studies for cancer diagnosis. We then summarize the multi-modality COVID-19 data investigated in the literature including symptoms and other clinical data, laboratory tests, imaging, pathology, physiology, and other omics data. Publicly available multimodal COVID-19 data provided by MIDRC and other sources are summarized. After an overview of machine learning developments using multimodal data for COVID-19, we present our perspectives on the future development of multimodal machine learning models for COVID-19.

Keywords: COVID-19; Machine learning; Multimodal data.

Publication types

Review