Comparing and integrating US COVID-19 data from multiple sources with anomaly detection and repairing

J Appl Stat. 2021 May 23;50(11-12):2408-2434. doi: 10.1080/02664763.2021.1928016. eCollection 2023.

Abstract

Over the past few months, the outbreak of Coronavirus disease (COVID-19) has been expanding over the world. A reliable and accurate dataset of the cases is vital for scientists to conduct related research and policy-makers to make better decisions. We collect the United States COVID-19 daily reported data from four open sources: the New York Times, the COVID-19 Data Repository by Johns Hopkins University, the COVID Tracking Project at the Atlantic, and the USAFacts, then compare the similarities and differences among them. To obtain reliable data for further analysis, we first examine the cyclical pattern and the following anomalies, which frequently occur in the reported cases: (1) the order dependencies violation, (2) the point or period anomalies, and (3) the issue of reporting delay. To address these detected issues, we propose the corresponding repairing methods and procedures if corrections are necessary. In addition, we integrate the COVID-19 reported cases with the county-level auxiliary information of the local features from official sources, such as health infrastructure, demographic, socioeconomic, and environmental information, which are also essential for understanding the spread of the virus.

Keywords: Anomaly detection; Coronavirus; count time series; data comparison; data integration; outlier correction.

Grants and funding

Zhiling Gu and Li Wang's research was partially supported by the National Science Foundation award DMS-1916204. Shan Yu's research was partially supported by the Iowa State University Plant Sciences Institute Scholars Program. Myungjin Kim's research was partially supported by the National Science Foundation award CCF-1934884 and Laurence H. Baker Center for Bioinformatics & Biological Statistics.