National and subnational short-term forecasting of COVID-19 in Germany and Poland during early 2021

Johannes Bracher; Daniel Wolffram; Jannik Deuschel; Konstantin Görgen; Jakob L Ketterer; Alexander Ullrich; Sam Abbott; Maria V Barbarossa; Dimitris Bertsimas; Sangeeta Bhatia; Marcin Bodych; Nikos I Bosse; Jan Pablo Burgard; Lauren Castro; Geoffrey Fairchild; Jochen Fiedler; Jan Fuhrmann; Sebastian Funk; Anna Gambin; Krzysztof Gogolewski; Stefan Heyder; Thomas Hotz; Yuri Kheifetz; Holger Kirsten; Tyll Krueger; Ekaterina Krymova; Neele Leithäuser; Michael L Li; Jan H Meinke; Błażej Miasojedow; Isaac J Michaud; Jan Mohring; Pierre Nouvellet; Jedrzej M Nowosielski; Tomasz Ozanski; Maciej Radwan; Franciszek Rakowski; Markus Scholz; Saksham Soni; Ajitesh Srivastava; Tilmann Gneiting; Melanie Schienle

doi:10.1038/s43856-022-00191-8

National and subnational short-term forecasting of COVID-19 in Germany and Poland during early 2021

Commun Med (Lond). 2022 Oct 31;2(1):136. doi: 10.1038/s43856-022-00191-8.

Authors

Johannes Bracher^{1

2}, Daniel Wolffram^{3

4

5}, Jannik Deuschel³, Konstantin Görgen³, Jakob L Ketterer³, Alexander Ullrich⁶, Sam Abbott⁷, Maria V Barbarossa⁸, Dimitris Bertsimas⁹, Sangeeta Bhatia¹⁰, Marcin Bodych¹¹, Nikos I Bosse⁷, Jan Pablo Burgard¹², Lauren Castro¹³, Geoffrey Fairchild¹³, Jochen Fiedler¹⁴, Jan Fuhrmann¹⁵, Sebastian Funk⁷, Anna Gambin¹⁶, Krzysztof Gogolewski¹⁶, Stefan Heyder¹⁷, Thomas Hotz¹⁷, Yuri Kheifetz¹⁸, Holger Kirsten¹⁸, Tyll Krueger¹¹, Ekaterina Krymova¹⁹, Neele Leithäuser¹⁴, Michael L Li²⁰, Jan H Meinke²¹, Błażej Miasojedow¹⁶, Isaac J Michaud²², Jan Mohring¹⁴, Pierre Nouvellet²³, Jedrzej M Nowosielski²⁴, Tomasz Ozanski¹¹, Maciej Radwan²⁴, Franciszek Rakowski²⁴, Markus Scholz¹⁸, Saksham Soni²⁰, Ajitesh Srivastava²⁵, Tilmann Gneiting^{4

26}, Melanie Schienle^{27

28}

Affiliations

¹ Chair of Statistical Methods and Econometrics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany. johannes.bracher@kit.edu.
² Computational Statistics Group, Heidelberg Institute for Theoretical Studies (HITS), Heidelberg, Germany. johannes.bracher@kit.edu.
³ Chair of Statistical Methods and Econometrics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany.
⁴ Computational Statistics Group, Heidelberg Institute for Theoretical Studies (HITS), Heidelberg, Germany.
⁵ HIDSS4Health - Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany.
⁶ Robert Koch Institute (RKI), Berlin, Germany.
⁷ London School of Hygiene and Tropical Medicine, London, UK.
⁸ Frankfurt Institute for Advanced Studies, Frankfurt, Germany.
⁹ Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA, USA.
¹⁰ MRC Centre for Global Infectious Disease Analysis, Abdul Latif Jameel Institute for Disease and Emergency Analytics (J-IDEA), Imperial College London, London, UK.
¹¹ Wroclaw University of Science and Technology, Wroclaw, Poland.
¹² Economic and Social Statistics Department, University of Trier, Trier, Germany.
¹³ Information Systems and Modeling, Los Alamos National Laboratory, Los Alamos, NM, USA.
¹⁴ Fraunhofer Institute for Industrial Mathematics (ITWM), Kaiserslautern, Germany.
¹⁵ Institute for Applied Mathematics, University of Heidelberg, Heidelberg, Germany.
¹⁶ Faculty of Mathematics, Informatics, and Mechanics, University of Warsaw, Warsaw, Poland.
¹⁷ Institute of Mathematics, Technische Universität Ilmenau, Ilmenau, Germany.
¹⁸ Institute for Medical Informatics, Statistics and Epidemiology, University of Leipzig, Leipzig, Germany.
¹⁹ Swiss Data Science Center, ETH Zürich and EPF Lausanne, Zürich, Switzerland.
²⁰ Operations Research Center, Massachusetts Institute of Technology, Cambridge, MA, USA.
²¹ Jülich Supercomputing Centre, Forschungszentrum Jülich, Jülich, Germany.
²² Statistical Sciences Group, Los Alamos National Laboratory, Los Alamos, NM, USA.
²³ School of Life Sciences, University of Sussex, Brighton, UK.
²⁴ Interdisciplinary Centre for Mathematical and Computational Modelling, University of Warsaw, Warsaw, Poland.
²⁵ Ming Hsieh Department of Computer and Electrical Engineering, University of Southern California, Los Angeles, CA, USA.
²⁶ Institute for Stochastics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany.
²⁷ Chair of Statistical Methods and Econometrics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany. melanie.schienle@kit.edu.
²⁸ Computational Statistics Group, Heidelberg Institute for Theoretical Studies (HITS), Heidelberg, Germany. melanie.schienle@kit.edu.

Abstract

Background: During the COVID-19 pandemic there has been a strong interest in forecasts of the short-term development of epidemiological indicators to inform decision makers. In this study we evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland for the period from January through April 2021.

Methods: We evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland. These were issued by 15 different forecasting models, run by independent research teams. Moreover, we study the performance of combined ensemble forecasts. Evaluation of probabilistic forecasts is based on proper scoring rules, along with interval coverage proportions to assess calibration. The presented work is part of a pre-registered evaluation study.

Results: We find that many, though not all, models outperform a simple baseline model up to four weeks ahead for the considered targets. Ensemble methods show very good relative performance. The addressed time period is characterized by rather stable non-pharmaceutical interventions in both countries, making short-term predictions more straightforward than in previous periods. However, major trend changes in reported cases, like the rebound in cases due to the rise of the B.1.1.7 (Alpha) variant in March 2021, prove challenging to predict.

Conclusions: Multi-model approaches can help to improve the performance of epidemiological forecasts. However, while death numbers can be predicted with some success based on current case and hospitalization data, predictability of case numbers remains low beyond quite short time horizons. Additional data sources including sequencing and mobility data, which were not extensively used in the present study, may help to improve performance.

Plain language summary

We compare forecasts of weekly case and death numbers for COVID-19 in Germany and Poland based on 15 different modelling approaches. These cover the period from January to April 2021 and address numbers of cases and deaths one and two weeks into the future, along with the respective uncertainties. We find that combining different forecasts into one forecast can enable better predictions. However, case numbers over longer periods were challenging to predict. Additional data sources, such as information about different versions of the SARS-CoV-2 virus present in the population, might improve forecasts in the future.

Grants and funding

SIMCARD/Helmholtz Association