How well did experts and laypeople forecast the size of the COVID-19 pandemic?

PLoS One. 2021 May 5;16(5):e0250935. doi: 10.1371/journal.pone.0250935. eCollection 2021.

Abstract

Throughout the COVID-19 pandemic, social and traditional media have disseminated predictions from experts and nonexperts about its expected magnitude. How accurate were the predictions of 'experts'-individuals holding occupations or roles in subject-relevant fields, such as epidemiologists and statisticians-compared with those of the public? We conducted a survey in April 2020 of 140 UK experts and 2,086 UK laypersons; all were asked to make four quantitative predictions about the impact of COVID-19 by 31 Dec 2020. In addition to soliciting point estimates, we asked participants for lower and higher bounds of a range that they felt had a 75% chance of containing the true answer. Experts exhibited greater accuracy and calibration than laypersons, even when restricting the comparison to a subset of laypersons who scored in the top quartile on a numeracy test. Even so, experts substantially underestimated the ultimate extent of the pandemic, and the mean number of predictions for which the expert intervals contained the actual outcome was only 1.8 (out of 4), suggesting that experts should consider broadening the range of scenarios they consider plausible. Predictions of the public were even more inaccurate and poorly calibrated, suggesting that an important role remains for expert predictions as long as experts acknowledge their uncertainty.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Adult
  • COVID-19 / epidemiology*
  • COVID-19 / pathology
  • COVID-19 / virology
  • Female
  • Forecasting*
  • Humans
  • Male
  • Mass Media
  • Pandemics
  • SARS-CoV-2 / isolation & purification
  • Surveys and Questionnaires
  • Uncertainty
  • United Kingdom / epidemiology

Grants and funding

This work was funded by the Winton Centre for Risk and Evidence Communication (https://wintoncentre.maths.cam.ac.uk/), which is supported by a donation from the David and Claudia Harding Foundation. There is no associated grant number. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.