Identifying and cultivating superforecasters as a method of improving probabilistic predictions

Perspect Psychol Sci. 2015 May;10(3):267-81. doi: 10.1177/1745691615577794.

Abstract

Across a wide range of tasks, research has shown that people make poor probabilistic predictions of future events. Recently, the U.S. Intelligence Community sponsored a series of forecasting tournaments designed to explore the best strategies for generating accurate subjective probability estimates of geopolitical events. In this article, we describe the winning strategy: culling off top performers each year and assigning them into elite teams of superforecasters. Defying expectations of regression toward the mean 2 years in a row, superforecasters maintained high accuracy across hundreds of questions and a wide array of topics. We find support for four mutually reinforcing explanations of superforecaster performance: (a) cognitive abilities and styles, (b) task-specific skills, (c) motivation and commitment, and (d) enriched environments. These findings suggest that superforecasters are partly discovered and partly created-and that the high-performance incentives of tournaments highlight aspects of human judgment that would not come to light in laboratory paradigms focused on typical performance.

Keywords: expertise; forecasts; predictions; probability training; teams.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Area Under Curve
  • Cognition
  • Environment
  • Forecasting* / methods
  • Humans
  • Judgment*
  • Learning
  • Models, Psychological
  • Motivation
  • Probability
  • ROC Curve
  • Time