Whose data can we trust: How meta-predictions can be used to uncover credible respondents in survey data

PLoS One. 2019 Dec 2;14(12):e0225432. doi: 10.1371/journal.pone.0225432. eCollection 2019.

Abstract

Many areas of economics use subjective data, although it had been known to present problems regarding its reliability. To improve data quality, researchers may use scoring rules that reward respondents so that it is most profitable for them to tell the truth. However, if the subjects are not well informed about the topic or if they do not pay sufficient attention, they will produce data that could not be dependably used for decision-making even though subjects gave their honest answer. In this paper we show how meta-predictions (respondents' predictions about choices of others) can be used for identification of respondents who produce dependable data. We use purchase intention survey, a popular method to elicit early adoption forecasts for a new concept, as a test bed for our approach. We present results from three online experiments, demonstrating that corrected purchase intentions are closer to the real outcomes.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Attention
  • Commerce
  • Consumer Behavior*
  • Data Accuracy*
  • Decision Making / physiology
  • Forecasting
  • Humans
  • Intention
  • Knowledge*
  • Models, Theoretical
  • Reproducibility of Results
  • Reward
  • Surveys and Questionnaires*
  • Trust / psychology*

Grants and funding

This research was supported by a Marie Curie International Outgoing Fellowship within the 7th European Community Framework Programme, PIOF-GA-2013-622868 - BayInno to SR. https://ec.europa.eu/research/mariecurieactions/ The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.