Opportunistic biases: Their origins, effects, and an integrated solution

Abstract

Researchers commonly explore their data in multiple ways before deciding which analyses they will include in the final versions of their papers. While this improves the chances of researchers finding publishable results, it introduces an "opportunistic bias," such that the reported relations are stronger or otherwise more supportive of the researcher's theories than they would be without the exploratory process. The magnitudes of opportunistic biases can often be stronger than those of the effects being investigated, leading to invalid conclusions and a lack of clarity in research results. Authors typically do not report their exploratory procedures, so opportunistic biases are very difficult to detect just by reading the final version of a research report. In this article, we explain how a number of accepted research practices can lead to opportunistic biases, discuss the prevalence of these practices in psychology, consider the different effects that opportunistic biases have on psychological science, evaluate the strategies that methodologists have proposed to prevent or correct for the effects of these biases, and introduce an integrated solution to reduce the prevalence and influence of opportunistic biases. The recent prominence of articles discussing questionable research practices both in scientific journals and in the public media underscores the importance of understanding how opportunistic biases are created and how we might undo their effects.