Is replication possible without fidelity?

Psychol Methods. 2023 Dec;28(6):1446-1455. doi: 10.1037/met0000473. Epub 2022 Jan 20.

Abstract

Failure of replication attempts in experimental psychology might extend beyond p-hacking, publication bias or hidden moderators; reductions in experimental power can be caused by violations of fidelity to a set of experimental protocols. In this article, we run a series of simulations to systematically explore how manipulating fidelity influences effect size. We find statistical patterns that mimic those found in ManyLabs style replications and meta-analyses, suggesting that fidelity violations are present in many replications attempts in psychology. Scholars in intervention science, medicine, and education have developed methods of improving and measuring fidelity, and as replication becomes more mainstream in psychology, the field would benefit from adopting such approaches as well. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

MeSH terms

  • Humans
  • Publication Bias
  • Reproducibility of Results
  • Research Design*