Computationally reproducing results from meta-analyses in ecology and evolutionary biology using shared code and data

PLoS One. 2024 Mar 13;19(3):e0300333. doi: 10.1371/journal.pone.0300333. eCollection 2024.

Abstract

Many journals in ecology and evolutionary biology encourage or require authors to make their data and code available alongside articles. In this study we investigated how often this data and code could be used together, when both were available, to computationally reproduce results published in articles. We surveyed the data and code sharing practices of 177 meta-analyses published in ecology and evolutionary biology journals published between 2015-17: 60% of articles shared data only, 1% shared code only, and 15% shared both data and code. In each of the articles which had shared both (n = 26), we selected a target result and attempted to reproduce it. Using the shared data and code files, we successfully reproduced the targeted results in 27-73% of the 26 articles, depending on the stringency of the criteria applied for a successful reproduction. The results from this sample of meta-analyses in the 2015-17 literature can provide a benchmark for future meta-research studies gauging the computational reproducibility of published research in ecology and evolutionary biology.

MeSH terms

  • Biological Evolution
  • Ecology*
  • Publications*
  • Reproducibility of Results

Grants and funding

SK received support from a Melbourne Research Scholarship (https://scholarships.unimelb.edu.au/awards/melbourne-research-scholarship) and an Australian Government Research Training Program (RTP) Scholarship (https://www.education.gov.au/research-block-grants/research-training-program). FF received funding from Australian Research Council Future Fellowship FT150100297 (https://www.arc.gov.au/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.