A meta-evaluation of the quality of reporting and execution in ecological meta-analyses

PLoS One. 2023 Oct 12;18(10):e0292606. doi: 10.1371/journal.pone.0292606. eCollection 2023.

Abstract

Quantitatively summarizing results from a collection of primary studies with meta-analysis can help answer ecological questions and identify knowledge gaps. The accuracy of the answers depends on the quality of the meta-analysis. We reviewed the literature assessing the quality of ecological meta-analyses to evaluate current practices and highlight areas that need improvement. From each of the 18 review papers that evaluated the quality of meta-analyses, we calculated the percentage of meta-analyses that met criteria related to specific steps taken in the meta-analysis process (i.e., execution) and the clarity with which those steps were articulated (i.e., reporting). We also re-evaluated all the meta-analyses available from Pappalardo et al. [1] to extract new information on ten additional criteria and to assess how the meta-analyses recognized and addressed non-independence. In general, we observed better performance for criteria related to reporting than for criteria related to execution; however, there was a wide variation among criteria and meta-analyses. Meta-analyses had low compliance with regard to correcting for phylogenetic non-independence, exploring temporal trends in effect sizes, and conducting a multifactorial analysis of moderators (i.e., explanatory variables). In addition, although most meta-analyses included multiple effect sizes per study, only 66% acknowledged some type of non-independence. The types of non-independence reported were most often related to the design of the original experiment (e.g., the use of a shared control) than to other sources (e.g., phylogeny). We suggest that providing specific training and encouraging authors to follow the PRISMA EcoEvo checklist recently developed by O'Dea et al. [2] can improve the quality of ecological meta-analyses.

Publication types

  • Review
  • Meta-Analysis
  • Research Support, U.S. Gov't, Non-P.H.S.

MeSH terms

  • Checklist*
  • Phylogeny

Grants and funding

During the development of this project CWO received support from the National Science Foundation (DEB-1655426 and OCE-1851032); BH, PP and CWO were supported by a grant from the US Department of Energy, Office of Science, Biological and Environmental Research Program (DE-SC-0010632). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.