A methodological review finds mismatch between overall and pairwise overlap analysis in a sample of overviews

Abstract

Objectives: Overlap of primary studies is a key methodological challenge for overviews. There are limited reports of methods used to address overlap, and there is no detailed assessment of the corrected covered area (CCA) of a representative sample of overviews. To describe the approaches used to address overlap, and to estimate the overall and pairwise CCA.

Methods: We searched PubMed for overviews published in 2018. Two authors conducted the screening process. We described the strategy used for assessing overlap, and calculated overall and pairwise CCA for each overview.

Results: We analyzed a random sample of 30 out of 89 eligible articles. Eleven did not address the overlap. Of the remainder, most frequent strategies were visual assessment and discussion of overlap as a limitation. Median overall CCA among the included overviews was 6.7%. The pairwise analysis showed that 52.8% of SR pairs had slight overlap, while 28.3% had very high overlap.

Conclusion: Reported strategies for addressing overlap vary considerably among overview authors. The pairwise approach for assessing the CCA revealed highly overlapped pairs of SRs in overviews with overall slight overlap and vice versa. We encourage authors to complement the overall CCA assessment with a pairwise approach.

Keywords: Corrected covered area; Meta-reviews; Methods; Overlap; Overview; Overview of reviews; Review of reviews; Systematic reviews as topic; Umbrella reviews.