Quality and transparency of overviews of systematic reviews

Abstract

Objective: To evaluate the reporting and methodological quality of overviews of systematic reviews.

Method: We developed an 18-item assessment tool for overviews of systematic reviews. We then performed a systematic search for such overviews using the terms ('overview' AND ('meta analys*' OR 'systematic review*')) OR 'umbrella review' in the title. We only included those overviews that were limited to systematic reviews or meta-analyses. Their methodological and reporting quality were assessed by two independent reviewers using the checklist, and differences were resolved by a third reviewer. Data analyses was conducted by SPSS version 15.0 for Windows.

Results: We identified 41 overviews of systematic reviews whose mean total reporting score was 10.78 (SD 3.84) and methodological score 3.05 (SD 2.09). Some important items were not adequately reported: only 69% reported defined eligibility criteria, 76% reported search strategy, 49% reported the process of review selection, 44% reported the data collection process, 5% reported evaluating the reporting quality, 46% reported evaluating methodological quality, and 20% reported assessing the evidence level for each outcome.

Conclusion: The reporting and methodological quality of overviews of systematic reviews was very poor, and there is still much room for improvement. A checklist for overviews of systematic reviews should be developed and used.