Aim: The aim of this study was to compare the methodological quality and input paper characteristics of systematic reviews and meta-analyses reported in the medical and surgical literature by performing a systematic 'overview of reviews'. Ulcerative colitis (UC) and Crohn's disease (CD) were used as the framework for this comparison as they are relatively common serious conditions, with both medical and surgical options for therapy.
Method: Medline, Embase, CINHAL and the Cochrane Database were searched to November 2013. Eligible papers were systematic reviews or meta-analyses that considered a question of therapy in CD or UC. Two independent reviewers selected the papers, extracted the data and scored their methodology using the AMSTAR scoring system. The papers were categorized into medical therapy (M), surgical therapy (S) or medical and surgical therapy (MS) groups. Following retrieval of the sample of meta-evidence papers, the original input studies used in their creation were identified and a search of Medline, Embase, CINHAL and the Cochrane Database was performed. A team of researchers then examined the collection of papers for bibliographic and financial information.
Results: Five hundred papers were identified in the meta-evidence search, of which 118 were deemed eligible. There was a difference in the AMSTAR-rated average quality of the papers between the S and M group (S 7.36 vs M 8.75, P = 0.01). On average S papers were published in journals with a lower impact factor (S 3.26, M 5.04, MS 5.30, P < 0.001). S papers also showed more heterogeneity (I(2) ; S 37%, M 24%, MS 10%, P < 0.001). Some 25% of S meta-analyses used data-sets with significant heterogeneity (I(2) > 75%), compared with 8% of M meta-analyses and 3% of the MS meta-analyses. Some 5% of S papers were done on data sets that had I(2) values > 90%. There was no difference in the average number of papers assessed in each group, the average number of patients per meta-paper, the average time covered by the reviews, the average number of papers considered within each meta-analysis, or the average number of patients considered within each meta-analysis. Considering the conclusions of each meta-analysis, S meta-evidence was 50% more likely than M meta-evidence to be unable to make recommendations for practice. A total of 1499 original input papers were identified, of which 283 were used in more than one review. Within the non-repeated papers (n = 1023) the average impact factor within the S group was lower than that of the M and the MS groups (3.720 vs 11.230 vs 7.563, respectively; ANOVAP < 0.001). M papers had higher rates of pharmaceutical sponsorship than S papers (M 56% vs S 1%) and twice the level of government support (M 16% vs S 8%). Of note, 21% of M papers had corporate sponsorship but did not list any conflict of interest.
Conclusion: Compared with M meta-analyses, S meta-analyses in the UC and CD domain are more likely to be of poorer methodological quality, are of a greater degree of heterogeneity and less often offer a positive conclusion. The papers used to generate meta-evidence in M papers have a greater degree of corporate and government sponsorship, and are more likely to come from journals with higher impact factors.
Keywords: IBD; gastroenterology; surgery.
Colorectal Disease © 2014 The Association of Coloproctology of Great Britain and Ireland.