Meta-analyses frequently include old trials that are associated with a larger intervention effect: a meta-epidemiological study

Abstract

Objective: To assess whether meta-analyses include older randomized controlled trials (RCTs) and whether intervention effect differ between older and recent RCTs.

Study design and setting: In this meta-epidemiological study of 295 meta-analyses (2940 RCTs) published in 2017-2018, we evaluated the difference in intervention effects between older (i.e., published before 2000) and recent RCTs. We also compared effects by quarters of publication year within each meta-analysis (from quarter 1 including the 25% oldest trials to quarter 4 including the 25% most recent trials). A ratio of odds ratio (ROR) <1 indicates larger effects in older than recent RCTs.

Results: Trials published before 2000 and before 1990 represented 25% and 10% of all trials, respectively. Intervention effects were significantly larger for old than recent RCTs (ROR = 0.92, 95% confidence interval [CI] 0.85-1.00, I² = 22%). Compared with the most recent trials (quarter 4), intervention effects were significantly larger for the oldest trials (quarter 1) (ROR = 0.85, 95% CI 0.79-0.92) and for trials in quarter 2 (ROR = 0.89, 95% CI 0.83-0.96) but not for trials in quarter 3 (ROR = 0.98, 95% CI 0.91-1.05).

Conclusions: Intervention effects were larger for older than recent RCTs. Meta-analyses including older trials should be interpreted cautiously.

Keywords: External validity; Meta-analysis; Meta-epidemiology; Publication date; Systematic review.