Has the degree of outcome reporting bias in surgical randomized trials changed? A meta-regression analysis

Abstract

Background: Outcome reporting bias in individual trials can compromise the validity of pooled estimates within systematic reviews. Recent strategies have attempted to address outcome reporting bias, which favours the full reporting of statistically significant outcomes over non-significant outcomes. We examined whether the association between full outcome reporting and statistical significance in surgical trials has changed from 2009 to 2019.

Methods: We systematically searched for 350 surgical randomized controlled trials (RCTs) from 2009 and 350 surgical RCTs from 2019. Outcomes were classified as fully reported, partially reported, qualitatively reported or unreported. For each outcome, a contingency table was populated with full outcome reporting (yes/no) and statistical significance (yes/no). We combined odds ratios in random effects meta-analysis to estimate the association between full outcome reporting and statistical significance in 2009 compared with 2019.

Results: Twenty-eight percent of outcomes in 2009 were incompletely reported, compared with 30% in 2019. In 2009, significant outcomes were more likely to be fully reported than non-significant outcomes (OR = 2.4, 95% CI 1.7-3.4, I² = 35%), but the opposite association was seen in 2019 (OR = 0.51, 95% CI 0.34-0.77, I² = 43%). RCTs from 2019 were less likely to demonstrate outcome reporting bias favouring significant outcomes (OR = 0.21, 95% CI 0.12-0.35, P < 0.001).

Conclusion: Outcome reporting bias favouring the full reporting of significant over non-significant outcomes was demonstrated in 2009, but the opposite association was seen in 2019. There remains a high prevalence of incomplete outcome reporting. We recommend ongoing adherence to trial protocol guidelines to improve outcome reporting transparency and completeness.

Keywords: clinical protocols; meta-analysis as topic; outcome assessment; publication bias; randomized controlled trials as topic/standard; registries; surgical procedures.