Objective: To assess for the presence of positive outcome bias in the otolaryngology clinical trial literature. Specifically, we investigate the prevalence of clinical trials with positive findings (CTP) and clinical trials with negative findings (CTN), as well as their quality of evidence and subsequent impact.
Study design: Retrospective analysis.
Setting: Clinical Trials in the Influential Otolaryngology Literature.
Methods: We reviewed all clinical trials published in 4 major otolaryngology journals between 2000 and 2020. We constructed several multivariable regression models to investigate the relationship of finding direction with randomization status and citation count. Subsequently, we incorporated an interaction term between year and the primary covariate of each model to assess the temporal trajectory of these relationships. All models accounted for sample size, journal, subspecialty, and the affiliated program prestige.
Results: Of the 1367 trials analyzed, 1143 (84%) were CTPs, a rate that persisted throughout the study period (aOR 1.00, 95% CI 0.98-1.03). CTPs were significantly less likely to be randomized compared to CTNs (aOR 0.25, 0.17-0.37), a relationship that persisted over time (aOR 1.05, 0.99-1.03). CTPs received significantly more citations compared to CTNs (aIRR 1.41, 1.25-1.60), a disparity that also persisted over time (aIRR 0.99, 0.97-1.01).
Conclusion: The otolaryngology clinical trial literature has been heavily dominated by positive findings. CTPs were more frequently cited and published even with a lower level of evidence compared to CTNs. This bias may influence the objectivity of evidence used to guide clinical practice and warrants attention when reviewing findings and changing practices.
Keywords: Clinical Trial; otolaryngology; positive finding; publication bias.
© 2023 American Academy of Otolaryngology-Head and Neck Surgery Foundation.