Assessment of 30 Years of Randomized Controlled Trials in The American Journal of Sports Medicine: 1990-2020

Ajay Shah; Graeme Hoit; Lucy Lan; Daniel B Whelan

doi:10.1177/23259671231161293

Assessment of 30 Years of Randomized Controlled Trials in The American Journal of Sports Medicine: 1990-2020

Orthop J Sports Med. 2023 May 15;11(5):23259671231161293. doi: 10.1177/23259671231161293. eCollection 2023 May.

Authors

Ajay Shah¹, Graeme Hoit^{1

2}, Lucy Lan³, Daniel B Whelan^{1

4}

Affiliations

¹ Division of Orthopaedic Surgery, Department of Surgery, University of Toronto, Toronto, Ontario, Canada.
² Institute for Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada.
³ Michael G. DeGroote School of Medicine, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada.
⁴ Division of Orthopaedic Surgery, Saint Michael's Hospital, Unity Health, Toronto, Ontario, Canada.

Abstract

Background: Randomized controlled trials (RCTs) stand atop the evidence-based hierarchy of study designs for their ability to arrive at results with the lowest risk of bias. Even for RCTs, however, critical appraisal is essential before applying results to clinical practice.

Purpose: To analyze the quality of reporting of RCTs published in The American Journal of Sports Medicine (AJSM) from 1990 to 2020 and to identify trends over time and areas of improvement for future trials.

Study design: Systematic review; Level of evidence, 1.

Methods: We queried the AJSM database for RCTs published between January 1990 and December 2020. Data pertaining to study characteristics were recorded. Quality assessments were conducted using the Detsky quality-of-reporting index and the modified Cochrane risk-of-bias (mROB) tool. Univariate and multivariable models were generated to establish factors with associations to study quality. The Fragility Index was calculated for eligible studies.

Results: A total of 277 RCTs were identified with a median sample size of 70 patients. A total of 19 RCTs were published between 1990 and 2000 (t₁); 82 RCTs between 2001 and 2010 (t₂); and 176 RCTs between 2011 and 2020 (t₃). From t₁ to t₃, significant increases were observed in the overall mean-transformed Detsky score (from 68.2% ± 9.8% to 87.4% ± 10.2%, respectively; P < .001) and mROB score (from 4.7 ± 1.6 to 6.9 ± 1.6, respectively; P < .001). Multivariable regression analysis revealed that trials with follow-up periods of <5 years clearly stated primary outcomes, and a focus on the elbow, shoulder, or knee were associated with higher mean-transformed Detsky and mROB scores. The median Fragility Index was 2 (interquartile range, 0-5) for trials with statistically significant. Studies with small sample sizes (<100 patients) were more likely to have low Fragility Index scores and less likely to have a statistically significant finding in any outcome.

Conclusion: The quantity and quality of published RCTs published in AJSM increased over the past 3 decades. However, single-center trials with small sample sizes were prone to fragile results.

Keywords: critical appraisal; evidence-based medicine; quality appraisal; research methodology; sports medicine.

Publication types

Review