The Statistical Fragility of Orbital Fractures: A Systematic Review of Randomized Controlled Trials

Abstract

Background: The P value has often been used as a tool to determine the statistical significance and evaluate the statistical robustness of study findings in orthopedic literature. The purpose of this study is to apply both the fragility index (FI) and the fragility quotient (FQ) to evaluate the degree of statistical fragility in orbital fracture literature. We hypothesized that the dichotomous outcomes within the orbital fracture literature will be vulnerable to a small number of outcome event reversals and will be statistically fragile.

Methods: Using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), the authors identified all dichotomous data for randomized controlled trials (RCTs) in orbital fracture literature and performed a PubMed search from 2000 to 2022. The FI of each outcome was calculated through the reversal of a single outcome event until significance was reversed. The FQ was calculated by dividing each FI by study sample size. The interquartile range (IQR) was also calculated for the FI and FQ.

Results: Of the 3,329 studies screened, 28 met the criteria with 10 RCTs evaluating orbital fractures included for analysis. A total of 58 outcome events with 22 significant (P < .05) outcomes and 36 nonsignificant (P ≥ .05) outcomes were identified. The overall FI and FQ for all 58 outcomes was 5 (IQR: 4 to 5) and 0.140 (IQR: 0.075 to 0.250), respectively. Fragility analysis of statistical significant outcomes and nonsignificant outcomes had an FI of 3.5 with no IQR and 5 (IQR 4-5), respectively. All of the studies reported a loss to follow-up data, where 20% (2) was greater than the overall FI of 5.

Conclusion: The orbital fracture literature provides treatment guidance by relying on statistical significant results from RCTs. However, the RCTs in the orbital fracture peer-reviewed literature may not be statistically stable as previously thought. The sole reliance of the P value may depict misleading results. Thus, we recommend standardizing the reporting of the P value, FI, and FQ in the orbital fracture literature to aid readers in reliably drawing conclusions based on fragility outcome measures impacting clinical decision-making.