The fragility of statistical significance in distal femur fractures: systematic review of randomized controlled trials

Abstract

Purpose: The purpose of this study was to apply both the fragility index (FI) and fragility quotient (FQ) to evaluate the degree of statistical fragility in the distal femur fracture (DFF) literature. We hypothesized that the dichotomous outcomes within the DFF literature are statistically fragile.

Methods: Using preferred reporting items for systematic reviews and meta-analyses, we performed a PubMed search for distal femur fractures clinical trials from 2000 to 2022 reporting dichotomous outcomes. The FI of each outcome was calculated through the reversal of a single outcome event until significance was reversed. The FQ was calculated by dividing each fragility index by study sample size. The interquartile range (IQR) was also calculated for the FI and FQ.

Results: Of the 4258 articles screened, 92 met the search criteria, with eleven RCTs included for analysis. Ninety eight outcome events with 25 significant (P < 0.05) outcomes and 73 nonsignificant (P > 0.05) outcomes were identified. The overall FI and FQ for all 98 outcomes were 5 (IQR 4-6) and 0.130 (IQR 0.087-0.174), respectively. Three studies (33.3%) reported loss to follow (LTF) greater than 5.

Conclusions: The randomized controlled trials in the peer-reviewed distal femur fracture literature may not be as robust as previously thought, as incorporating statistical analyses solely on a P value threshold is misleading. Standardized reporting of the P value, FI and FQ can help the clinician reliably draw conclusions based on the fragility of outcome measures.

Keywords: Distal femur fracture; Fragility index; Fragility quotient; P value; Statistical fragility.