Objective: Randomized controlled trials (RCTs) have come under scrutiny due to a frequent lack of reproducibility, due in part to shortcomings of the common P < 0.05 threshold for significance. Here, we utilize fragility indices to assess the statistical robustness of RCTs evaluating low-dose ketamine during scoliosis surgery to reduce opioid tolerance and postoperative pain.
Methods: RCTs evaluating outcomes after intraoperative ketamine infusion in adolescent idiopathic scoliosis patients were included. Relevant outcomes included pain, opioid consumption, quality of life, anesthesia, sedation, adverse effects, and length of stay. The dichotomous fragility index or continuous fragility index (FI or CFI) was determined by manipulating each outcome event until reversal of significance (a = 0.05) was achieved. The corresponding fragility quotients were calculated by dividing the FI or CFI by the sample size.
Results: Of 27 studies screened, 6 studies (61 outcome events) were included. The median FI for dichotomous events was 2.0 (fragility quotient = 0.045), suggesting that altering the outcome of only 2 patients (or 4.5 out of 100) would reverse trial significance. For continuous events, altering the treatment of only 6 patients (or 14.1 out of 100) would reverse significance. Outcome events that were originally reported as significant (P < 0.05) were considerably more fragile (FI = 1.5; CFI = 3.5) than events that were reported as nonsignificant (FI = 2.0; CFI = 7.0).
Conclusions: While evidence for ketamine use is promising, our fragility analysis suggests that RCT findings may be underpowered in some cases. Given the importance of RCTs in clinical decision-making, fragility indices should be reported alongside P values to indicate the strength of statistical findings.
Keywords: Fragility Index (FI); Idiopathic scoliosis; Ketamine; Opioid use; P values; Postoperative pain; Randomized controlled trials (RCTs); Statistical fragility.
Copyright © 2022 Elsevier Inc. All rights reserved.