Purpose: RCTs are the gold standard in determining intervention efficacy with journal impact factor assumed to index research quality. Flint et al's (2017) systematic review examined neurocognitive outcomes following paediatric temporal lobe epilepsy surgery. Retrieved evidence was restricted to non-RCTs, which pose greater risk of bias and thus diminish research quality. The current study evaluated risk of bias in sources retrieved by Flint et al. and explored whether impact factor related to research quality within this selected field.
Methods: Methodological and reporting bias was evaluated using categories of bias specified by Cochrane. The relationship between the identified number of biases and journal impact factors of retrieved sources was examined.
Results: All studies carried substantial risk for bias. Methodology bias included low sample size (76.71%; 56/73), risk of confounding cognitive outcomes due to failure to report pre-surgery neurocognitive data (21.92%; 16/73) and to determine whether patients were prescribed antiepileptic drugs at follow-up (53.42%; 39/73). Reporting bias included overstating claims based on findings (53.42%; 39/73), failure to report individual patient characteristics (66%; 33/50) and omitting the nature of surgical interventions (15.07%; 11/73). The number of sources of common bias within studies was not associated significantly with journal impact factor (p = .878).
Conclusion: This evaluation highlights risk of bias when sources are predominantly uncontrolled non-RCTs and provides evidence that journal impact factor is not a reliable indicator of quality within this field. Authors should limit bias in their methods and reporting of results, to ensure the highest quality evidence possible is used to inform treatment decisions and prognosis.
Keywords: Bias; Cognitive; Pediatric epilepsy surgery; Research quality.
Copyright © 2019 Elsevier B.V. All rights reserved.