Misinterpretations of Null Hypothesis Significance Testing Results Near the P-Value Threshold in the Neurosurgical Literature

Abstract

Background: Although statistical errors have been shown to be prevalent in the medical literature, there has been a paucity of studies focusing on the prevalence of statistical errors in neurosurgery. We examined the contemporary neurosurgical literature for a common statistical mistake, namely, misinterpretations of null hypothesis significance testing results near the P-value threshold by conveying the idea of a trend.

Methods: PubMed/Medline was used to identify all articles published in 6 major neurosurgical journals between 2000 and 2020. The abstracts of these articles were extracted and scrutinized to determine when the word "trend" was used to express near statistical significance.

Results: This study included a total of 45,244 articles. Of those, 461 (1.02% [95% confidence interval, 0.86%-1.18%]) employed the word "trend" to indicate near statistical significance, for a total of 3.8 (95% confidence interval, 2.93-4.75) errors per issue per year. The error under study occurred more frequently in Journal of Neurosurgery: Spine and less frequently in Acta Neurochirurgica (P = 0.007). On an annual basis, there was no linear correlation between the total number of articles published per journal and the number of such errors in that journal (r = 0.34, P = 0.01).

Conclusions: Misinterpretations of null hypothesis significance testing results near the P-value threshold are present in at least 1% of the neurosurgical literature. While we believe that most statistical errors in medicine in general and neurosurgery in particular may be unintentional in nature, additional measures should be put in place to prevent the subsequent adoption of such undesirable methodological practices among future researchers.

Keywords: Data; Data science; Evidence based; Neurosurgery; Statistical errors; Statistics.