Statistical power of clinical trials increased while effect size remained stable: an empirical analysis of 136,212 clinical trials between 1975 and 2014

Herm J Lamberink; Willem M Otte; Michel R T Sinke; Daniël Lakens; Paul P Glasziou; Joeri K Tijdink; Christiaan H Vinkers

doi:10.1016/j.jclinepi.2018.06.014

Statistical power of clinical trials increased while effect size remained stable: an empirical analysis of 136,212 clinical trials between 1975 and 2014

J Clin Epidemiol. 2018 Oct:102:123-128. doi: 10.1016/j.jclinepi.2018.06.014. Epub 2018 Jul 5.

Authors

Herm J Lamberink¹, Willem M Otte², Michel R T Sinke³, Daniël Lakens⁴, Paul P Glasziou⁵, Joeri K Tijdink⁶, Christiaan H Vinkers⁷

Affiliations

¹ Department of Child Neurology, Brain Center Rudolf Magnus, University Medical Center Utrecht and Utrecht University, P.O. Box 85090, Utrecht 3508 AB, The Netherlands. Electronic address: h.j.lamberink@umcutrecht.nl.
² Department of Child Neurology, Brain Center Rudolf Magnus, University Medical Center Utrecht and Utrecht University, P.O. Box 85090, Utrecht 3508 AB, The Netherlands; Biomedical MR Imaging and Spectroscopy group, Center for Image Sciences, University Medical Center Utrecht and Utrecht University, Heidelberglaan 100, Utrecht 3584 CX, The Netherlands.
³ Biomedical MR Imaging and Spectroscopy group, Center for Image Sciences, University Medical Center Utrecht and Utrecht University, Heidelberglaan 100, Utrecht 3584 CX, The Netherlands.
⁴ School of Innovation Sciences, Eindhoven University of Technology, Den Dolech 1, Eindhoven 5600 MB, The Netherlands.
⁵ Centre for Research in Evidence-Based Practice, Faculty of Health Sciences and Medicine, Bond University, Gold Coast, Queensland, Australia.
⁶ Department of Philosophy, VU University, De Boelelaan 1105, Amsterdam 1081 HV, The Netherlands.
⁷ Department of Psychiatry, Brain Center Rudolf Magnus, University Medical Center Utrecht and Utrecht University, Heidelberglaan 100, Utrecht 3584 CX, The Netherlands.

PMID: 29981870
DOI: 10.1016/j.jclinepi.2018.06.014

Abstract

Objectives: To study the statistical power of randomized clinical trials and examine developments over time.

Study design and setting: We analyzed the statistical power in 136,212 clinical trials between 1975 and 2014 extracted from meta-analyses from the Cochrane database of systematic reviews. We determined study power to detect standardized effect sizes, where power was based on the meta-analyzed effect size. Average power, effect size, and temporal patterns were examined for all meta-analyses and a subset of significant meta-analyses.

Results: The number of trials with power ≥80% was low (7%) but increased over time: from 5% in 1975-1979 to 9% in 2010-2014. In significant meta-analyses, the proportion of trials with sufficient power increased from 9% to 15% in these years (median power increased from 16% to 23%). This increase was mainly due to increasing sample sizes, while effect sizes remained stable with a median Cohen's h of 0.09 (interquartile range 0.04-0.22) and a median Cohen's d of 0.20 (0.11-0.40).

Conclusion: This study demonstrates that sufficient power in clinical trials is still problematic, although the situation is slowly improving. Our data encourage further efforts to increase statistical power in clinical trials to guarantee rigorous and reproducible evidence-based medicine.

Keywords: Clinical trial; Randomized; Statistical power.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Data Interpretation, Statistical
Evidence-Based Medicine
Humans
Randomized Controlled Trials as Topic*
Research Design / trends*
Sample Size