Rounding, but not randomization method, non-normality, or correlation, affected baseline P-value distributions in randomized trials

Abstract

Objectives: To investigate whether comparing observed with expected P-value distributions for baseline continuous variables in randomized controlled trials (RCTs) might be limited by randomization methods, normality and correlation of variables, or calculation of P-values from rounded summary statistics.

Study design and setting: We assessed how each factor affects differences from expected for P-value distributions and area under the curve of the cumulative distribution function (AUC-CDF) of baseline P-values in 13 RCTs and in simulations.

Results: The P-value distributions and AUC-CDF for variables with possible non-normal distribution and in simulations using eight different randomization methods were consistent with the theoretical uniform distribution and AUC-CDF, respectively, although stratification and minimization produced smaller-than-expected proportions of P-values <0.10. Seventy-seven percentage of 3,813 pairwise correlations between baseline variables in the 13 individual RCTs were between -0.2 and 0.2. P-value distribution and AUC-CDF remained consistent with the uniform distribution in simulations with incrementally increasing correlation strength. The P-value distributions calculated from rounded summary statistics were not uniform, but expected distributions could be empirically generated.

Conclusions: Randomization methods, non-normality, and strength of correlation of baseline variables did not have important effects on baseline P-value distribution or AUC-CDF, but baseline P-values calculated from rounded summary statistics are non-uniformly distributed.

Keywords: Correlation; P-values; Randomisation; Research integrity; Rounding; Statistical methods.