When should meta-analysis avoid making hidden normality assumptions?

Abstract

Meta-analysis is a widely used statistical technique. The simplicity of the calculations required when performing conventional meta-analyses belies the parametric nature of the assumptions that justify them. In particular, the normal distribution is extensively, and often implicitly, assumed. Here, we review how the normal distribution is used in meta-analysis. We discuss when the normal distribution is likely to be adequate and also when it should be avoided. We discuss alternative and more advanced methods that make less use of the normal distribution. We conclude that statistical methods that make fewer normality assumptions should be considered more often in practice. In general, statisticians and applied analysts should understand the assumptions made by their statistical analyses. They should also be able to defend these assumptions. Our hope is that this article will foster a greater appreciation of the extent to which assumptions involving the normal distribution are made in statistical methods for meta-analysis. We also hope that this article will stimulate further discussion and methodological work.

Keywords: central limit theorem; distributional assumptions; normal approximation; random effects models.