Variability in meta-analysis estimates of continuous outcomes using different standardization and scale-specific re-expression methods

Abstract

Objectives: To explore the impact of using different data standardization and scale-specific re-expression methods (i.e., processes to convert standardized data into scale-specific units) in meta-analyses using standardized mean differences (SMDs).

Study design and setting: We used data assessed by the Short Physical Performance Battery and the Barthel Index from a meta-analysis of randomized controlled trials which synthesized evidence of physical activity effectiveness on the functional capacity of hospitalized older adults. We standardized the data using study-specific pooled standard deviations (SDs), an internal, and an external SD references. Bayesian meta-analyses were performed for each method to compare the posterior distributions of the meta-analysis parameters. Posterior estimates were re-expressed into scale-specific units applying different methods established in the Cochrane guidelines.

Results: Meta-analysis estimates depend on the used standardization method. Analyses including data standardized using the largest SD reference presented lower estimates with less uncertainty in both scales. The method applied for re-expressing SMDs into scale-specific units impacted in their posterior clinical interpretation. The most similar results across models were obtained when using the same SD reference to standardize and re-express data.

Conclusion: Different data standardization methods yielded different meta-analysis estimates on the SMD scale. To avoid the introduction of bias, the use of a single scale-specific SD reference to standardize data is recommended and instead of study-specific pooled sample SDs. Meta-analysis software packages may therefore change their default methods to allow this method by a single scale-specific SD. To re-express the SMDs into scale-specific units, we suggest the application of the same SD reference that was used for data standardization.

Keywords: Clinical interpretation; Effect size; Evidence synthesis; Meta-analysis; Standardization; Standardized mean difference.