Measuring balance and model selection in propensity score methods

Abstract

Purpose: Propensity score (PS) methods focus on balancing confounders between groups to estimate an unbiased treatment or exposure effect. However, there is lack of attention in actually measuring, reporting and using the information on balance, for instance for model selection. We propose to use a measure for balance in PS methods and describe several of such measures: the overlapping coefficient, the Kolmogorov-Smirnov distance, and the Lévy distance.

Methods: We performed simulation studies to estimate the association between these three and several mean based measures for balance and bias (i.e., discrepancy between the true and the estimated treatment effect).

Results: For large sample sizes (n = 2000) the average Pearson's correlation coefficients between bias and Kolmogorov-Smirnov distance (r = 0.89), the Lévy distance (r = 0.89) and the absolute standardized mean difference (r = 0.90) were similar, whereas this was lower for the overlapping coefficient (r = -0.42). When sample size decreased to 400, mean based measures of balance had stronger correlations with bias. Models including all confounding variables, their squares and interaction terms resulted in smaller bias than models that included only main terms for confounding variables.

Conclusions: We conclude that measures for balance are useful for reporting the amount of balance reached in propensity score analysis and can be helpful in selecting the final PS model.