Comparing analytical strategies for balancing site-level characteristics in stepped-wedge cluster randomized trials: a simulation study

Clement Ma; Alina Lee; Darren Courtney; David Castle; Wei Wang

doi:10.1186/s12874-023-02027-y

Comparing analytical strategies for balancing site-level characteristics in stepped-wedge cluster randomized trials: a simulation study

BMC Med Res Methodol. 2023 Sep 12;23(1):206. doi: 10.1186/s12874-023-02027-y.

Authors

Clement Ma^{1

2

3}, Alina Lee^{1

2}, Darren Courtney^{4

5}, David Castle^{6

7}, Wei Wang^{8

9

10}

Affiliations

¹ Biostatistics Core, Centre for Addiction and Mental Health, Toronto, ON, Canada.
² Center for Complex Interventions, Centre for Addiction and Mental Health, Toronto, ON, Canada.
³ Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada.
⁴ Cundill Centre for Child and Youth Depression, Centre for Addiction and Mental Health, Toronto, ON, Canada.
⁵ Department of Psychiatry, University of Toronto, Toronto, ON, Canada.
⁶ Department of Psychiatry, University of Tasmania, Hobart, TAS, Australia.
⁷ Centre for Mental Health Service Innovation, Statewide Mental Health Service, Hobart, TAS, Australia.
⁸ Biostatistics Core, Centre for Addiction and Mental Health, Toronto, ON, Canada. wei.wang@camh.ca.
⁹ Center for Complex Interventions, Centre for Addiction and Mental Health, Toronto, ON, Canada. wei.wang@camh.ca.
¹⁰ College of Public Health, University of South Florida, Tampa, FL, USA. wei.wang@camh.ca.

Abstract

Background: Stepped-wedge cluster randomized trials (SWCRTs) are a type of cluster-randomized trial in which clusters are randomized to cross-over to the active intervention sequentially at regular intervals during the study period. For SWCRTs, sequential imbalances of cluster-level characteristics across the random sequence of clusters may lead to biased estimation. Our study aims to examine the effects of balancing cluster-level characteristics in SWCRTs.

Methods: To quantify the level of cluster-level imbalance, a novel imbalance index was developed based on the Spearman correlation and rank regression of the cluster-level characteristic with the cross-over timepoints. A simulation study was conducted to assess the impact of sequential cluster-level imbalances across different scenarios varying the: number of sites (clusters), sample size, number of cross-over timepoints, site-level intra-cluster correlation coefficient (ICC), and effect sizes. SWCRTs assumed either an immediate "constant" treatment effect, or a gradual "learning" treatment effect which increases over time after crossing over to the active intervention. Key performance metrics included the relative root mean square error (RRMSE) and relative mean bias.

Results: Fully-balanced designs almost always had the highest efficiency, as measured by the RRMSE, regardless of the number of sites, ICC, effect size, or sample sizes at each time for SWCRTs with learning effect. A consistent decreasing trend of efficiency was observed by increasing RRMSE as imbalance increased. For example, for a 12-site study with 20 participants per site/timepoint and ICC of 0.10, between the most balanced and least balanced designs, the RRMSE efficiency loss ranged from 52.5% to 191.9%. In addition, the RRMSE was decreased for larger sample sizes, larger number of sites, smaller ICC, and larger effect sizes. The impact of pre-balancing diminished when there was no learning effect.

Conclusion: The impact of pre-balancing on preventing efficiency loss was easily observed when there was a learning effect. This suggests benefit of pre-balancing with respect to impacting factors of treatment effects.

Keywords: Cluster-level imbalance; Simulation; Stepped-wedge cluster randomized trials.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Benchmarking*
Computer Simulation
Humans
Randomized Controlled Trials as Topic
Sample Size

Grants and funding

CIHR/Canada