Site differences in implementation trial outcomes are common but often not examined. In a Hybrid Type 1 trial examining the effectiveness-implementation of a peer-led group life-style balance (PGLB) intervention for people with serious mental illness (SMI) in three supportive housing agencies, we found that PGLB recipients' physical health outcomes differed by study sites. The matrixed multiple case study methodology was used to explore how implementation outcomes and changes in context of usual care (UC) services contributed to these site differences. Two implementation outcomes (i.e. PGLB fidelity ratings and intervention recipients' acceptability of PGLB and UC) and changes in healthcare services integration at the study sites were examined. ANOVAs were used to examine site differences in fidelity ratings and client satisfaction. Directed content analysis was used to analyze leadership interviews to identify changes in the context of UC services. Site 3 showed a trend approaching significance (P = .05) towards higher fidelity ratings. High levels of satisfaction with PGLB were reported at all sites. Significant differences in PGLB recipients' satisfaction with UC were found, with Site 3 reporting the lowest levels of satisfaction. Agency leaders reported an increase in prioritizing client's health throughout the trial with sites differing in how these priorities were put into action. Differences in PGLB recipients' satisfaction with UC, and changes in healthcare service integration seemed to have contributed to the site differences in our trial. The matrixed multiple case study methodology is a useful approach to identify implementation outcomes contributing to the heterogeneity of multisite implementation trial results.
Keywords: healthy lifestyle interventions; implementation; mixed methods; serious mental illness; supportive housing.
A healthy lifestyle intervention delivered by people with lived experience of serious mental illness (SMI) led to improved health outcomes among participants living with SMI in three supportive housing agencies, but these outcomes differed by study sites. This study aimed to identify which factors influenced these site differences in study outcomes by examining if the intervention was delivered as intended, intervention recipients’ satisfaction with the intervention and the pre-existing health care services delivered at each study site, and what changes occurred in health care services offered at each of the supportive housing agencies throughout the study. The intervention was found to have high levels of satisfaction across all three agencies but the agency which had the best outcomes also had the highest levels of intervention delivery adherence and the lowest levels of intervention recipients’ satisfaction with existing health care services. Additionally, during the study period, the same agency described minimal changes in their existing health care services while the other two agencies were found to have made more significant changes integrating health care services to their day-to-day operations. As a result, findings from this study emphasize the importance of understanding the context in which interventions are delivered in routine practice settings to ensure their overall success.
Published by Oxford University Press on behalf of the Society of Behavioral Medicine 2023.