Background: With the proliferation of digital mental health interventions (DMHIs) guided by relational agents, little is known about the behavioral, cognitive, and affective engagement components associated with symptom improvement over time. Obtaining a better understanding could lend clues about recommended use for particular subgroups of the population, the potency of different intervention components, and the mechanisms underlying the intervention's success.
Objective: This exploratory study applied clustering techniques to a range of engagement indicators, which were mapped to the intervention's active components and the connect, attend, participate, and enact (CAPE) model, to examine the prevalence and characterization of each identified cluster among users of a relational agent-guided DMHI.
Methods: We invited adults aged 18 years or older who were interested in using digital support to help with mood management or stress reduction through social media to participate in an 8-week DMHI guided by a natural language processing-supported relational agent, Woebot. Users completed assessments of affective and cognitive engagement, working alliance as measured by goal and task working alliance subscale scores, and enactment (ie, application of therapeutic recommendations in real-world settings). The app passively collected data on behavioral engagement (ie, utilization). We applied agglomerative hierarchical clustering analysis to the engagement indicators to identify the number of clusters that provided the best fit to the data collected, characterized the clusters, and then examined associations with baseline demographic and clinical characteristics as well as mental health outcomes at week 8.
Results: Exploratory analyses (n=202) supported 3 clusters: (1) "typical utilizers" (n=81, 40%), who had intermediate levels of behavioral engagement; (2) "early utilizers" (n=58, 29%), who had the nominally highest levels of behavioral engagement in week 1; and (3) "efficient engagers" (n=63, 31%), who had significantly higher levels of affective and cognitive engagement but the lowest level of behavioral engagement. With respect to mental health baseline and outcome measures, efficient engagers had significantly higher levels of baseline resilience (P<.001) and greater declines in depressive symptoms (P=.01) and stress (P=.01) from baseline to week 8 compared to typical utilizers. Significant differences across clusters were found by age, gender identity, race and ethnicity, sexual orientation, education, and insurance coverage. The main analytic findings remained robust in sensitivity analyses.
Conclusions: There were 3 distinct engagement clusters found, each with distinct baseline demographic and clinical traits and mental health outcomes. Additional research is needed to inform fine-grained recommendations regarding optimal engagement and to determine the best sequence of particular intervention components with known potency. The findings represent an important first step in disentangling the complex interplay between different affective, cognitive, and behavioral engagement indicators and outcomes associated with use of a DMHI incorporating a natural language processing-supported relational agent.
Trial registration: ClinicalTrials.gov NCT05672745; https://classic.clinicaltrials.gov/ct2/show/NCT05672745.
Keywords: anxiety; clustering; depression; digital health; digital mental health intervention; mental health; relational agents; user engagement.
©Valerie Hoffman, Megan Flom, Timothy Y Mariano, Emil Chiauzzi, Andre Williams, Andrew Kirvin-Quamme, Sarah Pajarito, Emily Durden, Olga Perski. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 13.10.2023.