Usability of the Experience Sampling Method in Specialized Mental Health Care: Pilot Evaluation Study

Jeroen Dennis Merlijn Weermeijer; Martien Wampers; Lena de Thurah; Rafaël Bonnier; Maarten Piot; Peter Kuppens; Inez Myin-Germeys; Glenn Kiekens

doi:10.2196/48821

Usability of the Experience Sampling Method in Specialized Mental Health Care: Pilot Evaluation Study

JMIR Form Res. 2023 Nov 21:7:e48821. doi: 10.2196/48821.

Authors

Jeroen Dennis Merlijn Weermeijer¹, Martien Wampers¹, Lena de Thurah¹, Rafaël Bonnier¹, Maarten Piot², Peter Kuppens², Inez Myin-Germeys¹, Glenn Kiekens^{1

2

3

4}

Affiliations

¹ Center for Contextual Psychiatry, KU Leuven, Leuven, Belgium.
² Quantitative Psychology and Individual Differences, KU Leuven, Leuven, Belgium.
³ Research Unit of Clinical Psychology, KU Leuven, Leuven, Belgium.
⁴ Department of Medical and Clinical Psychology, Tilburg University, Tilburg, Netherlands.

PMID: 37988137
PMCID: PMC10698657
DOI: 10.2196/48821

Abstract

Background: Mental health problems occur in interactions in daily life. Yet, it is challenging to bring contextual information into the therapy room. The experience sampling method (ESM) may facilitate this by assessing clients' thoughts, feelings, symptoms, and behaviors as they are experienced in everyday life. However, the ESM is still primarily used in research settings, with little uptake in clinical practice. One aspect that may facilitate clinical implementation concerns the use of ESM protocols, which involves providing practitioners with ready-to-use ESM questionnaires, sampling schemes, visualizations, and training.

Objective: This pilot study's objective was to evaluate the usability of an ESM protocol for using the ESM in a specialized mental health care setting.

Methods: We created the ESM protocol using the m-Path software platform and tested its usability in clinical practice. The ESM protocol consists of a dashboard for practitioners (ie, including the setup of the template and data visualizations) and an app for clients (ie, for completing the ESM questionnaires). A total of 8 practitioners and 17 clients used the ESM in practice between December 1, 2020, and July 31, 2021. Usability was assessed using questionnaires, ESM compliance rates, and semistructured interviews.

Results: The usability was overall rated reasonable to good by practitioners (mean scores of usability items ranging from 5.33, SD 0.91, to 6.06, SD 0.73, on a scale ranging from 1 to 7). However, practitioners expressed difficulty in personalizing the template and reported insufficient guidelines on how to use the ESM in clinical practice. On average, clients completed 55% (SD 25%) of the ESM questionnaires. They rated the usability as reasonable to good, but their scores were slightly lower and more variable than those of the practitioners (mean scores of usability items ranging from 4.18, SD 1.70, to 5.94, SD 1.50 on a scale ranging from 1 to 7). Clients also voiced several concerns over the piloted ESM template, with some indicating no interest in the continued use of the ESM.

Conclusions: The findings suggest that using an ESM protocol may facilitate the implementation of the ESM as a mobile health assessment tool in psychiatry. However, additional adaptions should be made before further implementation. Adaptions include providing training on personalizing questionnaires, adding additional sampling scheme formats as well as an open-text field, and creating a dynamic data visualization interface. Future studies should also identify factors determining the suitability of the ESM for specific treatment goals among different client populations.

Keywords: digital mental health; ecological momentary assessment; experience sampling; implementation; mobile phone.

©Jeroen Dennis Merlijn Weermeijer, Martien Wampers, Lena de Thurah, Rafaël Bonnier, Maarten Piot, Peter Kuppens, Inez Myin-Germeys, Glenn Kiekens. Originally published in JMIR Formative Research (https://formative.jmir.org), 21.11.2023.