The Effect of Training on Participant Adherence With a Reporting Time Frame for Momentary Subjective Experiences in Ecological Momentary Assessment: Cognitive Interview Study

Cheng K Fred Wen; Doerte U Junghaenel; David B Newman; Stefan Schneider; Marilyn Mendez; Sarah E Goldstein; Sarah Velasco; Joshua M Smyth; Arthur A Stone

doi:10.2196/28007

The Effect of Training on Participant Adherence With a Reporting Time Frame for Momentary Subjective Experiences in Ecological Momentary Assessment: Cognitive Interview Study

JMIR Form Res. 2021 May 26;5(5):e28007. doi: 10.2196/28007.

Authors

Affiliations

¹ Dornsife Center for Self-Report Science, University of Southern California, Los Angeles, CA, United States.
² Department of Psychiatry, University of California, San Francisco, San Francisco, CA, United States.
³ Department of Biobehavioral Health, Pennsylvania State University, University Park, PA, United States.
⁴ Department of Psychology, University of Southern California, Los Angeles, CA, United States.

^# Contributed equally.

PMID: 34037524
PMCID: PMC8190649
DOI: 10.2196/28007

Abstract

Background: Ecological momentary assessment (EMA) has the potential to minimize recall bias by having people report on their experiences in the moment (momentary model) or over short periods (coverage model). This potential hinges on the assumption that participants provide their ratings based on the reporting time frame instructions prescribed in the EMA items. However, it is unclear what time frames participants actually use when answering the EMA questions and whether participant training improves participants' adherence to the reporting instructions.

Objective: This study aims to investigate the reporting time frames participants used when answering EMA questions and whether participant training improves participants' adherence to the EMA reporting timeframe instructions.

Methods: Telephone-based cognitive interviews were used to investigate the research questions. In a 2×2 factorial design, participants (n=100) were assigned to receive either basic or enhanced EMA training and randomized to rate their experiences using a momentary (at the moment you were called) or a coverage (since the last phone call) model. Participants received five calls over the course of a day to provide ratings; after each rating, participants were immediately interviewed about the time frame they used to answer the EMA questions. A total of 2 raters independently coded the momentary interview responses into time frame categories (Cohen κ=0.64, 95% CI 0.55-0.73).

Results: The results from the momentary conditions showed that most of the calls referred to the period during the call (57/199, 28.6%) or just before the call (98/199, 49.2%) to provide ratings; the remainder were from longer reporting periods. Multinomial logistic regression results indicated a significant training effect (χ²₁=16.6; P<.001) in which the enhanced training condition yielded more reports within the intended reporting time frames for momentary EMA reports. Cognitive interview data from the coverage model did not lend themselves to reliable coding and were not analyzed.

Conclusions: The results of this study provide the first evidence about adherence to EMA instructions to reporting periods and that enhanced participant training improves adherence to the time frame specified in momentary EMA studies.

Keywords: EMA; cognitive interview; ecological momentary assessment; participant training; reporting period.

©Cheng K Fred Wen, Doerte U Junghaenel, David B Newman, Stefan Schneider, Marilyn Mendez, Sarah E Goldstein, Sarah Velasco, Joshua M Smyth, Arthur A Stone. Originally published in JMIR Formative Research (https://formative.jmir.org), 26.05.2021.