Background: Mobile health (mHealth) apps can promote physical activity; however, the pragmatic nature (ie, how well research translates into real-world settings) of these studies is unknown. The impact of study design choices, for example, intervention duration, on intervention effect sizes is also understudied.
Objective: This review and meta-analysis aims to describe the pragmatic nature of recent mHealth interventions for promoting physical activity and examine the associations between study effect size and pragmatic study design choices.
Methods: The PubMed, Scopus, Web of Science, and PsycINFO databases were searched until April 2020. Studies were eligible if they incorporated apps as the primary intervention, were conducted in health promotion or preventive care settings, included a device-based physical activity outcome, and used randomized study designs. Studies were assessed using the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) and Pragmatic-Explanatory Continuum Indicator Summary-2 (PRECIS-2) frameworks. Study effect sizes were summarized using random effect models, and meta-regression was used to examine treatment effect heterogeneity by study characteristics.
Results: Overall, 3555 participants were included across 22 interventions, with sample sizes ranging from 27 to 833 (mean 161.6, SD 193.9, median 93) participants. The study populations' mean age ranged from 10.6 to 61.5 (mean 39.6, SD 6.5) years, and the proportion of males included across all studies was 42.8% (1521/3555). Additionally, intervention lengths varied from 2 weeks to 6 months (mean 60.9, SD 34.9 days). The primary app- or device-based physical activity outcome differed among interventions: most interventions (17/22, 77%) used activity monitors or fitness trackers, whereas the rest (5/22, 23%) used app-based accelerometry measures. Data reporting across the RE-AIM framework was low (5.64/31, 18%) and varied within specific dimensions (Reach=44%; Effectiveness=52%; Adoption=3%; Implementation=10%; Maintenance=12.4%). PRECIS-2 results indicated that most study designs (14/22, 63%) were equally explanatory and pragmatic, with an overall PRECIS-2 score across all interventions of 2.93/5 (SD 0.54). The most pragmatic dimension was flexibility (adherence), with an average score of 3.73 (SD 0.92), whereas follow-up, organization, and flexibility (delivery) appeared more explanatory with means of 2.18 (SD 0.75), 2.36 (SD 1.07), and 2.41 (SD 0.72), respectively. An overall positive treatment effect was observed (Cohen d=0.29, 95% CI 0.13-0.46). Meta-regression analyses revealed that more pragmatic studies (-0.81, 95% CI -1.36 to -0.25) were associated with smaller increases in physical activity. Treatment effect sizes were homogenous across study duration, participants' age and gender, and RE-AIM scores.
Conclusions: App-based mHealth physical activity studies continue to underreport several key study characteristics and have limited pragmatic use and generalizability. In addition, more pragmatic interventions observe smaller treatment effects, whereas study duration appears to be unrelated to the effect size. Future app-based studies should more comprehensively report real-world applicability, and more pragmatic approaches are needed for maximal population health impacts.
Trial registration: PROSPERO CRD42020169102; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=169102.
Keywords: PRECIS-2; Pragmatic-Explanatory Continuum Indicator Summary-2; RE-AIM; Reach, Effectiveness, Adoption, Implementation, Maintenance; digital health; mHealth; meta-analysis; mobile health; mobile phone; physical activity; systematic review.
©Chad Stecher, Bjorn Pfisterer, Samantha M Harden, Dana Epstein, Jakob M Hirschmann, Kathrin Wunsch, Matthew P Buman. Originally published in JMIR mHealth and uHealth (https://mhealth.jmir.org), 04.05.2023.