Mobility-Focused Physical Outcome Measures Over Telecommunication Technology (Zoom): Intra and Interrater Reliability Trial

Marie-Louise Bird; Felix Peel; Matt Schmidt; Natalie A Fini; Emily Ramage; Brodie M Sakakibara; Dawn B Simpson; Carey Mather; Dominique A Cadilhac; Kiran D K Ahuja; Heather Bridgman; Coralie English

doi:10.2196/38101

Mobility-Focused Physical Outcome Measures Over Telecommunication Technology (Zoom): Intra and Interrater Reliability Trial

JMIR Rehabil Assist Technol. 2022 Aug 22;9(3):e38101. doi: 10.2196/38101.

Authors

Marie-Louise Bird^{1

2}, Felix Peel², Matt Schmidt³, Natalie A Fini⁴, Emily Ramage⁵, Brodie M Sakakibara⁶, Dawn B Simpson⁷, Carey Mather², Dominique A Cadilhac^{8

9}, Kiran D K Ahuja², Heather Bridgman², Coralie English^{7

10}

Affiliations

¹ Department of Physical Therapy, University of British Columbia, Vancouver, BC, Canada.
² School of Health Sciences, College of Health and Medicine, University of Tasmania, Launceston, Australia.
³ School of Health Sciences, College of Health and Medicine, University of Tasmania, Hobart, Australia.
⁴ Physiotherapy Department, University of Melbourne, Melbourne, Australia.
⁵ School of Health Sciences, University of Newcastle, Callaghan, Australia.
⁶ Occupational Science and Occupational Therapy, University of British Columbia, Kelowna, BC, Canada.
⁷ School of Health Sciences and Priority Research Centre for Stroke and Brain Injury, University of Newcastle, Callaghan, Australia.
⁸ Translational Public Health and Evaluation Division, Stroke and Ageing Research, Department of Medicine, School of Clinical Sciences at Monash Health, Monash University, Heidelberg, Australia.
⁹ Public Health, Stroke Division, Florey Institute of Neuroscience and Mental Health, University of Melbourne, Heidelberg, Australia.
¹⁰ Heart and Stroke Research Program, Hunter Medical Research Institute, Newcastle, Australia.

PMID: 35994327
PMCID: PMC9446136
DOI: 10.2196/38101

Abstract

Background: Rehabilitation provided via telehealth offers an alternative to currently limited in-person health care. Effective rehabilitation depends on accurate and relevant assessments that reliably measure changes in function over time. The reliability of a suite of relevant assessments to measure the impact of rehabilitation on physical function is unknown.

Objective: We aimed to measure the intrarater reliability of mobility-focused physical outcome measures delivered via Zoom (a commonly used telecommunication platform) and interrater reliability, comparing Zoom with in-person measures.

Methods: In this reliability trial, healthy volunteers were recruited to complete 7 mobility-focused outcome measures in view of a laptop, under instructions from a remotely based researcher who undertook the remote evaluations. An in-person researcher (providing the benchmark scores) concurrently recorded their scores. Interrater and intrarater reliability were assessed for Grip Strength, Functional Reach Test, 5-Time Sit to Stand, 3- and 4-Meter Walks and Timed Up and Go, using intraclass correlation coefficients (ICC) and Bland-Altman plots. These tests were chosen because they cover a wide array of physical mobility, strength, and balance constructs; require little to no assistance from a clinician; can be performed in the limits of a home environment; and are likely to be feasible over a telehealth delivery mode.

Results: A total of 30 participants (mean age 36.2, SD 12.5 years; n=19, 63% male) completed all assessments. Interrater reliability was excellent for Grip Strength (ICC=0.99) and Functional Reach Test (ICC=0.99), good for 5-Time Sit to Stand (ICC=0.842) and 4-Meter Walk (ICC=0.76), moderate for Timed Up and Go (ICC=0.64), and poor for 3-Meter Walk (ICC=-0.46). Intrarater reliability, accessed by the remote researcher, was excellent for Grip Strength (ICC=0.91); good for Timed Up and Go, 3-Meter Walk, 4-Meter Walk, and Functional Reach (ICC=0.84-0.89); and moderate for 5-Time Sit to Stand (ICC=0.67). Although recorded simultaneously, the following time-based assessments were recorded as significantly longer via Zoom: 5-Time Sit to Stand (1.2 seconds), Timed Up and Go (1.0 seconds), and 3-Meter Walk (1.3 seconds).

Conclusions: Untimed mobility-focused physical outcome measures have excellent interrater reliability between in-person and telehealth measurements. Timed outcome measures took approximately 1 second longer via Zoom, reducing the reliability of tests with a shorter duration. Small time differences favoring in-person attendance are of a similar magnitude to clinically important differences, indicating assessments undertaken using telecommunications technology (Zoom) cannot be compared directly with face-to-face delivery. This has implications for clinicians using blended (ie, some face-to-face and some via the internet) assessments. High intrarater reliability of mobility-focused physical outcome measures has been demonstrated in this study.

Keywords: Zoom; assessment; consultation; mobile health; mobility; physical function; rehabilitation; reliability; telehealth; telemedicine.

©Marie-Louise Bird, Felix Peel, Matt Schmidt, Natalie A Fini, Emily Ramage, Brodie M Sakakibara, Dawn B Simpson, Carey Mather, Dominique A Cadilhac, Kiran D K Ahuja, Heather Bridgman, Coralie English. Originally published in JMIR Rehabilitation and Assistive Technology (https://rehab.jmir.org), 22.08.2022.