Reliability and accuracy of telemedicine-based shoulder examinations

Abstract

Background: Telemedicine has become a critical component in the evaluation and management of patients with shoulder pathology. However, the interobserver reliability of findings on virtual physical examination relative to in-person evaluation is unclear. The purpose of this study was to determine the reliability of prerecorded video telemedicine for the evaluation of shoulder pathology compared with traditional in-person physical examination.

Methods: New patients with unilateral shoulder pain presenting to a single-surgeon shoulder clinic were recruited between July and November 2020. In 1 visit, patients were evaluated with standardized in-person and video-guided telemedicine physical examinations in randomized order. Patients were evaluated for range of motion (ROM) and symptoms including pain, weakness, and apprehension. The telemedicine examination was recorded and consisted of a video guide displaying self-directed shoulder examination maneuvers that patients performed during remote coaching by an independent non-physician observer. The in-person physical examination was performed by the treating physician. The telemedicine videos were evaluated by two independent observers for interobserver reliability. The treating physician subsequently evaluated the telemedicine videos after a minimum two-month washout period for intraobserver reliability and intra-platform reliability. Interobserver and intraobserver reliability analyses were conducted using Kuder-Richardson formula 20 (KR-20). Specificity and likelihood ratios were calculated with P < .05 representing statistical significance.

Results: A total of 32 patients (17 male and 15 female patients; average age, 50.2 ± 16.2 years) were included in the analysis. Overall Kuder-Richardson formula 20 (KR-20) reliability across 40 physical examination maneuvers was 0.391 ± 0.332 (76.4% ± 15.4% agreement) between the in-person and telemedicine examinations. Telemedicine maneuvers examining ROM limitations had the highest degree of reliability, sensitivity, specificity, and likelihood of also producing a positive finding on the in-person examination (0.700 ± 0.114, 66.5%, 81.0%, and 6.06, respectively). Telemedicine maneuvers identifying apprehension associated with glenohumeral instability were found to have the lowest reliability, sensitivity, and likelihood of producing a positive finding on the in-person examination (0.170 ± 0.440, 23.5%, and 0.518, respectively). All patients were satisfied with their telemedicine experience.

Conclusion: The overall reliability of a non-physician-directed video-guided telemedicine examination ranged from unacceptable to good. Shoulder ROM limitations identified during the telemedicine examinations were found to be the most reliable, whereas evaluations of instability were found to be the least reliable. Although initial telemedicine evaluation by a non-physician may be appropriate for ROM evaluation, in-person physician evaluation is recommended to confirm suspected diagnoses, especially if clinical concern for shoulder instability exists. Alternative methods of telemedicine delivery should be explored to improve the reliability of self-directed physical examination maneuvers.

Keywords: Shoulder; reliability; telemed; telemedicine; televisit; virtual examination.