Proposal of a Wearable Multimodal Sensing-Based Serious Games Approach for Hand Movement Training After Stroke

Xinyu Song; Shirdi Shankara van de Ven; Shugeng Chen; Peiqi Kang; Qinghua Gao; Jie Jia; Peter B Shull

doi:10.3389/fphys.2022.811950

Proposal of a Wearable Multimodal Sensing-Based Serious Games Approach for Hand Movement Training After Stroke

Front Physiol. 2022 Jun 3:13:811950. doi: 10.3389/fphys.2022.811950. eCollection 2022.

Authors

Xinyu Song¹, Shirdi Shankara van de Ven¹, Shugeng Chen², Peiqi Kang¹, Qinghua Gao¹, Jie Jia², Peter B Shull¹

Affiliations

¹ The State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, China.
² The Department of Rehabilitation Medicine, The National Clinical Research Center for Aging and Medicine, Huashan Hospital, Fudan University, Shanghai, China.

Abstract

Stroke often leads to hand motor dysfunction, and effective rehabilitation requires keeping patients engaged and motivated. Among the existing automated rehabilitation approaches, data glove-based systems are not easy to wear for patients due to spasticity, and single sensor-based approaches generally provided prohibitively limited information. We thus propose a wearable multimodal serious games approach for hand movement training after stroke. A force myography (FMG), electromyography (EMG), and inertial measurement unit (IMU)-based multi-sensor fusion model was proposed for hand movement classification, which was worn on the user's affected arm. Two movement recognition-based serious games were developed for hand movement and cognition training. Ten stroke patients with mild to moderate motor impairments (Brunnstrom Stage for Hand II-VI) performed experiments while playing interactive serious games requiring 12 activities-of-daily-living (ADLs) hand movements taken from the Fugl Meyer Assessment. Feasibility was evaluated by movement classification accuracy and qualitative patient questionnaires. The offline classification accuracy using combined FMG-EMG-IMU was 81.0% for the 12 movements, which was significantly higher than any single sensing modality; only EMG, only FMG, and only IMU were 69.6, 63.2, and 47.8%, respectively. Patients reported that they were more enthusiastic about hand movement training while playing the serious games as compared to conventional methods and strongly agreed that they subjectively felt that the proposed training could be beneficial for improving upper limb motor function. These results showed that multimodal-sensor fusion improved hand gesture classification accuracy for stroke patients and demonstrated the potential of this proposed approach to be used as upper limb movement training after stroke.

Keywords: EMG; FMG; IMU; movement training; stroke.