A modular, deep learning-based holistic intent sensing system tested with Parkinson's disease patients and controls

Joseph Russell; Jemma Inches; Camille B Carroll; Jeroen H M Bergmann

doi:10.3389/fneur.2023.1260445

A modular, deep learning-based holistic intent sensing system tested with Parkinson's disease patients and controls

Front Neurol. 2023 Nov 1:14:1260445. doi: 10.3389/fneur.2023.1260445. eCollection 2023.

Authors

Joseph Russell¹, Jemma Inches², Camille B Carroll^{2

3

4}, Jeroen H M Bergmann¹

Affiliations

¹ Natural Interaction Lab, Department of Engineering Science, Institute of Biomedical Engineering, University of Oxford, Oxford, United Kingdom.
² University Hospitals Plymouth NHS Trust, Plymouth, Devon, United Kingdom.
³ Newcastle University Translational and Clinical Research Institute, Campus for Ageing and Vitality, Newcastle-upon-Tyne, United Kingdom.
⁴ Faculty of Health, University of Plymouth, Plymouth, United Kingdom.

Abstract

People living with mobility-limiting conditions such as Parkinson's disease can struggle to physically complete intended tasks. Intent-sensing technology can measure and even predict these intended tasks, such that assistive technology could help a user to safely complete them. In prior research, algorithmic systems have been proposed, developed and tested for measuring user intent through a Probabilistic Sensor Network, allowing multiple sensors to be dynamically combined in a modular fashion. A time-segmented deep-learning system has also been presented to predict intent continuously. This study combines these principles, and so proposes, develops and tests a novel algorithm for multi-modal intent sensing, combining measurements from IMU sensors with those from a microphone and interpreting the outputs using time-segmented deep learning. It is tested on a new data set consisting of a mix of non-disabled control volunteers and participants with Parkinson's disease, and used to classify three activities of daily living as quickly and accurately as possible. Results showed intent could be determined with an accuracy of 97.4% within 0.5 s of inception of the idea to act, which subsequently improved monotonically to a maximum of 99.9918% over the course of the activity. This evidence supports the conclusion that intent sensing is viable as a potential input for assistive medical devices.

Keywords: Parkinson’s disease; assistive medical devices; deep learning; intent sensing; wearable sensors.

Grants and funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This research was performed as part of JR’s DPhil, funded through an EPSRC Doctoral Training Award (Student Number 811504). The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.