A multi-modal brain-computer interface based on threshold discrimination and its application in wheelchair control

Enzeng Dong; Haoran Zhang; Lin Zhu; Shengzhi Du; Jigang Tong

doi:10.1007/s11571-021-09779-7

A multi-modal brain-computer interface based on threshold discrimination and its application in wheelchair control

Cogn Neurodyn. 2022 Oct;16(5):1123-1133. doi: 10.1007/s11571-021-09779-7. Epub 2022 Jan 24.

Authors

Enzeng Dong¹, Haoran Zhang¹, Lin Zhu², Shengzhi Du³, Jigang Tong¹

Affiliations

¹ Tianjin Key Laboratory of Control Theory and Applications in Complicated Systems, Tianjin University of Technology, Tianjin, 300384 China.
² China North Industries Group 210 Research Institute, Beijing, China.
³ Department of Electrical Engineering, Tshwane University of Technology, Pretoria, 0001 South Africa.

Abstract

In this study, we propose a novel multi-modal brain-computer interface (BCI) system based on the threshold discrimination, which is proposed for the first time to distinguish between SSVEP and MI potentials. The system combines these two heterogeneous signals to increase the number of control commands and improve the performance of asynchronous control of external devices. In this research, an electric wheelchair is controlled as an example. The user can continuously control the wheelchair to turn left/right through motion imagination (MI) by imagining left/right-hand movement and generate another 6 commands for the wheelchair control by focusing on the SSVEP stimulation panel. Ten subjects participated in a MI training session and eight of them completed a mobile obstacle-avoidance experiment in a complex environment requesting high control accuracy for successful manipulation. Comparing with the single-modal BCI-controlled wheelchair system, the results demonstrate that the proposed multi-modal method is effective by providing more satisfactory control accuracy, and show the potential of BCI-controlled systems to be applied in complex daily tasks.

Keywords: BCI controlled wheelchair; Brain–computer interface (BCI); Motor imagination (MI); Multi-modal EEG signals; Steady-state visual evoked potential (SSVEP); Threshold discrimination; Threshold strategy.