Cross-Modal Transfer Learning From EEG to Functional Near-Infrared Spectroscopy for Classification Task in Brain-Computer Interface System

Yuqing Wang; Zhiqiang Yang; Hongfei Ji; Jie Li; Lingyu Liu; Jie Zhuang

doi:10.3389/fpsyg.2022.833007

Cross-Modal Transfer Learning From EEG to Functional Near-Infrared Spectroscopy for Classification Task in Brain-Computer Interface System

Front Psychol. 2022 Apr 7:13:833007. doi: 10.3389/fpsyg.2022.833007. eCollection 2022.

Authors

Yuqing Wang¹, Zhiqiang Yang¹, Hongfei Ji¹, Jie Li¹, Lingyu Liu², Jie Zhuang³

Affiliations

¹ Shanghai Yangzhi Rehabilitation Hospital (Shanghai Sunshine Rehabilitation Center), Collage of Electronic and Information Engineering, Tongji University, Shanghai, China.
² Department of Neurorehabilitation, Shanghai Yangzhi Rehabilitation Hospital (Shanghai Sunshine Rehabilitation Center), School of Medicine, Tongji University, Shanghai, China.
³ School of Psychology, Shanghai University of Sport, Shanghai, China.

Abstract

The brain-computer interface (BCI) based on functional near-infrared spectroscopy (fNIRS) has received more and more attention due to its vast application potential in emotion recognition. However, the relatively insufficient investigation of the feature extraction algorithms limits its use in practice. In this article, to improve the performance of fNIRS-based BCI, we proposed a method named R-CSP-E, which introduces EEG signals when computing fNIRS signals' features based on transfer learning and ensemble learning theory. In detail, we used the Independent Component Analysis (ICA) algorithm for the correspondence between the sources of the two signals. We then introduced the EEG signals when computing the spatial filter based on a modified Common Spatial Pattern (CSP) algorithm. Experimental results on public datasets show that the proposed method in this paper outperforms traditional methods without transfer. In general, the mean classification accuracy can be increased by up to 5%. To our knowledge, it is an innovation that we tried to apply transfer learning between EEG and fNIRS. Our study's findings not only prove the potential of the transfer learning algorithm in cross-model brain-computer interface, but also offer a new and innovative perspective to research the hybrid brain-computer interface.

Keywords: ICA; RCSP; brain computer interface; fNIRS signals; transfer learning.