A novel deep learning model based on the ICA and Riemannian manifold for EEG-based emotion recognition

Minchao Wu; Shiang Hu; Bing Wei; Zhao Lv

doi:10.1016/j.jneumeth.2022.109642

A novel deep learning model based on the ICA and Riemannian manifold for EEG-based emotion recognition

J Neurosci Methods. 2022 Aug 1:378:109642. doi: 10.1016/j.jneumeth.2022.109642. Epub 2022 Jun 8.

Authors

Minchao Wu¹, Shiang Hu², Bing Wei³, Zhao Lv⁴

Affiliations

¹ Anhui Province Key Laboratory of Multimodal Cognitive Computation, School of Computer Science and Technology, Anhui University, Hefei 230601, China; Zhejiang Key Laboratory for Brain-Machine Collaborative Intelligence, Hangzhou Dianzi University, Hangzhou 310018, China.
² Anhui Province Key Laboratory of Multimodal Cognitive Computation, School of Computer Science and Technology, Anhui University, Hefei 230601, China.
³ Anhui Province Key Laboratory of Multimodal Cognitive Computation, School of Computer Science and Technology, Anhui University, Hefei 230601, China; School of Computer Science, Hefei Normal University, Hefei 230601, China.
⁴ Anhui Province Key Laboratory of Multimodal Cognitive Computation, School of Computer Science and Technology, Anhui University, Hefei 230601, China; Institute of Physical Science and Information Technology, Anhui University, Hefei 230601, China. Electronic address: kjlz@ahu.edu.cn.

PMID: 35690333
DOI: 10.1016/j.jneumeth.2022.109642

Abstract

Background: The EEG-based emotion recognition is one of the primary research orientations in the field of emotional intelligence and human-computer interaction (HCI).

New method: We proposed a novel model, denoted as ICRM-LSTM, for EEG-based emotion recognition by combining the independent component analysis (ICA), the Riemannian manifold (RM), and the long short-term memory network (LSTM). The SEED and MAHNOB-HCI dataset were employed to verify the performance of the proposed model. Firstly, ICA was used to perform blind source separation (BSS) for the preprocessed EEG signals provided by the two datasets. Then, a series of the covariance matrices according to time order were extracted from the blind source signals, and the symmetric positive definiteness of covariance matrix allowed us to project them from RM space to Euclid space by logarithmic mapping. Finally, the transformed covariance matrices were taken as inputs of the LSTM network to perform the emotion recognition.

Results: To validate the effect of the ICRM method on the performance of the proposed model, we designed three groups of comparative experiments. The average accuracy of the ICRM-LSTM model were 96.75 % and 98.09 % in SEED and MAHNOB-HCI, respectively. Then we compared our model with the models didn't perform the ICA or RM method, where we found that the proposed model had better performance. Furthermore, to verify the robustness, we added three groups of Gaussian noise with different signal-to-noise ratios (SNR) to the preprocessed EEG signals, and the proposed model achieved a good performance in both the two datasets.

Comparison with existing method: The performance of our model was superior to most of existing methods which also employed the SEED or MAHNOB-HCI dataset.

Conclusion: This paper demonstrated that the ICA and RM were helpful for EEG-based emotion recognition, and provided the evidence that the RM method could effectively alleviate the problem of the uncertain ordering of ICA.

Keywords: EEG; Emotion recognition; Independent component analysis; Long Short-term memory; Riemannian manifold.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Algorithms
Deep Learning*
Electroencephalography* / methods
Emotions
Humans
Memory, Long-Term