SIFIAE: An adaptive emotion recognition model with EEG feature-label inconsistency consideration

Abstract

Background: A common but easily overlooked affective overlap problem has not been received enough attention in electroencephalogram (EEG)-based emotion recognition research. In real life, affective overlap refers to the current emotional state of human being is sometimes influenced easily by his/her historical mood. In stimulus-evoked EEG collection experiment, due to the short rest interval in consecutive trials, the inner mechanisms of neural responses make subjects cannot switch their emotion state easily and quickly, which might lead to the affective overlap. For example, we might be still in sad state to some extent even if we are watching a comedy because we just saw a tragedy before. In pattern recognition, affective overlap usually means that there exists the feature-label inconsistency in EEG data.

New methods: To alleviate the impact of inconsistent EEG data, we introduce a variable to adaptively explore the sample inconsistency in emotion recognition model development. Then, we propose a semi-supervised emotion recognition model for joint sample inconsistency and feature importance exploration (SIFIAE). Accordingly, an efficient optimization method to SIFIAE model is proposed.

Results: Extensive experiments on the SEED-V dataset demonstrate the effectiveness of SIFIAE. Specifically, SIFIAE achieves 69.10%, 67.01%, 71.50%, 73.26%, 72.07% and 71.35% average accuracies in six cross-session emotion recognition tasks.

Conclusion: The results illustrated that the sample weights have a rising trend in the beginning of most trials, which coincides with the affective overlap hypothesis. The feature importance factor indicated the critical bands and channels are more obvious compared with some models without considering EEG feature-label inconsistency.

Keywords: Affective overlap; Automatic weighting; EEG; Emotion recognition; Feature-label inconsistency; Pure semi-supervised learning.