Music emotion recognition based on temporal convolutional attention network using EEG

Yinghao Qiao; Jiajia Mu; Jialan Xie; Binghui Hu; Guangyuan Liu

doi:10.3389/fnhum.2024.1324897

Music emotion recognition based on temporal convolutional attention network using EEG

Front Hum Neurosci. 2024 Mar 28:18:1324897. doi: 10.3389/fnhum.2024.1324897. eCollection 2024.

Authors

Yinghao Qiao^{1

2

3}, Jiajia Mu^{1

2

3}, Jialan Xie^{1

2

3}, Binghui Hu^{1

2

3}, Guangyuan Liu^{1

2

3}

Affiliations

¹ School of Electronic and Information Engineering, Southwest University, Chongqing, China.
² Institute of Affective Computing and Information Processing, Southwest University, Chongqing, China.
³ Chongqing Key Laboratory of Nonlinear Circuits and Intelligent Information Processing, Southwest University, Chongqing, China.

Abstract

Music is one of the primary ways to evoke human emotions. However, the feeling of music is subjective, making it difficult to determine which emotions music triggers in a given individual. In order to correctly identify emotional problems caused by different types of music, we first created an electroencephalogram (EEG) data set stimulated by four different types of music (fear, happiness, calm, and sadness). Secondly, the differential entropy features of EEG were extracted, and then the emotion recognition model CNN-SA-BiLSTM was established to extract the temporal features of EEG, and the recognition performance of the model was improved by using the global perception ability of the self-attention mechanism. The effectiveness of the model was further verified by the ablation experiment. The classification accuracy of this method in the valence and arousal dimensions is 93.45% and 96.36%, respectively. By applying our method to a publicly available EEG dataset DEAP, we evaluated the generalization and reliability of our method. In addition, we further investigate the effects of different EEG bands and multi-band combinations on music emotion recognition, and the results confirm relevant neuroscience studies. Compared with other representative music emotion recognition works, this method has better classification performance, and provides a promising framework for the future research of emotion recognition system based on brain computer interface.

Keywords: BiLSTM; CNN; EEG; music emotion recognition; self-attention.

Grants and funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was supported in part by the National Natural Science Foundation of China under Grant Nos. 61872301 and 61472330.