Automated robust human emotion classification system using hybrid EEG features with ICBrainDB dataset

Erkan Deniz; Nebras Sobahi; Naaman Omar; Abdulkadir Sengur; U Rajendra Acharya

doi:10.1007/s13755-022-00201-y

Automated robust human emotion classification system using hybrid EEG features with ICBrainDB dataset

Health Inf Sci Syst. 2022 Nov 10;10(1):31. doi: 10.1007/s13755-022-00201-y. eCollection 2022 Dec.

Authors

Erkan Deniz¹, Nebras Sobahi², Naaman Omar³, Abdulkadir Sengur¹, U Rajendra Acharya^{4

5

6}

Affiliations

¹ Electrical and Electronics Engineering Department, Technology Faculty, Firat University, Elazig, Turkey.
² Department of Electrical and Computer Engineering, King Abdulaziz University, Jeddah, Saudi Arabia.
³ Department of Information Technology Management, Administration Technical College, Duhok Polytechnic University, Duhok, Iraq.
⁴ Department of Electronics and Computer Engineering, Ngee Ann Polytechnic, Singapore, 599489 Singapore.
⁵ Biomedical Engineering, School of Science and Technology, SUSS University, Singapore, Singapore.
⁶ Biomedical Informatics and Medical Engineering, Asia University, Taichung, Taiwan.

Abstract

Emotion identification is an essential task for human-computer interaction systems. Electroencephalogram (EEG) signals have been widely used in emotion recognition. So far, there have been several EEG-based emotion recognition datasets that the researchers have used to validate their developed models. Hence, we have used a new ICBrainDB EEG dataset to classify angry, neutral, happy, and sad emotions in this work. Signal processing-based wavelet transform (WT), tunable Q-factor wavelet transform (TQWT), and image processing-based histogram of oriented gradients (HOG), local binary pattern (LBP), and convolutional neural network (CNN) features have been used extracted from the EEG signals. The WT is used to extract the rhythms from each channel of the EEG signal. The instantaneous frequency and spectral entropy are computed from each EEG rhythm signal. The average, and standard deviation of instantaneous frequency, and spectral entropy of each rhythm of the signal are the final feature vectors. The spectral entropy in each channel of the EEG signal after performing the TQWT is used to create the feature vectors in the second signal side method. Each EEG channel is transformed into time-frequency plots using the synchrosqueezed wavelet transform. Then, the feature vectors are constructed individually using windowed HOG and LBP features. Also, each channel of the EEG data is fed to a pretrained CNN to extract the features. In the feature selection process, the ReliefF feature selector is employed. Various feature classification algorithms namely, k-nearest neighbor (KNN), support vector machines, and neural networks are used for the automated classification of angry, neutral, happy, and sad emotions. Our developed model obtained an average accuracy of 90.7% using HOG features and a KNN classifier with a tenfold cross-validation strategy.

Keywords: EEG signals; Emotion recognition; ICBrainDB dataset; Image-based features; Signal.

© The Author(s), under exclusive licence to Springer Nature Switzerland AG 2022, Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.