EEG classification model for virtual reality motion sickness based on multi-scale CNN feature correlation

Chengcheng Hua; Jianlong Tao; Zhanfeng Zhou; Lining Chai; Ying Yan; Jia Liu; Rongrong Fu

doi:10.1016/j.cmpb.2024.108218

EEG classification model for virtual reality motion sickness based on multi-scale CNN feature correlation

Comput Methods Programs Biomed. 2024 May 7:251:108218. doi: 10.1016/j.cmpb.2024.108218. Online ahead of print.

Authors

Chengcheng Hua¹, Jianlong Tao¹, Zhanfeng Zhou¹, Lining Chai¹, Ying Yan¹, Jia Liu¹, Rongrong Fu²

Affiliations

¹ School of Automation, C-IMER, CICAEET, Nanjing University of Information Science & Technology, Nanjing 210044, China.
² Measurement Technology and Instrumentation Key Laboratory of Hebei Province, Department of Electrical Engineering, Yanshan University, Qinhuangdao 066000, China. Electronic address: frr1102@aliyun.com.

PMID: 38728828
DOI: 10.1016/j.cmpb.2024.108218

Abstract

Background: Virtual reality motion sickness (VRMS) is a key issue hindering the development of virtual reality technology, and accurate detection of its occurrence is the first prerequisite for solving the issue.

Objective: In this paper, a convolutional neural network (CNN) EEG detection model based on multi-scale feature correlation is proposed for detecting VRMS.

Methods: The model uses multi-scale 1D convolutional layers to extract multi-scale temporal features from the multi-lead EEG data, and then calculates the feature correlations of the extracted multi-scale features among all the leads to form the feature adjacent matrixes, which converts the time-domain features to correlation-based brain network features, thus strengthen the feature representation. Finally, the correlation features of each layer are fused. The fused features are then fed into the channel attention module to filter the channels and classify them using a fully connected network. Finally, we recruit subjects to experience 6 different modes of virtual roller coaster scenes, and collect resting EEG data before and after the task to verify the model.

Results: The results show that the accuracy, precision, recall and F1-score of this model for the recognition of VRMS are 98.66 %, 98.65 %, 98.68 %, and 98.66 %, respectively. The proposed model outperforms the current classic and advanced EEG recognition models.

Significance: It shows that this model can be used for the recognition of VRMS based on the resting state EEG.

Keywords: Channel attention; Feature correlation matrix; Multi-scale feature fusion; Resting state EEG; Virtual reality motion sickness.