A deep learning method for single-trial EEG classification in RSVP task based on spatiotemporal features of ERPs

Abstract

Objective. Single-trial electroencephalography (EEG) classification is of great importance in the rapid serial visual presentation (RSVP) task. Convolutional neural networks (CNNs), as one of the mainstream deep learning methods, have been proven to be effective in extracting RSVP EEG features. However, most existing CNN models for EEG classification do not consider the phase-locked characteristic of event-related potential (ERP) components very well in the architecture design. Here, we propose a novel CNN model to make better use of the phase-locked characteristic to extract spatiotemporal features for single-trial RSVP EEG classification. Based on the phase-locked characteristic, the spatial distributions of the main ERP component in different periods can be learned separately.Approach.In this work, we propose a novel CNN model to achieve superior performance on single-trial RSVP EEG classification. We introduce the combination of the standard convolutional layer, the permute layer and the depthwise convolutional layer to separately operate the spatial convolution in different periods, which more fully utilizes the phase-locked characteristic of ERPs for classification. We compare our model with several traditional and deep-learning methods in the classification performance. Moreover, we use spatial topography and saliency map to visually analyze the ERP features extracted by our model.Main results. The results show that our model obtains better classification performance than those of reference methods. The spatial topographies of each subject exhibit the typical ERP spatial distribution in different time periods. And the saliency map of each subject illustrates the discriminant electrodes and the meaningful temporal features.Significance. Our model is designed with better consideration of the phase-locked ERP characteristic and reaches excellent performance on single-trial RSVP EEG classification.

Keywords: EEG; RSVP; convolutional neural network; deep learning; event-related potential.