Objective.The understanding of cognitive states is important for the development of human-machine systems (HMSs), and one of the fundamental but challenging issues is the understanding and assessment of the operator's mental stress state in real task scenarios.Approach.In this paper, a virtual unmanned vehicle (UAV) driving task with multi-challenge-level was created to explore the operator's mental stress, and the human brain activity during the task was tracked in real time via electroencephalography (EEG). A mental stress analysis dataset for the virtual UAV task was then developed and used to explore the neural activation patterns associated with mental stress activity. Finally, a multiple attention-based convolutional neural network (MACN) was constructed for automatic stress assessment using the extracted stress-sensitive neural activation features.Main Results.The statistical results of EEG power spectral density (PSD) showed that frontal theta-PSD decreased with increasing task difficulty, and central beta-PSD increased with increasing task difficulty, indicating that neural patterns showed different trends under different levels of mental stress. The performance of the proposed MACN was evaluated based on the dimensional model, and results showed that average three-class classification accuracies of 89.49%/89.88% were respectively achieved for arousal/valence.Significance.The results of this paper suggest that objective assessment of mental stress in a HMS based on a virtual UAV scenario is feasible, and the proposed method provides a promising solution for cognitive computing and applications in human-machine tasks.
Keywords: deep learning; electroencephalogram (EEG); human–machine system; mental stress; virtual UAV.
© 2023 IOP Publishing Ltd.