Enhanced Accuracy for Multiclass Mental Workload Detection Using Long Short-Term Memory for Brain-Computer Interface

Umer Asgher; Khurram Khalil; Muhammad Jawad Khan; Riaz Ahmad; Shahid Ikramullah Butt; Yasar Ayaz; Noman Naseer; Salman Nazir

doi:10.3389/fnins.2020.00584

Enhanced Accuracy for Multiclass Mental Workload Detection Using Long Short-Term Memory for Brain-Computer Interface

Front Neurosci. 2020 Jun 23:14:584. doi: 10.3389/fnins.2020.00584. eCollection 2020.

Authors

Umer Asgher¹, Khurram Khalil¹, Muhammad Jawad Khan¹, Riaz Ahmad^{1

2}, Shahid Ikramullah Butt¹, Yasar Ayaz^{1

3}, Noman Naseer⁴, Salman Nazir⁵

Affiliations

¹ School of Mechanical and Manufacturing Engineering (SMME), National University of Sciences and Technology (NUST), Islamabad, Pakistan.
² Directorate of Quality Assurance and International Collaboration, National University of Sciences and Technology (NUST), Islamabad, Pakistan.
³ National Center of Artificial Intelligence (NCAI) - NUST, Islamabad, Pakistan.
⁴ Department of Mechatronics Engineering, Air University, Islamabad, Pakistan.
⁵ Training and Assessment Research Group, Department of Maritime Operations, University of South-Eastern Norway, Kongsberg, Norway.

Abstract

Cognitive workload is one of the widely invoked human factors in the areas of human-machine interaction (HMI) and neuroergonomics. The precise assessment of cognitive and mental workload (MWL) is vital and requires accurate neuroimaging to monitor and evaluate the cognitive states of the brain. In this study, we have decoded four classes of MWL using long short-term memory (LSTM) with 89.31% average accuracy for brain-computer interface (BCI). The brain activity signals are acquired using functional near-infrared spectroscopy (fNIRS) from the prefrontal cortex (PFC) region of the brain. We performed a supervised MWL experimentation with four varying MWL levels on 15 participants (both male and female) and 10 trials of each MWL per participant. Real-time four-level MWL states are assessed using fNIRS system, and initial classification is performed using three strong machine learning (ML) techniques, support vector machine (SVM), k-nearest neighbor (k-NN), and artificial neural network (ANN) with obtained average accuracies of 54.33, 54.31, and 69.36%, respectively. In this study, novel deep learning (DL) frameworks are proposed, which utilizes convolutional neural network (CNN) and LSTM with 87.45 and 89.31% average accuracies, respectively, to solve high-dimensional four-level cognitive states classification problem. Statistical analysis, t-test, and one-way F-test (ANOVA) are also performed on accuracies obtained through ML and DL algorithms. Results show that the proposed DL (LSTM and CNN) algorithms significantly improve classification performance as compared with ML (SVM, ANN, and k-NN) algorithms.

Keywords: brain–computer interface; convolutional neural network; deep learning; deep neural networks; functional near-infrared spectroscopy; long short-term memory; mental workload.