Autonomic nervous system (ANS) responses such as heart rate (HR) and galvanic skin responses (GSR) have been linked with cerebral activity in the context of emotion. Although much work has focused on the summative effect of emotions on ANS responses, their interaction in a continuously changing context is less clear. Here, we used a multimodal data set of human affective states, which includes electroencephalogram (EEG) and peripheral physiological signals of participants' moment-by-moment reactions to emotional provoking video clips and modeled HR and GSR changes using machine learning techniques, specifically, long short-term memory (LSTM), decision tree (DT), and linear regression (LR). We found that LSTM achieved a significantly lower error rate compared with DT and LR due to its inherent ability to handle sequential data. Importantly, the prediction error was significantly reduced for DT and LR when used together with particle swarm optimization to select relevant/important features for these algorithms. Unlike summative analysis, and contrary to expectations, we found a significantly lower error rate when the prediction was made across different participants than within a participant. Moreover, the predictive selected features suggest that the patterns predictive of HR and GSR were substantially different across electrode sites and frequency bands. Overall, these results indicate that specific patterns of cerebral activity track autonomic body responses. Although individual cerebral differences are important, they might not be the only factors influencing the moment-by-moment changes in ANS responses.
Keywords: EEG; autonomic nervous system; feature selection; machine learning; peripheral physiology; psychophysics.
© 2023 The Authors. Psychophysiology published by Wiley Periodicals LLC on behalf of Society for Psychophysiological Research.