Distinct processing of the state prediction error signals in frontal and parietal correlates in learning the environment model

Abstract

Goal-directed reinforcement learning constructs a model of how the states in the environment are connected and prospectively evaluates action values by simulating experience. State prediction error (SPE) is theorized as a crucial signal for learning the environment model. However, the underlying neural mechanisms remain unclear. Here, using electroencephalogram, we verified in a two-stage Markov task two neural correlates of SPEs: an early negative correlate transferring from frontal to central electrodes and a late positive correlate over parietal regions. Furthermore, by investigating the effects of explicit knowledge about the environment model and rewards in the environment, we found that, for the parietal correlate, rewards enhanced the representation efficiency (beta values of regression coefficient) of SPEs, whereas explicit knowledge elicited a larger SPE representation (event-related potential activity) for rare transitions. However, for the frontal and central correlates, rewards increased activities in a content-independent way and explicit knowledge enhanced activities only for common transitions. Our results suggest that the parietal correlate of SPEs is responsible for the explicit learning of state transition structure, whereas the frontal and central correlates may be involved in cognitive control. Our study provides novel evidence for distinct roles of the frontal and the parietal cortices in processing SPEs.

Keywords: EEG; explicit knowledge; model-based reinforcement learning; reward; state prediction error.