Feedback delay impaired reinforcement learning: Principal components analysis of Reward Positivity

Abstract

An immediate feedback after action facilitated reinforcement learning in dynamically varying environments. With several seconds delay, a series of event-related potential (ERP) studies have recently conducted to explore how delayed feedback influences learning processes and corresponding brain activities by measuring the Reward Positivity and N170 component. However, it remains unclear how does our brain process a feedback that is delayed longer and interrupted by other trials. In the present study, participants were asked to undertake a time-estimation task in two different conditions. Feedback was presented right after their actions in the immediate feedback condition, while it was presented after another five trials in the delayed feedback condition. By recording feedback related activities, we aim to test whether, or not, delayed feedback impairs reinforcement learning, the Reward Positivity and N170 amplitude. The behavioural results show that delayed feedback can reduce behavioural adjustment efficiency from trial-to-trial. To reduce component overlapping, we adopted the temporospatial principal components analysis (PCA) to separate the Reward Positivity from other ERP components. Results indicate that the Reward Positivity is decreased in the delayed feedback condition compared to the immediate feedback condition, however, no difference of N170 amplitude is found between the two conditions. These results indicate that delayed feedback impairs reinforcement learning process in terms of behavioural adjustment and brain activities even though these feedbacks are truly associated with participants' previous actions.

Keywords: FRN; Feedback delay; PCA; Reinforcement learning; Reward Positivity.