Believer-Skeptic Meets Actor-Critic: Rethinking the Role of Basal Ganglia Pathways during Decision-Making and Reinforcement Learning

Kyle Dunovan; Timothy Verstynen

doi:10.3389/fnins.2016.00106

Believer-Skeptic Meets Actor-Critic: Rethinking the Role of Basal Ganglia Pathways during Decision-Making and Reinforcement Learning

Front Neurosci. 2016 Mar 24:10:106. doi: 10.3389/fnins.2016.00106. eCollection 2016.

Authors

Kyle Dunovan¹, Timothy Verstynen²

Affiliations

¹ Department of Psychology, University of PittsburghPittsburgh, PA, USA; Center for the Neural Basis of Cognition, University of Pittsburgh and Carnegie Mellon UniversityPittsburgh, PA, USA.
² Center for the Neural Basis of Cognition, University of Pittsburgh and Carnegie Mellon UniversityPittsburgh, PA, USA; Department of Psychology, Carnegie Mellon UniversityPittsburgh, PA, USA.

Abstract

The flexibility of behavioral control is a testament to the brain's capacity for dynamically resolving uncertainty during goal-directed actions. This ability to select actions and learn from immediate feedback is driven by the dynamics of basal ganglia (BG) pathways. A growing body of empirical evidence conflicts with the traditional view that these pathways act as independent levers for facilitating (i.e., direct pathway) or suppressing (i.e., indirect pathway) motor output, suggesting instead that they engage in a dynamic competition during action decisions that computationally captures action uncertainty. Here we discuss the utility of encoding action uncertainty as a dynamic competition between opposing control pathways and provide evidence that this simple mechanism may have powerful implications for bridging neurocomputational theories of decision making and reinforcement learning.

Keywords: basal ganglia; decision making; diffusion model; exploration-exploitation tradeoff; reinforcement learning; speed-accuracy tradeoff.