Functional properties of the basal ganglia's re-entrant loop architecture: selection and reinforcement

Abstract

Multifunctional agents with limited motor resources must decide what actions will best ensure their survival. Moreover, given that in an unpredictable world things don't always work out, considerable advantage is to be gained by learning from experience - instrumental behaviour that maximises reward and minimises punishment. In this review we will argue that the re-entrant looped architecture of the basal ganglia represents biological solutions to these fundamental behavioural problems of selection and reinforcement. A potential solution to the selection problem is provided for by selective disinhibition within the parallel loop architecture that connects the basal ganglia with external neural structures. The relay points within these loops permit the signals of a particular channel to be modified by external influences. In part, these influences have the capacity to modify overall selections so that the probability of re-selecting reinforced behaviours in the future is altered. This is the basic process of instrumental learning, which we suggest decomposes into two sub-problems for the agent: (i) learning which external events it causes to happen and learning precisely what it is doing that is causal; and (ii) having determined agency and discovered novel action-outcome routines, how best to exploit this knowledge to maximise future reward acquisitions. Considerations of connectional architecture and signal timing suggest that the short-latency, sensory-evoked dopamine response, which can modulate the re-entrant loop structure within the basal ganglia, is ideally suited to reinforce the determination of agency and the discovery of novel actions. Alternatively, recent studies showing that presence or absence of reward can selectively modulate the magnitude of signals in structures providing input signals to the basal ganglia, offer an alternative mechanism for biasing selection within the re-entrant loop architecture. We suggest that this mechanism may be better suited to ensure the prioritisation of inputs associated with reward.