A silent eligibility trace enables dopamine-dependent synaptic plasticity for reinforcement learning in the mouse striatum

Abstract

Dopamine-dependent synaptic plasticity is a candidate mechanism for reinforcement learning. A silent eligibility trace - initiated by synaptic activity and transformed into synaptic strengthening by later action of dopamine - has been hypothesized to explain the retroactive effect of dopamine in reinforcing past behaviour. We tested this hypothesis by measuring time-dependent modulation of synaptic plasticity by dopamine in adult mouse striatum, using whole-cell recordings. Presynaptic activity followed by postsynaptic action potentials (pre-post) caused spike-timing-dependent long-term depression in D1-expressing neurons, but not in D2 neurons, and not if postsynaptic activity followed presynaptic activity. Subsequent experiments focused on D1 neurons. Applying a dopamine D1 receptor agonist during induction of pre-post plasticity caused long-term potentiation. This long-term potentiation was hidden by long-term depression occurring concurrently and was unmasked when long-term depression blocked an L-type calcium channel antagonist. Long-term potentiation was blocked by a Ca²⁺ -permeable AMPA receptor antagonist but not by an NMDA antagonist or an L-type calcium channel antagonist. Pre-post stimulation caused transient elevation of rectification - a marker for expression of Ca²⁺ -permeable AMPA receptors - for 2-4-s after stimulation. To test for an eligibility trace, dopamine was uncaged at specific time points before and after pre- and postsynaptic conjunction of activity. Dopamine caused potentiation selectively at synapses that were active 2-s before dopamine release, but not at earlier or later times. Our results provide direct evidence for a silent eligibility trace in the synapses of striatal neurons. This dopamine-timing-dependent plasticity may play a central role in reinforcement learning.

Keywords: dopamine; learning; reinforcement; temporal difference.