Dorsal anterior cingulate-brainstem ensemble as a reinforcement meta-learner

PLoS Comput Biol. 2018 Aug 24;14(8):e1006370. doi: 10.1371/journal.pcbi.1006370. eCollection 2018 Aug.

Abstract

Optimal decision-making is based on integrating information from several dimensions of decisional space (e.g., reward expectation, cost estimation, effort exertion). Despite considerable empirical and theoretical efforts, the computational and neural bases of such multidimensional integration have remained largely elusive. Here we propose that the current theoretical stalemate may be broken by considering the computational properties of a cortical-subcortical circuit involving the dorsal anterior cingulate cortex (dACC) and the brainstem neuromodulatory nuclei: ventral tegmental area (VTA) and locus coeruleus (LC). From this perspective, the dACC optimizes decisions about stimuli and actions, and using the same computational machinery, it also modulates cortical functions (meta-learning), via neuromodulatory control (VTA and LC). We implemented this theory in a novel neuro-computational model-the Reinforcement Meta Learner (RML). We outline how the RML captures critical empirical findings from an unprecedented range of theoretical domains, and parsimoniously integrates various previous proposals on dACC functioning.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Animals
  • Brain Mapping / methods
  • Brain Stem / physiology
  • Cognition / physiology
  • Computer Simulation
  • Decision Making / physiology*
  • Gyrus Cinguli / physiology*
  • Humans
  • Learning / physiology
  • Locus Coeruleus / physiology
  • Models, Theoretical
  • Reinforcement, Psychology
  • Reward
  • Ventral Tegmental Area / physiology

Grants and funding

MS was funded from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie Grant Agreement No. 795919. EV was funded from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie Grant Agreement No. 705630. EA was supported by Research Foundation Flanders under contract number 12C4715N. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.