A unifying framework for functional organization in early and higher ventral visual cortex

Eshed Margalit; Hyodong Lee; Dawn Finzi; James J DiCarlo; Kalanit Grill-Spector; Daniel L K Yamins

doi:10.1016/j.neuron.2024.04.018

A unifying framework for functional organization in early and higher ventral visual cortex

Neuron. 2024 May 8:S0896-6273(24)00279-4. doi: 10.1016/j.neuron.2024.04.018. Online ahead of print.

Authors

Eshed Margalit¹, Hyodong Lee², Dawn Finzi³, James J DiCarlo⁴, Kalanit Grill-Spector⁵, Daniel L K Yamins⁶

Affiliations

¹ Neurosciences Graduate Program, Stanford University, Stanford, CA 94305, USA. Electronic address: eshed.margalit@gmail.com.
² Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.
³ Department of Psychology, Stanford University, Stanford, CA 94305, USA; Department of Computer Science, Stanford University, Stanford, CA 94305, USA.
⁴ Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Center for Brains Minds and Machines, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.
⁵ Department of Psychology, Stanford University, Stanford, CA 94305, USA; Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA 94305, USA.
⁶ Department of Psychology, Stanford University, Stanford, CA 94305, USA; Department of Computer Science, Stanford University, Stanford, CA 94305, USA; Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA 94305, USA.

PMID: 38733985
DOI: 10.1016/j.neuron.2024.04.018

Abstract

A key feature of cortical systems is functional organization: the arrangement of functionally distinct neurons in characteristic spatial patterns. However, the principles underlying the emergence of functional organization in the cortex are poorly understood. Here, we develop the topographic deep artificial neural network (TDANN), the first model to predict several aspects of the functional organization of multiple cortical areas in the primate visual system. We analyze the factors driving the TDANN's success and find that it balances two objectives: learning a task-general sensory representation and maximizing the spatial smoothness of responses according to a metric that scales with cortical surface area. In turn, the representations learned by the TDANN are more brain-like than in spatially unconstrained models. Finally, we provide evidence that the TDANN's functional organization balances performance with between-area connection length. Our results offer a unified principle for understanding the functional organization of the primate ventral visual system.

Keywords: dimensionality; neural network; topography; ventral visual cortex; vision; wiring length.