Task-Dependent Warping of Semantic Representations during Search for Visual Action Categories

Mo Shahdloo; Emin Çelik; Burcu A Urgen; Jack L Gallant; Tolga Çukur

doi:10.1523/JNEUROSCI.1372-21.2022

Task-Dependent Warping of Semantic Representations during Search for Visual Action Categories

J Neurosci. 2022 Aug 31;42(35):6782-6799. doi: 10.1523/JNEUROSCI.1372-21.2022.

Authors

Mo Shahdloo^{1

2

3}, Emin Çelik^{2

4}, Burcu A Urgen^{2

5

4}, Jack L Gallant⁶, Tolga Çukur^{7

3

4

6}

Affiliations

¹ Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford OX3 9DU, United Kingdom.
² National Magnetic Resonance Research Centre, Bilkent University, 06800 Ankara, Turkey.
³ Departments of Electrical and Electronics Engineering and.
⁴ Neuroscience Program, Aysel Sabuncu Brain Research Centre, Bilkent University, 06800 Ankara, Turkey.
⁵ Psychology, Bilkent University, 06800 Ankara, Turkey.
⁶ Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, California 94720.
⁷ National Magnetic Resonance Research Centre, Bilkent University, 06800 Ankara, Turkey cukur@ee.bilkent.edu.tr.

Abstract

Object and action perception in cluttered dynamic natural scenes relies on efficient allocation of limited brain resources to prioritize the attended targets over distractors. It has been suggested that during visual search for objects, distributed semantic representation of hundreds of object categories is warped to expand the representation of targets. Yet, little is known about whether and where in the brain visual search for action categories modulates semantic representations. To address this fundamental question, we studied brain activity recorded from five subjects (one female) via functional magnetic resonance imaging while they viewed natural movies and searched for either communication or locomotion actions. We find that attention directed to action categories elicits tuning shifts that warp semantic representations broadly across neocortex and that these shifts interact with intrinsic selectivity of cortical voxels for target actions. These results suggest that attention serves to facilitate task performance during social interactions by dynamically shifting semantic selectivity toward target actions and that tuning shifts are a general feature of conceptual representations in the brain.SIGNIFICANCE STATEMENT The ability to swiftly perceive the actions and intentions of others is a crucial skill for humans that relies on efficient allocation of limited brain resources to prioritize the attended targets over distractors. However, little is known about the nature of high-level semantic representations during natural visual search for action categories. Here, we provide the first evidence showing that attention significantly warps semantic representations by inducing tuning shifts in single cortical voxels, broadly spread across occipitotemporal, parietal, prefrontal, and cingulate cortices. This dynamic attentional mechanism can facilitate action perception by efficiently allocating neural resources to accentuate the representation of task-relevant action categories.

Keywords: attention; fMRI; natural movies; visual actions; voxelwise modeling.