Beyond sense-specific processing: decoding texture in the brain from touch and sonified movement

C Landelle; J Caron-Guyon; B Nazarian; J L Anton; J Sein; L Pruvost; M Amberg; F Giraud; O Félician; J Danna; A Kavounoudias

doi:10.1016/j.isci.2023.107965

Beyond sense-specific processing: decoding texture in the brain from touch and sonified movement

iScience. 2023 Sep 20;26(10):107965. doi: 10.1016/j.isci.2023.107965. eCollection 2023 Oct 20.

Authors

C Landelle^{1

2}, J Caron-Guyon^{2

3}, B Nazarian⁴, J L Anton⁴, J Sein⁴, L Pruvost⁵, M Amberg⁶, F Giraud⁶, O Félician⁷, J Danna^{2

8}, A Kavounoudias²

Affiliations

¹ McGill University, McConnell Brain Imaging Centre, Department of Neurology and Neurosurgery, Montreal Neurological Institute, Montreal, QC, Canada.
² Aix-Marseille Université, CNRS, Laboratoire de Neurosciences Cognitives, LNC UMR 7291, Marseille, France.
³ University of Louvain, Institute for Research in Psychology (IPSY) & Institute of Neuroscience (IoNS), Louvain Bionics Center, Crossmodal Perception and Plasticity Laboratory, Louvain-la-Neuve, Belgium.
⁴ Aix-Marseille Université, CNRS, Centre IRM-INT@CERIMED, Institut de Neurosciences de la Timone, INT UMR 7289, Marseille, France.
⁵ Aix-Marseille Université, CNRS, Perception, Représentations, Image, Son, Musique, PRISM UMR 7061, Marseille, France.
⁶ Université Lille, Laboratoire d'Electrotechnique et d'Electronique de Puissance, EA 2697-L2EP, Lille, France.
⁷ Aix Marseille Université, INSERM, Institut des Neurosciences des Systèmes, INS UMR 1106, Marseille, France.
⁸ Université de Toulouse, CNRS, Laboratoire Cognition, Langues, Langage, Ergonomie, CLLE UMR5263, Toulouse, France.

Abstract

Texture, a fundamental object attribute, is perceived through multisensory information including touch and auditory cues. Coherent perceptions may rely on shared texture representations across different senses in the brain. To test this hypothesis, we delivered haptic textures coupled with a sound synthesizer to generate real-time textural sounds. Participants completed roughness estimation tasks with haptic, auditory, or bimodal cues in an MRI scanner. Somatosensory, auditory, and visual cortices were all activated during haptic and auditory exploration, challenging the traditional view that primary sensory cortices are sense-specific. Furthermore, audio-tactile integration was found in secondary somatosensory (S2) and primary auditory cortices. Multivariate analyses revealed shared spatial activity patterns in primary motor and somatosensory cortices, for discriminating texture across both modalities. This study indicates that primary areas and S2 have a versatile representation of multisensory textures, which has significant implications for how the brain processes multisensory cues to interact more efficiently with our environment.

Keywords: Cognitive neuroscience; Neuroscience; Sensory neuroscience.