Supramodal Sentence Processing in the Human Brain: fMRI Evidence for the Influence of Syntactic Complexity in More Than 200 Participants

Julia Uddén; Annika Hultén; Jan-Mathijs Schoffelen; Nietzsche Lam; Karin Harbusch; Antal van den Bosch; Gerard Kempen; Karl Magnus Petersson; Peter Hagoort

doi:10.1162/nol_a_00076

Supramodal Sentence Processing in the Human Brain: fMRI Evidence for the Influence of Syntactic Complexity in More Than 200 Participants

Neurobiol Lang (Camb). 2022 Sep 29;3(4):575-598. doi: 10.1162/nol_a_00076. eCollection 2022.

Authors

Julia Uddén^{1

2

3

4}, Annika Hultén^{1

2}, Jan-Mathijs Schoffelen², Nietzsche Lam^{1

2}, Karin Harbusch⁵, Antal van den Bosch², Gerard Kempen¹, Karl Magnus Petersson^{1

2}, Peter Hagoort^{1

2}

Affiliations

¹ Max Planck Institute for Psycholinguistics, Nijmegen, the Netherlands.
² Donders Institute for Brain, Cognition and Behaviour, Centre for Cognitive Neuroimaging, Radboud University, Nijmegen, the Netherlands.
³ Department of Linguistics, Stockholm University, Stockholm, Sweden.
⁴ Department of Psychology, Stockholm University, Stockholm, Sweden.
⁵ Department of Computer Science, University of Koblenz-Landau, Koblenz, Germany.

Abstract

This study investigated two questions. One is: To what degree is sentence processing beyond single words independent of the input modality (speech vs. reading)? The second question is: Which parts of the network recruited by both modalities is sensitive to syntactic complexity? These questions were investigated by having more than 200 participants read or listen to well-formed sentences or series of unconnected words. A largely left-hemisphere frontotemporoparietal network was found to be supramodal in nature, i.e., independent of input modality. In addition, the left inferior frontal gyrus (LIFG) and the left posterior middle temporal gyrus (LpMTG) were most clearly associated with left-branching complexity. The left anterior temporal lobe showed the greatest sensitivity to sentences that differed in right-branching complexity. Moreover, activity in LIFG and LpMTG increased from sentence onset to end, in parallel with an increase of the left-branching complexity. While LIFG, bilateral anterior temporal lobe, posterior MTG, and left inferior parietal lobe all contribute to the supramodal unification processes, the results suggest that these regions differ in their respective contributions to syntactic complexity related processing. The consequences of these findings for neurobiological models of language processing are discussed.

Keywords: complexity; fMRI; sentence processing; supramodal; unification.