Semantic processing during continuous speech production: an analysis from eye movements and EEG

Jinfeng Huang; Gaoyan Zhang; Jianwu Dang; Yu Chen; Shoko Miyamoto

doi:10.3389/fnhum.2023.1253211

Semantic processing during continuous speech production: an analysis from eye movements and EEG

Front Hum Neurosci. 2023 Sep 1:17:1253211. doi: 10.3389/fnhum.2023.1253211. eCollection 2023.

Authors

Jinfeng Huang^{1

2}, Gaoyan Zhang³, Jianwu Dang³, Yu Chen⁴, Shoko Miyamoto¹

Affiliations

¹ Faculty of Human Sciences, University of Tsukuba, Ibaraki, Japan.
² Research Institute, NeuralEcho Technology Co., Ltd., Beijing, China.
³ Tianjin Key Laboratory of Cognitive Computing and Application, College of Intelligence and Computing, Tianjin University, Tianjin, China.
⁴ Technical College for the Deaf, Tianjin University of Technology, Tianjin, China.

Abstract

Introduction: Speech production involves neurological planning and articulatory execution. How speakers prepare for articulation is a significant aspect of speech production research. Previous studies have focused on isolated words or short phrases to explore speech planning mechanisms linked to articulatory behaviors, including investigating the eye-voice span (EVS) during text reading. However, these experimental paradigms lack real-world speech process replication. Additionally, our understanding of the neurological dimension of speech planning remains limited.

Methods: This study examines speech planning mechanisms during continuous speech production by analyzing behavioral (eye movement and speech) and neurophysiological (EEG) data within a continuous speech production task. The study specifically investigates the influence of semantic consistency on speech planning and the occurrence of "look ahead" behavior.

Results: The outcomes reveal the pivotal role of semantic coherence in facilitating fluent speech production. Speakers access lexical representations and phonological information before initiating speech, emphasizing the significance of semantic processing in speech planning. Behaviorally, the EVS decreases progressively during continuous reading of regular sentences, with a slight increase for non-regular sentences. Moreover, eye movement pattern analysis identifies two distinct speech production modes, highlighting the importance of semantic comprehension and prediction in higher-level lexical processing. Neurologically, the dual pathway model of speech production is supported, indicating a dorsal information flow and frontal lobe involvement. The brain network linked to semantic understanding exhibits a negative correlation with semantic coherence, with significant activation during semantic incoherence and suppression in regular sentences.

Discussion: The study's findings enhance comprehension of speech planning mechanisms and offer insights into the role of semantic coherence in continuous speech production. Furthermore, the research methodology establishes a valuable framework for future investigations in this domain.

Keywords: brain network; continuous speech production; eye-voice span; semantic processing; speech planning.