The ZuCo benchmark on cross-subject reading task classification with EEG and eye-tracking data

Nora Hollenstein; Marius Tröndle; Martyna Plomecka; Samuel Kiegeland; Yilmazcan Özyurt; Lena A Jäger; Nicolas Langer

doi:10.3389/fpsyg.2022.1028824

The ZuCo benchmark on cross-subject reading task classification with EEG and eye-tracking data

Front Psychol. 2023 Jan 12:13:1028824. doi: 10.3389/fpsyg.2022.1028824. eCollection 2022.

Authors

Nora Hollenstein¹, Marius Tröndle², Martyna Plomecka², Samuel Kiegeland³, Yilmazcan Özyurt³, Lena A Jäger^{4

5}, Nicolas Langer²

Affiliations

¹ Center for Language Technology, University of Copenhagen, Copenhagen, Denmark.
² Department of Psychology, University of Zurich, Zurich, Switzerland.
³ Department of Computer Science, ETH Zurich, Zurich, Switzerland.
⁴ Department of Computational Linguistics, University of Zurich, Zurich, Switzerland.
⁵ Department of Computer Science, University of Potsdam, Potsdam, Germany.

Abstract

We present a new machine learning benchmark for reading task classification with the goal of advancing EEG and eye-tracking research at the intersection between computational language processing and cognitive neuroscience. The benchmark task consists of a cross-subject classification to distinguish between two reading paradigms: normal reading and task-specific reading. The data for the benchmark is based on the Zurich Cognitive Language Processing Corpus (ZuCo 2.0), which provides simultaneous eye-tracking and EEG signals from natural reading of English sentences. The training dataset is publicly available, and we present a newly recorded hidden testset. We provide multiple solid baseline methods for this task and discuss future improvements. We release our code and provide an easy-to-use interface to evaluate new approaches with an accompanying public leaderboard: www.zuco-benchmark.com.

Keywords: EEG; cross-subject evaluation; eye-tracking; machine learning; reading research; reading task classification.

Grants and funding

This work was partially supported by the Swiss National Science Foundation under grant 100014_175875 (NL) and by the German Federal Ministry of Education and Research under grant 01|S20043 (LJ).