Using process features to investigate scientific problem-solving in large-scale assessments

Tao Gong; Lan Shuai; Yang Jiang; Burcu Arslan

doi:10.3389/fpsyg.2023.1131019

Using process features to investigate scientific problem-solving in large-scale assessments

Front Psychol. 2023 Apr 18:14:1131019. doi: 10.3389/fpsyg.2023.1131019. eCollection 2023.

Authors

Tao Gong^{1

2

3}, Lan Shuai², Yang Jiang², Burcu Arslan²

Affiliations

¹ School of Foreign Languages, Zhejiang University of Finance and Economics, Hangzhou, Zhejiang, China.
² Educational Testing Service, Princeton, NJ, United States.
³ Google, New York, NY, United States.

Abstract

Introduction: This study investigates the process data from scientific inquiry tasks of fair tests [requiring test-takers to manipulate a target variable while keeping other(s) constant] and exhaustive tests (requiring test-takers to construct all combinations of given variables) in the National Assessment of Educational Progress program.

Methods: We identify significant associations between item scores and temporal features of preparation time, execution time, and mean execution time.

Results: Reflecting, respectively, durations of action planning and execution, and execution efficiency, these process features quantitatively differentiate the high- and low-performing students: in the fair tests, high-performing students tended to exhibit shorter execution time than low-performing ones, but in the exhaustive tests, they showed longer execution time; and in both types of tests, high-performing students had shorter mean execution time than low-performing ones.

Discussion: This study enriches process features reflecting scientific problem-solving process and competence and sheds important light on how to improve performance in large-scale, online delivered scientific inquiry tasks.

Keywords: execution time; exhaustive test; fair test; preparation time; scientific problem solving.