Comparing the psychometric properties of two primary school Computational Thinking (CT) assessments for grades 3 and 4: The Beginners' CT test (BCTt) and the competent CT test (cCTt)

Laila El-Hamamsy; María Zapata-Cáceres; Pedro Marcelino; Barbara Bruno; Jessica Dehler Zufferey; Estefanía Martín-Barroso; Marcos Román-González

doi:10.3389/fpsyg.2022.1082659

Comparing the psychometric properties of two primary school Computational Thinking (CT) assessments for grades 3 and 4: The Beginners' CT test (BCTt) and the competent CT test (cCTt)

Front Psychol. 2022 Dec 12:13:1082659. doi: 10.3389/fpsyg.2022.1082659. eCollection 2022.

Authors

Laila El-Hamamsy^{1

2}, María Zapata-Cáceres³, Pedro Marcelino⁴, Barbara Bruno⁵, Jessica Dehler Zufferey², Estefanía Martín-Barroso³, Marcos Román-González⁶

Affiliations

¹ MOBOTS Group, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland.
² LEARN - Center for Learning Sciences, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland.
³ Laboratory of Information Technologies in Education, Rey Juan Carlos University, Madrid, Spain.
⁴ Computational Thinking Department, TreeTree2 (T2), Lisbon, Portugal.
⁵ CHILI Laboratory, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland.
⁶ Faculty of Education, Universidad Nacional de Educación a Distancia (UNED), Madrid, Spain.

Abstract

Introduction: With the increasing amount of research around Computational Thinking (CT) and endeavors introducing CT into curricula worldwide, assessing CT at all levels of formal education is of utmost importance to ensure that CT-related learning objectives are met. This has contributed to a progressive increase in the number of validated and reliable CT assessments for K-12, including primary school. Researchers and practitioners are thus required to choose among multiple instruments, often overlapping in their age validity.

Methods: In this study, we compare the psychometric properties of two of these instruments: the Beginners' CT test (BCTt), developed for grades 1-6, and the competent CT test (cCTt), validated for grades 3-4. Classical Test Theory and Item Response Theory (IRT) were employed on data acquired from 575 students in grades 3-4 to compare the properties of the two instruments and refine the limits of their validity.

Results: The findings (i) establish the detailed psychometric properties of the BCTt in grades 3-4 for the first time, and (ii) through a comparison with students from the same country, indicate that the cCTt should be preferred for grades 3-4 as the cCTt is able to discriminate between students of low and medium ability. Conversely, while the BCTt, which is easier, shows a ceiling effect, it is better suited to discriminate between students in the low ability range. For these grades, the BCTt can thus be employed as a screening mechanism to identify low ability students.

Discussion: In addition to providing recomendations for use of these instruments, the findings highlight the importance of comparing the psychometric properties of existing assessments, so that researchers and practitioners, including teachers and policy makers involved in digital education curricular reforms, may take informed decisions when selecting assessments.

Keywords: Computational Thinking; assessment; developmental appropriateness; primary school; psychometrics; validation.