Recent studies pointing to evaluation methods in natural environments suggest that their use in the analysis of metacognitive skills provides more precise information than the use of off-line evaluation methods. In this research, mixed methods are used over one academic year for the evaluation of the metacognitive skills that students of Secondary Education apply to solve physics problems. The objectives of this study are to analyze the use of metacognitive skills in natural environments and to study behavioral patterns of student learning through a longitudinal study. A total of 509 recordings of think-aloud protocols are analyzed through the categorization of the responses (liquefying) and the protocol of Van der Stel and Veenman for the analysis of the quality of metacognitive skills. Fewer conceptual errors and less uncertainty over vocabulary were noted during the academic year. Nevertheless, a degree of ambiguity persisted in the understanding of physics concepts. The metacognitive skills of Orientation and Planning were used more than any others. The technique of graph analysis is also applied, to establish the patterns of behavior of each student throughout the academic year. Different patterns were found, the analysis of which helped to identify academically challenged and at-risk students. The use of mixed observation techniques and graph analysis facilitated information on the pace of learning of each student. Future studies will be directed at proposals for the automation of these evaluation techniques in natural learning environments.
Keywords: indirect observation; metacognitive skills; mixed methods; natural environments; on-line evaluation methods; personalized learning; quality; systematic observation.
Copyright © 2019 Sáiz Manzanares, Queiruga Dios, García-Osorio, Montero García and Rodríguez-Medina.