Framework for identifying and visualising emotional atmosphere in online learning environments in the COVID-19 Era

Fei Yan; Nan Wu; Abdullah M Iliyasu; Kazuhiko Kawamoto; Kaoru Hirota

doi:10.1007/s10489-021-02916-z

Framework for identifying and visualising emotional atmosphere in online learning environments in the COVID-19 Era

Appl Intell (Dordr). 2022;52(8):9406-9422. doi: 10.1007/s10489-021-02916-z. Epub 2022 Jan 6.

Authors

Fei Yan¹, Nan Wu², Abdullah M Iliyasu^{3

4}, Kazuhiko Kawamoto², Kaoru Hirota⁴

Affiliations

¹ School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, China.
² Graduate School of Science and Engineering, Chiba University, Chiba, Japan.
³ College of Engineering, Prince Sattam Bin Abdulaziz University, Al-Kharj, Saudi Arabia.
⁴ School of Computing, Tokyo Institute of Technology, Tokyo, Japan.

Abstract

In addition to the almost five million lives lost and millions more than that in hospitalisations, efforts to mitigate the spread of the COVID-19 pandemic, which that has disrupted every aspect of human life deserves the contributions of all and sundry. Education is one of the areas most affected by the COVID-imposed abhorrence to physical (i.e., face-to-face (F2F)) communication. Consequently, schools, colleges, and universities worldwide have been forced to transition to different forms of online and virtual learning. Unlike F2F classes where the instructors could monitor and adjust lessons and content in tandem with the learners' perceived emotions and engagement, in online learning environments (OLE), such tasks are daunting to undertake. In our modest contribution to ameliorate disruptions to education caused by the pandemic, this study presents an intuitive model to monitor the concentration, understanding, and engagement expected of a productive classroom environment. The proposed apposite OLE (i.e., AOLE) provides an intelligent 3D visualisation of the classroom atmosphere (CA), which could assist instructors adjust and tailor both content and instruction for maximum delivery. Furthermore, individual learner status could be tracked via visualisation of his/her emotion curve at any stage of the lesson or learning cycle. Considering the enormous emotional and psychological toll caused by COVID and the attendant shift to OLE, the emotion curves could be progressively compared through the duration of the learning cycle and the semester to track learners' performance through to the final examinations. In terms of learning within the CA, our proposed AOLE is assessed within a class of 15 students and three instructors. Correlation of the outcomes reported with those from administered questionnaires validate the potential of our proposed model as a support for learning and counselling during these unprecedentedtimes that we find ourselves.

Keywords: COVID-19; Classroom atmosphere; Emotion space; Facial expression recognition; Online learning environment.