Sensing technologies and machine learning methods for emotion recognition in autism: Systematic review

Oresti Banos; Zhoe Comas-González; Javier Medina; Aurora Polo-Rodríguez; David Gil; Jesús Peral; Sandra Amador; Claudia Villalonga

doi:10.1016/j.ijmedinf.2024.105469

Sensing technologies and machine learning methods for emotion recognition in autism: Systematic review

Int J Med Inform. 2024 Jul:187:105469. doi: 10.1016/j.ijmedinf.2024.105469. Epub 2024 May 3.

Authors

Oresti Banos¹, Zhoe Comas-González², Javier Medina³, Aurora Polo-Rodríguez⁴, David Gil⁵, Jesús Peral⁶, Sandra Amador⁵, Claudia Villalonga³

Affiliations

¹ Department of Computer Engineering, Automation and Robotics, University of Granada, Granada, Spain. Electronic address: oresti@ugr.es.
² Department of Computer Engineering, Automation and Robotics, University of Granada, Granada, Spain; Department of Computer Science and Electronics, Universidad de la Costa, Barranquilla, Colombia.
³ Department of Computer Engineering, Automation and Robotics, University of Granada, Granada, Spain.
⁴ Department of Computer Engineering, Automation and Robotics, University of Granada, Granada, Spain; Department of Computer Science, University of Jaén, Jaén, Spain.
⁵ Department of Computer Technology and Computation, University of Alicante, Alicante, Spain.
⁶ Department of Sotware and Computing Systems, University of Alicante, Alicante, Spain. Electronic address: jperal@ua.es.

PMID: 38723429
DOI: 10.1016/j.ijmedinf.2024.105469

Abstract

Background: Human Emotion Recognition (HER) has been a popular field of study in the past years. Despite the great progresses made so far, relatively little attention has been paid to the use of HER in autism. People with autism are known to face problems with daily social communication and the prototypical interpretation of emotional responses, which are most frequently exerted via facial expressions. This poses significant practical challenges to the application of regular HER systems, which are normally developed for and by neurotypical people.

Objective: This study reviews the literature on the use of HER systems in autism, particularly with respect to sensing technologies and machine learning methods, as to identify existing barriers and possible future directions.

Methods: We conducted a systematic review of articles published between January 2011 and June 2023 according to the 2020 PRISMA guidelines. Manuscripts were identified through searching Web of Science and Scopus databases. Manuscripts were included when related to emotion recognition, used sensors and machine learning techniques, and involved children with autism, young, or adults.

Results: The search yielded 346 articles. A total of 65 publications met the eligibility criteria and were included in the review.

Conclusions: Studies predominantly used facial expression techniques as the emotion recognition method. Consequently, video cameras were the most widely used devices across studies, although a growing trend in the use of physiological sensors was observed lately. Happiness, sadness, anger, fear, disgust, and surprise were most frequently addressed. Classical supervised machine learning techniques were primarily used at the expense of unsupervised approaches or more recent deep learning models. Studies focused on autism in a broad sense but limited efforts have been directed towards more specific disorders of the spectrum. Privacy or security issues were seldom addressed, and if so, at a rather insufficient level of detail.

Keywords: Autism; Datasets; Human emotion recognition; Machine learning techniques.

Publication types

Systematic Review
Review

MeSH terms

Autistic Disorder* / psychology
Child
Emotions*
Facial Expression*
Humans
Machine Learning*