On the use of Action Units and fuzzy explanatory models for facial expression recognition

PLoS One. 2019 Oct 15;14(10):e0223563. doi: 10.1371/journal.pone.0223563. eCollection 2019.

Abstract

Facial expression recognition is related to the automatic identification of affective states of a subject by computational means. Facial expression recognition is used for many applications, such as security, human-computer interaction, driver safety, and health care. Although many works aim to tackle the problem of facial expression recognition, and the discriminative power may be acceptable, current solutions have limited explicative power, which is insufficient for certain applications, such as facial rehabilitation. Our aim is to alleviate the current limited explicative power by exploiting explainable fuzzy models over sequences of frontal face images. The proposed model uses appearance features to describe facial expressions in terms of facial movements, giving a detailed explanation of what movements are in the face, and why the model is making a decision. The model architecture was selected to keep the semantic meaning of the found facial movements. The proposed model can discriminate between the seven basic facial expressions, obtaining an average accuracy of 90.8±14%, with a maximum value of 92.9±28%.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Algorithms
  • Databases as Topic
  • Emotions
  • Facial Expression*
  • Facial Recognition*
  • Fuzzy Logic*
  • Humans
  • Models, Theoretical*

Grants and funding

EMV was supported by scholarship #702647 from the Mexican Consejo Nacional de Ciencia y Tecnología (CONACyT), as well as by the project “Analysis and Classification Techniques of Voice and Facial Expressions: Application to Neurological Diseases in Newborns and Adults" from AMEXCID, which is jointly supported by the Mexican and Italian governments. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.