Dynamics of facial actions for assessing smile genuineness

PLoS One. 2021 Jan 5;16(1):e0244647. doi: 10.1371/journal.pone.0244647. eCollection 2021.

Abstract

Applying computer vision techniques to distinguish between spontaneous and posed smiles is an active research topic of affective computing. Although there have been many works published addressing this problem and a couple of excellent benchmark databases created, the existing state-of-the-art approaches do not exploit the action units defined within the Facial Action Coding System that has become a standard in facial expression analysis. In this work, we explore the possibilities of extracting discriminative features directly from the dynamics of facial action units to differentiate between genuine and posed smiles. We report the results of our experimental study which shows that the proposed features offer competitive performance to those based on facial landmark analysis and on textural descriptors extracted from spatial-temporal blocks. We make these features publicly available for the UvA-NEMO and BBC databases, which will allow other researchers to further improve the classification scores, while preserving the interpretation capabilities attributed to the use of facial action units. Moreover, we have developed a new technique for identifying the smile phases, which is robust against the noise and allows for continuous analysis of facial videos.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Affect
  • Algorithms
  • Emotions
  • Facial Expression
  • Humans
  • Smiling*
  • Social Perception

Grants and funding

This work was supported by the National Science Centre, Poland, under Research Grants No. 2017/25/B/ST6/00474 (MK, JN), 2015/19/D/ST6/03252 (JK), 2012/07/B/ST6/01227 (BS, MK) and 2017/25/B/ST6/02219 (BS) and was also funded by the Silesian University of Technology, Poland, BK/200/RAU1/2020 (BS). This work was co-financed by SUT grant for maintaining and developing research potential. MK was supported by the Silesian University of Technology funds through the Rector’s Research and Development Grant 02/080/RGJ20/0004. JN was supported by the Silesian University of Technology grant for maintaining and developing research potential, and by the Rector’s Research and Development Grant 02/080/RGJ20/0003. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.