Decoding facial expressions based on face-selective and motion-sensitive areas

Hum Brain Mapp. 2017 Jun;38(6):3113-3125. doi: 10.1002/hbm.23578. Epub 2017 Mar 27.

Abstract

Humans can easily recognize others' facial expressions. Among the brain substrates that enable this ability, considerable attention has been paid to face-selective areas; in contrast, whether motion-sensitive areas, which clearly exhibit sensitivity to facial movements, are involved in facial expression recognition remained unclear. The present functional magnetic resonance imaging (fMRI) study used multi-voxel pattern analysis (MVPA) to explore facial expression decoding in both face-selective and motion-sensitive areas. In a block design experiment, participants viewed facial expressions of six basic emotions (anger, disgust, fear, joy, sadness, and surprise) in images, videos, and eyes-obscured videos. Due to the use of multiple stimulus types, the impacts of facial motion and eye-related information on facial expression decoding were also examined. It was found that motion-sensitive areas showed significant responses to emotional expressions and that dynamic expressions could be successfully decoded in both face-selective and motion-sensitive areas. Compared with static stimuli, dynamic expressions elicited consistently higher neural responses and decoding performance in all regions. A significant decrease in both activation and decoding accuracy due to the absence of eye-related information was also observed. Overall, the findings showed that emotional expressions are represented in motion-sensitive areas in addition to conventional face-selective areas, suggesting that motion-sensitive regions may also effectively contribute to facial expression recognition. The results also suggested that facial motion and eye-related information played important roles by carrying considerable expression information that could facilitate facial expression recognition. Hum Brain Mapp 38:3113-3125, 2017. © 2017 Wiley Periodicals, Inc.

Keywords: MVPA; eye-related information; fMRI; face-selective areas; facial expressions; facial motion; motion-sensitive areas.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Attention / physiology
  • Brain / diagnostic imaging*
  • Brain / physiology
  • Brain Mapping
  • Choice Behavior / physiology*
  • Face*
  • Facial Expression*
  • Female
  • Humans
  • Image Processing, Computer-Assisted
  • Magnetic Resonance Imaging
  • Male
  • Motion Perception / physiology*
  • Oxygen / blood
  • Pattern Recognition, Visual / physiology*
  • Photic Stimulation
  • Young Adult

Substances

  • Oxygen