Individuals with autism spectrum disorders (ASD) tend to make inadequate social judgments, particularly when the nonverbal and verbal emotional expressions of other people are incongruent. Although previous behavioral studies have suggested that ASD individuals have difficulty in using nonverbal cues when presented with incongruent verbal-nonverbal information, the neural mechanisms underlying this symptom of ASD remain unclear. In the present functional magnetic resonance imaging study, we compared brain activity in 15 non-medicated adult males with high-functioning ASD to that of 17 age-, parental-background-, socioeconomic-, and intelligence-quotient-matched typically-developed (TD) male participants. Brain activity was measured while each participant made friend or foe judgments of realistic movies in which professional actors spoke with conflicting nonverbal facial expressions and voice prosody. We found that the ASD group made significantly less judgments primarily based on the nonverbal information than the TD group, and they exhibited significantly less brain activity in the right inferior frontal gyrus, bilateral anterior insula, anterior cingulate cortex/ventral medial prefrontal cortex (ACC/vmPFC), and dorsal medial prefrontal cortex (dmPFC) than the TD group. Among these five regions, the ACC/vmPFC and dmPFC were most involved in nonverbal-information-biased judgments in the TD group. Furthermore, the degree of decrease of the brain activity in these two brain regions predicted the severity of autistic communication deficits. The findings indicate that diminished activity in the ACC/vmPFC and dmPFC underlies the impaired abilities of individuals with ASD to use nonverbal content when making judgments regarding other people based on incongruent social information.