Voice is not only the vehicle of speech, it is also an 'auditory face' that conveys a wealth of information on a person's identity and affective state. In contrast to speech perception, little is known about the neural bases of our ability to perceive these various types of paralinguistic vocal information. Using functional magnetic resonance imaging (fMRI), we identified regions along the superior temporal sulcus (STS) that were not only sensitive, but also highly selective to vocal sounds. In the present study, we asked how neural activity in the voice areas was influenced by (i) the presence or not of linguistic information in the vocal input (speech vs. nonspeech) and (ii) frequency scrambling. Speech sounds were found to elicit greater responses than nonspeech vocalizations in most parts of auditory cortex, including primary auditory cortex (A1), on both sides of the brain. In contrast, response attenuation due to frequency scrambling was much more pronounced in anterior STS areas than at the level of A1. Importantly, only right anterior STS regions responded more strongly to nonspeech vocal sounds than to their scrambled version, suggesting that these regions could be specifically involved in paralinguistic aspects of voice perception.