Effective social interactions require the ability to evaluate other people's actions and intentions, sometimes only on the basis of such subtle factors as body language, and these evaluative judgments may lead to powerful impressions. However, little is known about the impact of affective body language on evaluative responses in social settings and the associated neural correlates. This study investigated the neural correlates of observing social interactions in a business setting, in which whole-body dynamic stimuli displayed approach and avoidance behaviors that were preceded or not by a handshake and were followed by participants' ratings of these behaviors. First, approach was associated with more positive evaluations than avoidance behaviors, and a handshake preceding social interaction enhanced the positive impact of approach and diminished the negative impact of avoidance behavior on the evaluation of social interaction. Second, increased sensitivity to approach than to avoidance behavior in the amygdala and STS was linked to a positive evaluation of approach behavior and a positive impact of handshake. Third, linked to the positive effect of handshake on social evaluation, nucleus accumbens showed greater activity for Handshake than for No-handshake conditions. These findings shed light on the neural correlates of observing and evaluating nonverbal social interactions and on the role of handshake as a way of formal greeting.