This study tested the hypothesis that affective arousal in response to eye contact is negatively associated with face identification skills in children with autism spectrum disorder (ASD). Participants were 20 children and adolescents with ASD and 20 age- and IQ-matched typically developing (TD) children. Skin conductance response (SCR), a psychophysiological measure of autonomic arousal, was collected while participants viewed faces with gaze directed toward them and faces with gaze averted away from them. Participants also completed an independent match-to-sample face recognition test. Children with ASD exhibited significantly larger SCRs than TD children to faces with direct and averted gaze. There were no differences between SCRs to direct gaze and averted gaze in either group. Children with ASD exhibited a marginally significant decrease in face recognition accuracy relative to TD children, particularly when face recognition depended on the eye region of the face. Face recognition accuracy among children with ASD was negatively correlated with the amplitude of SCRs to direct gaze but not to averted gaze. There was no association between face recognition accuracy and SCRs to gaze in the TD group. These findings suggest that autonomic reactivity to eye contact may interfere with face identity processing in some children with ASD.