Eye gaze is an important social signal, and humans can accurately determine gaze direction in others. Recently, dramatic adaptation effects on gaze perception were shown, in which the perception of small gaze deviations to the adapted direction is diminished. We compared participants' perceptions of gaze direction before and after adaptation to the left or right gaze, and examined event-related potential correlates of this gaze adaptation. We found a striking impairment in perceiving gaze to the adapted side. Event-related potentials revealed no direction-specific gaze adaptation effects on N170, although small adaptation effects were seen later, approximately 250-350 ms. This suggests that, rather than modulating the gaze processing approximately 170 ms in posterior occipitotemporal areas, adaptation modulates subsequent processes that are possibly mediated by more anterior right-temporal areas.