Haxby et al. (Haxby JV, Hoffman EA, Gobbini MI. 2000. The distributed human neural system for face perception. Trends Cogn Sci. 4:223-233.) proposed that eye gaze processing results from an interaction between a "core" face-specific system involved in visual analysis and an "extended" system involved in spatial attention, more generally. However, the full gaze perception network has remained poorly specified. In the context of a functional magnetic resonance imaging study, we used psychophysiological interactions (PPIs) to identify brain regions that showed differential connectivity (correlation) with core face perception structures (posterior superior temporal sulcus [pSTS] and fusiform gyrus [FG]) when viewing gaze shifts relative to control eye movements (opening/closing the eyes). The PPIs identified altered connectivity between the pSTS and MT/V5, intraparietal sulcus, frontal eye fields, superior temporal gyrus (STG), supramarginal gyrus, and middle frontal gyrus (MFG). The FG showed altered connectivity with the same areas of the STG and MFG, demonstrating the contribution of both dorsal and ventral core face areas to gaze perception. We propose that this network provides an interactive system that alerts us to seen changes in other agents' gaze direction, makes us aware of their altered focus of spatial attention, and prepares a corresponding shift in our own attention.