Contextual modulation is a fundamental feature of sensory processing, both on perceptual and on single-neuron level. When the diameter of a visual stimulus is increased, the firing rate of a cell typically first increases (summation field) and then decreases (surround field). Such an area summation function draws a comprehensive profile of the receptive field structure of a neuron, including areas outside the classical receptive field. We investigated area summation in human vision with psychophysics and functional magnetic resonance imaging (fMRI). The stimuli were drifting sine wave gratings similar to those used in previous macaque single-cell area summation studies [corrected]. A model was developed to facilitate comparison of area summation in fMRI to area summation in psychophysics and single cells. The model consisted of units with an antagonistic receptive field structure found in single cells in the primary visual cortex. The receptive field centers of the model neurons were distributed in the region of the visual field covered by a single voxel. The measured area summation functions were qualitatively similar to earlier single-cell data. The model with parameters derived from psychophysics captured the spatial structure of the summation field in the primary visual cortex as measured with fMRI. The model also generalized to a novel situation in which the neural population was displaced from the stimulus center. The current study shows that contextual modulation arises from similar spatially antagonistic and overlapping excitatory and inhibitory mechanisms, both in single cells and in human vision.