Image gradients--smooth changes in color and luminance--may be caused by intrinsic surface reflectance properties or extrinsic illumination phenomena, including shading, shadowing, and inter-reflections. In turn, image gradients may provide the visual system with information concerning the origin of these factors, such as the orientation of surfaces with respect to the light source. Color gradients induced by mutual illumination (MI) may play a similar role to that of luminance gradients in shape-from-shading algorithms; it has been shown that 3D shape perception modulates the influence of MI on surface color perception (M. G. Bloj, D. Kersten, & A. C. Hurlbert, 1999). In this study, we assess human sensitivity to changes in color and luminance gradients that arise from changes in the light source position, within a complex scene. In Experiment 1, we tested whether observers were able to discriminate between gradients due to different light source positions. We found that observers reliably detected a change in the gradient information when the light source position differed by only 4 deg from the reference scene. This sensitivity was mainly based on the luminance information in the gradient (Experiments 2 and 3). Some observers make use of the spatial distribution of chromaticity and luminance values within gradients when discriminating between them (Experiment 4). The high sensitivity to gradient differences supports the notion that gradients contain information that may assist in the recovery of 3D shape and scene configuration properties.