It is generally believed that attention enhances the processing of sensory information during perception and learning. Here we report that, contrary to common belief, attention limits the degree of plasticity induced by repeated exposure to image features. Specifically, daily exposure to oriented stimuli that are not linked to a specific task causes an orientation-specific improvement in perceptual performance along the "exposed" axes. This effect is modulated by attention: human subjects showed a larger improvement in orientation discrimination when attention is directed toward the location where stimuli are presented. However, the capacity to perform discriminations away from the exposed orientation is enhanced when the exposure stimuli are unattended. Importantly, the improvement in orientation discrimination at the unattended location leads to a robust enhancement in the discrimination of complex stimuli, such as natural texture images, with orientation components along the exposed axes, whereas the improvement in orientation discrimination at the attended location exhibits only weak transfer to complex stimuli. These results indicate that sensory adaptation by passive stimulus exposure should be viewed as a form of perceptual learning that is complementary to practice-based learning in that it reduces constraints on generalization.