Visual perception is limited by both the strength of the neural signals, and by the noise in the visual nervous system; however, little is known about what aspects of the input noise the human visual system is sensitive to, i.e., what is the signal in noise? In order to investigate this question we asked observers to discriminate differences in the strength of one-dimensional white noise. We measured their response consistency and classification images and compared the results with an ideal energy detector. Our results and modelling show that discrimination of noise is limited by the observer's template (i.e., the weighted combination of energy in each stimulus component) plus higher order nonlinearities (systematic noise), and by sources of random internal noise. We found that systematic noise is present only near detection threshold. Surprisingly, we found that the human template is "adaptive"-- its shape depends on the spatial frequency band of the noise-suggesting that sensitivity to spatial noise is not simply determined via passive filtering.