We can discriminate departures from the vertical or horizontal more accurately than from other orientations. This may reflect perceptual learning, but the mechanisms behind such learning are not well understood. Here we derive a theory of discrimination learning based on criterion setting theory (CST; Treisman and Williams, 1984), an extension of signal detection theory in which judgment of the current stimulus is partly determined by previous discriminations and context. The CST-based theory of discrimination learning (CST-DL) describes mechanisms which use information from previous acts of discrimination to improve current decision making. CST-DL distinguishes between types of decision criteria and provides an account of anisotropies and context effects affecting discrimination. Predictions from this model are tested in experiments on anisotropies in orientation and depth perception. The results obtained support CST-DL. They also support the conclusion that the account of the retention of sensory information in delayed discrimination provided by CST is superior to the traditional belief that information retention relies on a fixed memory trace or representation of the stimulus.