The sensory cortex participates in emotional memory but its role is poorly understood. Here we show that inactivation of the higher order auditory cortex Te2 in rats during early memory consolidation impairs remote first- and second-order fear memories but not the association between two neutral cues. Furthermore, Te2 inactivation prevents changes in the valence of such information. Following the presentation of two auditory cues previously paired with either pleasant or painful stimuli, a large percentage of cells responds to both experiences but also a small fraction of neurons responds exclusively to one of them. The latter type of neurons signals the valence rather than the salience or the motor responses associated with the stimuli, and reflects selective associative processes. Pharmacogenetic silencing of memory-activated neurons causes amnesia. Thus, Te2 represents a crucial node for the assignment of the affective value to sensory stimuli and for the storage of such information.