Face recognition emerges from an interaction between bottom-up and top-down processing. Specifically, it relies on complex associations between the visual representation of a given face and previously stored knowledge about that face (e.g. biographical details). In the present experiment, the time-course of the interaction between bottom-up and top-down processing was investigated using event-related potentials (ERPs) and manipulating realistic, ecological contextual information. In the study phase, half of the faces (context faces) were framed in a newspaper page entitled with an action committed by the person depicted; these actions could have a positive or a negative value, so in this way emotional valence could be manipulated. The other half was presented on a neutral background (no-context faces). In the test phase, previously presented faces and new ones were presented on neutral backgrounds and an old/new discrimination was requested. The N170 component was modulated by both context (presence/absence at encoding) and valence (positive/negative). A reduction in amplitude was found for context faces as opposed to no-context faces. The same pattern was observed for negative faces compared to positive ones. Moreover, later activations associated with context and valence were differentially distributed over the scalp: context effects were prominent in left frontal areas, traditionally linked to person-specific information retrieval, whereas valence effects were broadly distributed over the scalp. In relation to recent neuroimaging findings on the neural basis of top-down modulations, present findings indicate that the information flow from higher-order areas might have modulated the N170 component and mediated the retrieval of semantic information pertaining to the study episode.