The emotions people feel can be simulated internally based on emotional situational contexts. In the present study, we assessed the behavioral and neuroelectric effects of seeing an unexpected emotional facial expression. We investigated the correct answer rate, response times and Event-Related Potential (ERP) effects during an incongruence paradigm between emotional faces and sentential contexts allowing emotional inferences. Most of the 36 healthy participants were recruited from a larger population (1 463 subjects), based on their scores on the Empathy Questionnaire (EQ). Regression analyses were conducted on these ratings using EQ factors as predictors (cognitive empathy, emotional reactivity and social skills). Recognition of pragmatic emotional incongruence was less accurate (P < .05) and slower (P < .05) than recognition of congruence. The incongruence effect on response times was inversely predicted by social skills. A significant N400 incongruence effect was found at the centro-parietal (P < .001) and centro-posterior midline (P < .01) electrodes. Cognitive empathy predicted the incongruence effect in the left occipital region, in the N400 time window. Finally, incongruence effects were also found on the LPP wave, in frontal midline and dorso-frontal regions, (P < .05), with no modulation by empathy. Processing pragmatic emotional incongruence is more cognitively demanding than congruence (as reflected by both behavioral and ERP data). This processing shows modulation by personality factors at the behavioral (through self-reported social skills) and neuroelectric levels (through self-reported cognitive empathy).