Previous studies have suggested that men and women process emotional stimuli differently. In this study, we examined if there would be any consistency in regions of activation in men and women when processing stimuli portraying happy or sad emotions presented in the form of facial expressions, scenes, and words. A blocked design BOLD functional magnetic resonance imaging paradigm was employed to monitor the neural activities of male and female healthy volunteers while they were presented with the experimental stimuli. The imaging data revealed that the right insula and left thalamus were consistently activated for men, but not women, during emotion recognition of all forms of stimuli studied. To further understand the imaging data acquired, we conducted the protocol analysis method to identify the cognitive processes engaged while the men and women were viewing the emotional stimuli and deciding whether they were happy or sad. The findings suggest that men rely on the recall of past emotional experiences to evaluate current emotional experiences. This may explain why the insula, a structure important for self-induced or internally generated recalled emotions, was consistently activated in men while processing emotional stimuli. Our findings suggest possible gender-related neural responses to emotional stimuli.