Studies have shown that fusiform face area (FFA) activity increases with visual expertise. We present an fMRI study showing that faces from a social category made relevant by an experimental manipulation (members of an experimentally created in-group) preferentially recruited the FFA even when they were matched in exposure to face stimuli from a less significant social category (members of an experimentally created out-group). Faces were randomly assigned to groups and fully counterbalanced so that no perceptual cues allowed participants to visually distinguish category membership. The results revealed a pattern of in-group enhancement (not out-group disregard), such that the FFA was selectively engaged following the presentation of in-group compared with out-group or unaffiliated control faces even when the intergroup distinction was arbitrary, and exposure to in-group and out-group faces was equivalent and brief. In addition, individual differences in FFA activity for in-group versus out-group faces were correlated with recognition memory differences for in-group and out-group faces. The effects of group membership on the FFA were not affected by task instruction to respond to in-group or out-group members and were functionally dissociated from early visual processing in the primary visual cortex. This study provides evidence that the FFA is sensitive to top-down influences and may be involved in subordinate level (vs. superordinate level) encoding of stimuli in the absence of long-term exposure or explicit task instructions.