Dual-process models of recognition memory suggest that two processes contribute to performance: recollection and familiarity. Recent work suggests that the two processes are dissociable at the level of the brain. Here we review 12 studies that used event-related functional magnetic resonance imaging (fMRI), and 21 studies of patients with damage to various brain regions, which examined recollection and familiarity using the 'Remember-Know' (R/K), process dissociation procedure (PDP), or receiver operator characteristic (ROC) memory paradigms, for insights into the neural basis of each process. Results show that recollection and familiarity are characterized by different patterns of brain activity in frontal, parietal, sensory, and medial temporal cortices. Results suggest that recollection and familiarity cannot be dissociated based on confidence levels alone, and that the two processes are not exclusive. Based on these results, we propose a model in which recollection and familiarity can be dissociated in two ways: recruitment of additional brain regions in frontal, medial temporal, and content-specific cortices during recollection, and in variations in coherence of brain networks activated during recollective- or familiarity-based processing.