An efficacious HIV vaccine is urgently needed to curb the AIDS pandemic. The modest protection elicited in the phase III clinical vaccine trial in Thailand provided hope that this goal might be achieved. However, new approaches are necessary for further advances. As HIV is transmitted primarily across mucosal surfaces, development of immunity at these sites is critical, but few clinical vaccine trials have targeted these sites or assessed vaccine-elicited mucosal immune responses. Pre-clinical studies in non-human primate models have facilitated progress in mucosal vaccine development by evaluating candidate vaccine approaches, developing methodologies for collecting and assessing mucosal samples, and providing clues to immune correlates of protective immunity for further investigation. In this review we have focused on non-human primate studies which have provided important information for future design of vaccine strategies, targeting of mucosal inductive sites, and assessment of mucosal immunity. Knowledge gained in these studies will inform mucosal vaccine design and evaluation in human clinical trials.