Faces are of essential importance for human social life. They provide valuable information about the identity, expression, gaze, health, and age of a person. Recent face-processing models assume highly interconnected neural structures between different temporal, occipital, and frontal brain areas with several feedback loops. A selective deficit in the visual learning and recognition of faces is known as prosopagnosia, which can be found both in acquired and congenital form. Recently, a hereditary sub-type of congenital prosopagnosia with a very high prevalence rate of 2.5% has been identified. Recent research results show that hereditary prosopagnosia is a clearly circumscribed face-processing deficit with a characteristic set of clinical symptoms. Comparing face processing of people of prosopagnosia with that of controls can help to develop a more conclusive and integrated model of face processing. Here, we provide a summary of the current state of face processing research. We also describe the different types of prosopagnosia and present the set of typical symptoms found in the hereditary type. Finally, we will discuss the implications for future face recognition research.