In the present article we review behavioral and neurophysiological studies on face processing in adults and in early development. From the existing empirical and theoretical literature we derive three aspects that distinguish face processing from the processing of other visual object categories. Each of these aspects is discussed from a developmental perspective. First, faces are recognized and represented at the individual level rather than at the basic level. Second, humans typically acquire extensive expertise in individuating faces from early on in development. And third, more than other objects, faces are processed holistically. There is a quantitative difference in the amount of visual experience for faces and other object categories in that the amount of expertise typically acquired for faces is greater than that for other object categories. In addition, we discuss possible qualitative differences in experience for faces and objects. For instance, there is evidence for a sensitive period in infancy for building up a holistic face representation and for perceptual narrowing for faces of one's own species and race. We conclude our literature review with questions for future research, for instance, regarding the exact relationship between behavioral and neuronal markers of face processing across development.