This set of three experiments assessed the influence of different psychophysical factors on the lateralization of the N170 event-related potential (ERP) component to words and faces. In all experiments, words elicited a left-lateralized N170, whereas faces elicited a right-lateralized or nonlateralized N170 depending on presentation conditions. Experiment 1 showed that lateralization for words (but not for faces) was influenced by spatial frequency. Experiment 2 showed that stimulus presentation time influenced N170 lateralization independently of spatial frequency composition. Finally, Experiment 3 showed that stimulus size and resolution did not influence N170 lateralization, but did influence N170 amplitude, albeit differentially for words and faces. These findings suggest that differential lateralization for words and faces, at least as measured by the N170, is influenced by spatial frequency (words), stimulus presentation time, and category.