In the hope of discovering early markers of autism, attention has recently turned to the study of infants at risk owing to being the younger siblings of children with autism. Because the condition is highly heritable, later-born siblings of diagnosed children are at substantially higher risk for developing autism or the broader autism phenotype than the general population. Currently, there are no strong predictors of autism in early infancy and diagnosis is not reliable until around 3 years of age. Because indicators of brain functioning may be sensitive predictors, and atypical social interactions are characteristic of the syndrome, we examined whether temporal lobe specialization for processing visual and auditory social stimuli during infancy differs in infants at risk. In a functional near-infrared spectroscopy study, infants aged 4-6 months at risk for autism showed less selective neural responses to social stimuli (auditory and visual) than low-risk controls. These group differences could not be attributed to overall levels of attention, developmental stage or chronological age. Our results provide the first demonstration of specific differences in localizable brain function within the first 6 months of life in a group of infants at risk for autism. Further, these differences closely resemble known patterns of neural atypicality in children and adults with autism. Future work will determine whether these differences in infant neural responses to social stimuli predict either later autism or the broader autism phenotype frequently seen in unaffected family members.