Despite the emphasis of autism spectrum disorders as a continuum of atypical social behaviors and the sexual heterogeneity of phenotypic manifestations, whether gaze processing constitutes an autistic endophenotype in both sexes remains unclear. Using the Autism-Spectrum Quotient and a psychophysical approach in a normal population (N = 128), here we demonstrated that individual differences in autistic traits predicted direct-gaze perception for males, but not for females. Our findings suggest that direct-gaze perception may not constitute an autistic endophenotype in both sexes, and highlight the importance of sex differences when considering relationships between autistic traits and behaviors.