Deficient rapid temporal processing may contribute to impaired language development by interfering with the processing of brief acoustic transitions crucial for speech perception. Using magnetoencephalography, evoked neural activity (M50, M100) to two 40 ms tones passively presented in rapid succession was recorded in 10 neurologically normal adults and 40 8-17-year-olds with autism, specific language impairment, Asperger syndrome or typical development. While 80% of study participants with intact language (Asperger syndrome, typical development, adults) showed identifiable responses to the second tone, which presented rapid temporal processing demands, 65% of study participants with impaired language (autism, specific language impairment) did not, despite having shown identifiable responses to the first tone. Rapid temporal processing impairments may be fundamentally associated with impairments in language rather than autism spectrum disorder.