Discussion of an overlap between specific language impairment (SLI) and autism spectrum disorder (ASD) is on going. The most intriguing overlap between both phenotypes is the similarity in the observed language deficits described in SLI and a subgroup of ASD with co-occurring linguistic impairment, ASD-LI. Examining whether a similar neuroanatomical substrate underlies this phenotypical linguistic overlap, we studied the white matter microstructural properties of the superior longitudinal fascicle (SLF) of 19 ASD-LI adolescents (mean age 13.8 ± 1.6 years) and 21 age-matched controls and compared them with 13 SLI children (mean age 10.1 ± 0.4 years) and 12 age-matched controls. A linguistic profile assessment and a diffusion tensor imaging analysis of the SLF were performed. Linguistic testing revealed a mixed receptive-expressive disorder profile in both groups, confirming their overlap at phenotypical level. At neuroanatomical level, no significant differences in mean SLF fractional anisotropy (FA) and mean SLF apparent diffusion coefficient values between ASD-LI participants and controls were seen. By contrast, the mean SLF FA was significantly reduced in the SLI children as compared with their controls. The observation of structural SLF disturbances in SLI but not in ASD-LI suggests the existence of a different neuroanatomical substrate for the language deficits in both disorders.