Great European mountain ranges have acted as barriers to gene flow for resident populations since prehistory and have offered a place for the settlement of small, and sometimes culturally diverse, communities. Therefore, the human groups that have settled in these areas are worth exploring as an important potential source of diversity in the genetic structure of European populations. In this study, we present new high resolution data concerning Y chromosomal variation in three distinct Alpine ethno-linguistic groups, Italian, Ladin and German. Combining unpublished and literature data on Y chromosome and mitochondrial variation, we were able to detect different genetic patterns. In fact, within and among population diversity values observed vary across linguistic groups, with German and Italian speakers at the two extremes, and seem to reflect their different demographic histories. Using simulations we inferred that the joint effect of continued genetic isolation and reduced founding group size may explain the apportionment of genetic diversity observed in all groups. Extending the analysis to other continental populations, we observed that the genetic differentiation of Ladins and German speakers from Europeans is comparable or even greater to that observed for well known outliers like Sardinian and Basques. Finally, we found that in south Tyroleans, the social practice of Geschlossener Hof, a hereditary norm which might have favored male dispersal, coincides with a significant intra-group diversity for mtDNA but not for Y chromosome, a genetic pattern which is opposite to those expected among patrilocal populations. Together with previous evidence regarding the possible effects of "local ethnicity" on the genetic structure of German speakers that have settled in the eastern Italian Alps, this finding suggests that taking socio-cultural factors into account together with geographical variables and linguistic diversity may help unveil some yet to be understood aspects of the genetic structure of European populations.