Why do areas with high numbers of small-range species occur where they do? We found that, for butterfly and plant species in Europe, and for bird species in the Western Hemisphere, such areas coincide with regions that have rare climates, and are higher and colder areas than surrounding regions. Species with small range sizes also tend to occur in climatically diverse regions, where species are likely to have been buffered from extinction in the past. We suggest that the centres of high small-range species richness we examined predominantly represent interglacial relict areas where cold-adapted species have been able to survive unusually warm periods in the last ca 10000 years. We show that the rare climates that occur in current centres of species rarity will shrink disproportionately under future climate change, potentially leading to high vulnerability for many of the species they contain.