Modern rapidly expanding cities generate intricate patterns of species diversity owing to immense complexity in urban spatial structure and current growth trajectories. We propose to identify and uncouple the drivers that give rise to these patterns by looking at the effect of urbanism on species diversity over a previously unexplored long temporal frame that covers early developments in urbanism. To provide this historical perspective we analyzed archaeozoological remains of small mammals from ancient urban and rural sites in the Near East from the 2nd to the 1st millennium BCE, and compared them to observations from modern urban areas. Our data show that ancient urban assemblages consistently comprised two main taxa (Mus musculus domesticus and Crocidura sp.), whereas assemblages of contemporaneous rural sites were significantly richer. Low species diversity also characterizes high-density core areas of modern cities, suggesting that similar ecological drivers have continued to operate in urban areas despite the vast growth in their size and population densities, as well as in the complexity of their technologies and social organization. Research in urban ecology has tended to emphasize the relatively high species diversity observed in low-density areas located on the outskirts of cities, where open and vegetated patches are abundant. The fact that over several millennia urban evolution did not significantly alter species diversity suggests that low diversity is an attribute of densely-populated settlements. The possibility that high diversity in peripheral urban areas arose only recently as a short-term phenomenon in urban ecology merits further research based on long-term data.