Animal species come in many shapes and sizes, as do the individuals and populations that make up each species. To us, humans might seem to show particularly high levels of morphological variation, but perhaps this perception is simply based on enhanced recognition of individual conspecifics relative to individual heterospecifics. We here more objectively ask how humans compare to other animals in terms of body size variation. We quantitatively compare levels of variation in body length (height) and mass within and among 99 human populations and 848 animal populations (210 species). We find that humans show low levels of within-population body height variation in comparison to body length variation in other animals. Humans do not, however, show distinctive levels of within-population body mass variation, nor of among-population body height or mass variation. These results are consistent with the idea that natural and sexual selection have reduced human height variation within populations, while maintaining it among populations. We therefore hypothesize that humans have evolved on a rugged adaptive landscape with strong selection for body height optima that differ among locations.