Understanding the causes of genetic variation in real populations has been elusive. Competing theories claim that neutral vs. selective processes have a greater influence on the genetic variation within a population. A key difference among theories is the relationship between population size and genetic diversity. Our study tests this empirically by sampling two species of herbivorous lizards (Dipsosaurus dorsalis and Sauromalus ater) and two species of carnivorous lizards (Crotaphytus bicinctores and Gambelia wislizenii) that vary in population size at the same locality, and comparing metrics of genetic diversity. Contrary to neutral expectations, results from four independent loci showed levels of diversity were usually higher for species with smaller population sizes. This suggests that selective processes may be having an important impact on intraspecific diversity in this reptile community, although tests showed little evidence for selection on the loci sequenced for this study. It is also possible that idiosyncratic histories of the focal species may be overriding predictions from simple neutral models. If future studies show that lack of correlation between population size and genetic diversity is common, methods using genetic diversity to estimate population parameters like population size or time to common ancestor should be used with caution, as these estimates are based on neutral theory predictions.