We describe new ESS models of density regulation driven by genic selection to explain the cyclical dynamics of a social system that exhibits a rock-paper-scissors (RPS) set of three alternative strategies. We tracked changes in morph frequency and fitness of Lacerta vivipara and found conspicuous RPS cycles. Morphs of Uta and Lacerta exhibited parallel survival-performance trade-offs. Frequency cycles in both species of lizards are driven by genic selection. In Lacerta, frequency of each allele in adult cohorts had significant impacts on juvenile recruitment, similar to mutualistic, altruistic, and antagonistic relations of RPS alleles in Uta. We constructed evolutionarily stable strategy (ESS) models in which adults impact juvenile recruitment as a function of self versus nonself color recognition. ESS models suggest that the rapid 4-year RPS cycles exhibited by Lacerta are not possible unless three factors are present: behaviors evolve that discriminate self versus nonself morphs at higher rates than random, self- versus non-self-recognition contributes to density regulation, and context-dependent mate choice evolves in females, which choose sire genotypes to enhance progeny survival. We suggest genic selection coupled to density regulation is widespread and thus fundamental to theories of social system evolution as well as theories of population regulation in diverse animal taxa.