The cliff-edge hypothesis introduces the counterintuitive idea that the trait value associated with the maximum of an asymmetrical fitness function is not necessarily the value that is selected for if the trait shows variability in its phenotypic expression. We develop a model of population dynamics to show that, in such a system, the evolutionary stable strategy depends on both the shape of the fitness function around its maximum and the amount of phenotypic variance. The model provides quantitative predictions of the expected trait value distribution and provides an alternative quantity that should be maximized ("genotype fitness") instead of the classical fitness function ("phenotype fitness"). We test the model's predictions on three examples: (1) litter size in guinea pigs, (2) sexual selection in damselflies, and (3) the geometry of the human lung. In all three cases, the model's predictions give a closer match to empirical data than traditional optimization theory models. Our model can be extended to most ecological situations, and the evolutionary conditions for its application are expected to be common in nature.