Risk aversion is a common behavior universal to humans and animals alike. Economists have traditionally defined risk preferences by the curvature of the utility function. Psychologists and behavioral economists also make use of concepts such as loss aversion and probability weighting to model risk aversion. Neurophysiological evidence suggests that loss aversion has its origins in relatively ancient neural circuitries (e.g., ventral striatum). Could there thus be an evolutionary origin to risk aversion? We study this question by evolving strategies that adapt to play the equivalent mean payoff gamble. We hypothesize that risk aversion in this gamble is beneficial as an adaptation to living in small groups, and find that a preference for risk averse strategies only evolves in small populations of less than 1,000 individuals, or in populations segmented into groups of 150 individuals or fewer - numbers thought to be comparable to what humans encountered in the past. We observe that risk aversion only evolves when the gamble is a rare event that has a large impact on the individual's fitness. As such, we suggest that rare, high-risk, high-payoff events such as mating and mate competition could have driven the evolution of risk averse behavior in humans living in small groups.