The debate over the role of stochasticity is central in evolutionary biology, often summarised by whether or not evolution is predictable or repeatable. Here we distinguish three types of stochasticity: stochasticity of mutation and variation, of individual life histories and of environmental change. We then explain when stochasticity matters in evolution, distinguishing four broad situations: stochasticity contributes to maladaptation or limits adaptation; it drives evolution on flat fitness landscapes (evolutionary freedom); it might promote jumps from one fitness peak to another (evolutionary revolutions); and it might shape the selection pressures themselves. We show that stochasticity, by directly steering evolution, has become an essential ingredient of evolutionary theory beyond the classical Wright-Fisher or neutralist-selectionist debates.