The evolution of cooperation has gained more attention after Smith introduced game theory in the study of evolutionary biology. Subsequent works have extensively explained this phenomenon, consistently showing the importance of spatial structure for the evolution of cooperation. Here we analyze the effect of stochasticity on the evolution of cooperation in group-structured populations. We find a simple formula for the fixation probability of cooperators and show that cooperation can be favored by selection if a condition similar to Hamilton's rule is satisfied, which is also valid for strong selection and high migration. In fact, cooperation can be favored even in the absence of population viscosity and in the limit of an infinite number of finite-size groups. We discuss the importance of stochastic fluctuations in helping cooperation. We argue that this may be a general principle because fluctuations favoring the cooperators are often much more impactful than those favoring the defectors.