We investigate conditions under which a model with stochastic demography or population structure converges to the coalescent with a linear change in timescale. We argue that this is a necessary condition for the existence of a meaningful effective population size. We find that such a linear timescale change is obtained when demographic fluctuations and coalescence events occur on different timescales. Simple models of population structure and randomly fluctuating population size are used to exemplify the ideas and provide an intuitive feel for the meaning of the conditions.