Evaluating the relative importance of ecological drivers responsible for natural population fluctuations in size is challenging. Longitudinal studies where most individuals are monitored from birth to death and where environmental conditions are known provide a valuable resource to characterize complex ecological interactions. We used a recently developed approach to decompose the observed fluctuation in population growth of the red deer population on the Isle of Rum into contributions from climate, density and their interaction and to quantify their relative importance. We also quantified the contribution of individual covariates, including phenotypic and life-history traits, to population growth. Fluctuations in composition in age and sex classes ((st)age structure) of the population contributed substantially to the population dynamics. Density, climate, birth weight and reproductive status contributed less and approximately equally to the population growth. Our results support the contention that fluctuations in the population's (st)age structure have important consequences for population dynamics and underline the importance of including information on population composition to understand the effect of human-driven changes on population performance of long-lived species.