Fossil abundance data can reveal ecological dynamics underpinning taxonomic declines. Using fossil dental metrics, we reconstructed body mass and mass-abundance distributions in Late Miocene to recent African large mammal communities. Despite collection biases, fossil and extant mass-abundance distributions are highly similar, with unimodal distributions likely reflecting savanna environments. Above 45 kilograms, abundance decreases exponentially with mass, with slopes close to -0.75, as predicted by metabolic scaling. Furthermore, communities before ~4 million years ago had considerably more large-sized individuals, with a greater proportion of total biomass allocated in larger size categories, than did later communities. Over time, individuals and biomass were redistributed into smaller size categories, reflecting a gradual loss of large-sized individuals from the fossil record paralleling the long-term decline of Plio-Pleistocene large mammal diversity.