Quantifying the drivers underlying the distribution of biodiversity during succession is a critical issue in ecology and conservation, and also can provide insights into the mechanisms of community assembly. Ninety plots were established in the Loess Plateau region of northern Shaanxi in China. The taxonomic and phylogenetic (alpha and beta) diversity were quantified within six succession stages. Null models were used to test whether phylogenetic distance observed differed from random expectations. Taxonomic beta diversity did not show a regular pattern, while phylogenetic beta diversity decreased throughout succession. The shrub stage occurred as a transition from phylogenetic overdispersion to clustering either for NRI (Net Relatedness Index) or betaNRI. The betaNTI (Nearest Taxon Index) values for early stages were on average phylogenetically random, but for the betaNRI analyses, these stages were phylogenetically overdispersed. Assembly of woody plants differed from that of herbaceous plants during late community succession. We suggest that deterministic and stochastic processes respectively play a role in different aspects of community phylogenetic structure for early succession stage, and that community composition of late succession stage is governed by a deterministic process. In conclusion, the long-lasting evolutionary imprints on the present-day composition of communities arrayed along the succession gradient.