Understanding the factors that influence the distribution of understory vegetation is important for biological conservation and forest management. We compared understory species composition by multi-response permutation procedure and indicator species analysis between plots dominated by Qinghai spruce (Picea crassifolia Kom.) and Qilian juniper (Sabina przewalskii Kom.) in coniferous forests of the Qilian Mountains, northwestern China. Understory species composition differed markedly between the forest types. Many heliophilous species were significantly associated with juniper forest, while only one species was indicative of spruce forest. Using constrained ordination and the variation partitioning model, we quantitatively assessed the relative effects of two sets of explanatory variables on understory species composition. The results showed that topographic variables had higher explanatory power than did site conditions for understory plant distributions. However, a large amount of the variation in understory species composition remained unexplained. Forward selection revealed that understory species distributions were primarily affected by elevation and aspect. Juniper forest had higher species richness and α-diversity and lower β-diversity in the herb layer of the understory plant community than spruce forest, suggesting that the former may be more important in maintaining understory biodiversity and community stability in alpine coniferous forest ecosystems.