Microbial communities in roots and shoots of plants and in soil are important for plant growth and health and take part in important ecosystem processes. Therefore, understanding the factors that affect their diversity is important. We have analyzed fungal and bacterial communities associated with plant shoots, roots and soil over a 1 km2 area in a semi-natural temperate grassland with 1-43 plant species per 0.1 m2, to describe the relationships between plant and microbial diversity and to identify the drivers of bacterial and fungal community composition. Microbial community composition differed between shoots, roots and soil. While both fungal and bacterial species richness in shoots increased with plant species richness, no correlation was found between plant and microbial diversity in roots and soil. Chemistry was a significant predictor of bacterial and fungal community composition in soil as was also the spatial location of the sampled site. In this species-rich grassland, the effects of plants on the microbiome composition seemed to be restricted to the shoot-associated taxa; in contrast, the microbiomes of roots or soil were not affected. The results support our hypothesis that the effect of plants on the microbiome composition decreases from shoots to roots and soil.