Plant diversity drives changes in the soil microbial community which may result in alterations in ecosystem functions. However, the governing factors between the composition of soil microbial communities and plant diversity are not well understood. We investigated the impact of plant diversity (plant species richness and functional group richness) and plant functional group identity on soil microbial biomass and soil microbial community structure in experimental grassland ecosystems. Total microbial biomass and community structure were determined by phospholipid fatty acid (PLFA) analysis. The diversity gradient covered 1, 2, 4, 8, 16 and 60 plant species and 1, 2, 3 and 4 plant functional groups (grasses, legumes, small herbs and tall herbs). In May 2007, soil samples were taken from experimental plots and from nearby fields and meadows. Beside soil texture, plant species richness was the main driver of soil microbial biomass. Structural equation modeling revealed that the positive plant diversity effect was mainly mediated by higher leaf area index resulting in higher soil moisture in the top soil layer. The fungal-to-bacterial biomass ratio was positively affected by plant functional group richness and negatively by the presence of legumes. Bacteria were more closely related to abiotic differences caused by plant diversity, while fungi were more affected by plant-derived organic matter inputs. We found diverse plant communities promoted faster transition of soil microbial communities typical for arable land towards grassland communities. Although some mechanisms underlying the plant diversity effect on soil microorganisms could be identified, future studies have to determine plant traits shaping soil microbial community structure. We suspect differences in root traits among different plant communities, such as root turnover rates and chemical composition of root exudates, to structure soil microbial communities.