An active and diverse soil biota is important for maintaining crop productivity and quality, and preservation of these traits is a major goal of sustainable farming. This study aimed at unravelling the impact of different management practices on soil fungal and bacterial biodiversity in vineyards as a model for permanent crops. Species diversity was assessed using an amplicon sequencing approach in a long-term field experiment in the Rheingau wine region of Germany where integrated, organic and biodynamic management practices had been in place for 10 years. Fungal community composition under integrated management differed significantly from organic and biodynamic management, whereas fungal species richness remained unaffected. Soil under integrated management had a significantly reduced bacterial species richness compared to organic, but community composition was similar to organically and biodynamically managed soils. Highest fungal richness was obtained under cover crop between rows in topsoil, arising from cover cropping and organic carbon supply.