Plausible mechanisms of how vitamin D deficiency may contribute to the development of diabetes mellitus have been proposed but longitudinal cohort studies have yielded heterogeneous results. In 7,791 initially diabetes-free participants of a German population-based cohort, aged 50-74 years, adjusted Cox regression models were employed to estimate hazard ratios (HR) with 95 % confidence intervals (CI) for the association of serum 25-hydroxyvitamin D (25(OH)D) quintiles and incident diabetes. Dose-response relationships were assessed with restricted cubic spline curves. Additionally, analyses accounting for the competing risks of diabetes and death were performed. During 8 years of follow-up, 829 study participants developed diabetes. In women, diabetes risk was significantly increased in the lowest 25(OH)D quintile (HR, 1.38; 1.09-1.75) and non-significantly increased in the 2nd quintile (HR, 1.24; 0.98-1.55) compared to women in 25(OH)D quintiles 3-5. The dose-response relationship showed a non-linear inverse association with risk starting to increase at 25(OH)D levels below 70 nmol/L (statistically significant: below 40 nmol/L). In men, 25(OH)D levels were not associated with diabetes incidence. Renal dysfunction was an effect modifier with a more than doubled diabetes risk in 25(OH)D quintile 1 and an about 1.5-fold risk in quintile 2 compared to quintiles 3-5 if subjects had renal dysfunction. The observed associations were not influenced by the competing risk of death. In this large cohort study of older adults, serum 25(OH)D levels were inversely associated with incident diabetes in women but not in men. The association was particularly strong in subjects with renal dysfunction.