The evolution of local ferroelectric lattice distortions in multiferroic Ge1-x Mn x Te is studied by x-ray diffraction, x-ray absorption spectroscopy and density functional theory. We show that the anion/cation displacements smoothly decrease with increasing Mn content, thereby reducing the ferroelectric transition from 700 to 100 K at x = 0.5, where the ferromagnetic Curie temperature reaches its maximum. First principles calculations explain this quenching by different local bond contributions of the Mn 3d shell compared to the Ge 4s shell in excellent quantitative agreement with the experiments.