The partial density of vibrational states has been measured for Fe in compressed FeO (wüstite) using nuclear resonant inelastic x-ray scattering. Substantial changes have been observed in the overall shape of the density of states close to the magnetic transition around 20 GPa from the paramagnetic (low pressure) to the antiferromagnetic (high pressure) state. The results indicate that strong magnetoelastic coupling in FeO is the driving force behind the changes in the phonon spectrum of FeO. The paper presents the first observation of changes in the density of terahertz acoustic phonon states under magnetic transition at high pressure.