We have performed the measurement of the site-specific phonon densities of states (PDOS) discerned using electronic states. As far as we know, no general method could give the site-specific PDOS, although oscillating properties of the individual atoms in nonequivalent positions are not necessarily equivalent. However, the combination of the energy and time domain measurements of the nuclear resonant scattering of synchrotron radiation allows the identification of site-specific PDOS. We measured the site-specific PDOS of iron atoms in magnetite, which is a mixed valent compound, and the difference between partial phonon densities of the iron sites was clearly observed.