The structural and magnetic properties of ultrathin FeO(111) films on Pt(111) with thicknesses from 1 to 16 monolayers (MLs) were studied using the nuclear inelastic scattering of synchrotron radiation. A distinct evolution of vibrational characteristics with thickness, revealed in the phonon density of states (PDOS), shows a textbook transition from 2D to 3D lattice dynamics. For the thinnest films of 1 and 2 ML, the low-energy part of the PDOS followed a linear ∝E dependence in energy that is characteristic for two-dimensional systems. This dependence gradually transforms with thickness to the bulk ∝E^{2} relationship. Density-functional theory phonon calculations perfectly reproduced the measured 1-ML PDOS within a simple model of a pseudomorphic FeO/Pt(111) interface. The calculations show that the 2D PDOS character is due to a weak coupling of the FeO film to the Pt(111) substrate. The evolution of the vibrational properties with an increasing thickness is closely related to a transient long-range magnetic order and stabilization of an unusual structural phase.