α-Sb2O4 (cervantite) and β-Sb2O4 (clinocervantite) are mixed valence compounds with equal proportions of SbIII and SbV as represented in the formula SbIIISbVO4. Their structure and properties can be difficult to calculate owing to the SbIII lone-pair electrons. Here, we present a study of the lattice dynamics and vibrational properties using a combination of inelastic neutron scattering, Mössbauer spectroscopy, nuclear inelastic scattering, and density functional theory (DFT) calculations. DFT calculations that account for lone-pair electrons match the experimental densities of phonon states. Mössbauer spectroscopy reveals the β phase to be significantly harder than the α phase. Calculations with O vacancies reveal the possibility for nonstoichiometric proportions of SbIII and SbV in both phases. An open question is what drives the stability of the α phase over the β phase, as the latter shows pronounced kinetic stability and lower symmetry despite being in the high-temperature phase. Since the vibrational entropy difference is small, it is unlikely to stabilize the α phase. Our results suggest that the α phase is more stable only because the material is not fully stoichiometric.