The local crystal structures of many perovskite-structured materials deviate from the average space-group symmetry. We demonstrate, from lattice-dynamics calculations based on quantum chemical force constants, that all of the cesium-lead and cesium-tin halide perovskites exhibit vibrational instabilities associated with octahedral titling in their high-temperature cubic phase. Anharmonic double-well potentials are found for zone-boundary phonon modes in all compounds with barriers ranging from 108 to 512 meV. The well depth is correlated with the tolerance factor and the chemistry of the composition, but is not proportional to the imaginary harmonic phonon frequency. We provide quantitative insights into the thermodynamic driving forces and distinguish between dynamic and static disorder based on the potential-energy landscape. A positive band gap deformation (spectral blue shift) accompanies the structural distortion, with implications for understanding the performance of these materials in applications areas including solar cells and light-emitting diodes.