Multiple energy scales contribute to the radiative properties of colloidal quantum dots, including magnetic interactions, crystal field splitting, Pauli exclusion, and phonons. Identification of the exact physical mechanism which couples first to the dark ground state of colloidal quantum dots, inducing a significant reduction in the radiative lifetime at low temperatures, has thus been under significant debate. Here we present measurements of this phenomenon on a variety of materials as well as on colloidal heterostructures. These show unambiguously that the dominant mechanism is coupling of the ground state to a confined acoustic phonon, and that this mechanism is universal.