In this paper, a systematic study of the decay times of the spin-enabled and spin-forbidden transitions of Yb2+ doped into the halidoperovskites CsMX3 (M = Ca, Sr; X = Cl, Br, I) is presented. The spin-forbidden transitions are characterized by ms decay times, which are typical for Yb2+. On the contrary, the spin-enabled transitions show much shorter decay times in the range of μs and have so far only been rarely observed. These results allow detailed conclusions about systematics of the decay times of Yb2+ doped in similar compounds and their correlation to the local structure of the coordination sphere of Yb2+ as well as the role of vibrational interaction between the excited high spin (HS) and low spin (LS) states. The halidoperovskites are ideally suited as host lattices in this context and may work as text book examples due to their comparable structures, which allows a detailed interpretation of the decay times in relation to the local structure. An understanding of the impact of the composition and structure of the host material on the decay times of Yb2+ will be of relevance for future applications of this activator in scintillators or lighting materials.