Cavity quantum electrodynamics (CQED) effects, such as Rabi splitting, Rabi oscillations, and superradiance, have been demonstrated with nitrogen vacancy (NV) center spins in diamond coupled to microwave resonators at cryogenic temperature. In this Letter, we explore the possibility to realize strong collective coupling and CQED effects with ensembles of NV spins at room temperature. Our calculations show that thermal excitation of the individual NV spins leads to population of collective Dicke states with low symmetry and a reduced collective coupling to the microwave resonators. Optical pumping can be applied to counteract the thermal excitation of the NV centers and to prepare the spin ensemble in Dicke states with high symmetry. The resulting strong coupling with high-quality resonators enables the study of intriguing CQED effects across the weak-to-strong coupling regime, and may have applications in quantum sensing and quantum information processing.