We have utilized photoluminescence spectroscopy and optical ellipsometry to characterize the dose-dependence of the photoluminescence emission intensity and changes in optical absorption of thoria single crystals subject to irradiation with energetic protons at room- and elevated-temperatures. The photoluminescence peaks and the optical absorption bands are attributed to creation of new electronic states emerging from defects resulting from displacement damage. These bands are most likely associated with electrons trapped at the oxygen vacancy sites similar to color centers formed in other irradiated oxides and halides. Our experimental observations are supported by a standard density functional theory calculation of the electronic structure in pristine and oxygen vacancy-bearing thoria crystals. The dose-dependence of the intensity of the photoluminescence peaks is used to parameterize a rate theory model that estimates the concentration of color centers in the irradiated crystals. This parameterization provides optimized migration barrier parameters for oxygen interstitials and vacancies that simultaneously capture the optical response of the crystals irradiated at room- and elevated-temperature. These optical spectroscopy techniques offer a promising pathway to characterize the population of color centers formed at the sites of oxygen anion vacancies, particularly in irradiated nuclear fuels, where atomic-level defects cannot be readily imaged using electron microscopy. When combined with other direct and indirect characterization tools, our approach can provide new insight into defect formation and accumulation in energy materials over single atomic to extended length scales.