We elucidate oxygen diffusivity in ThO2-CeO2 and ThO2-UO2 solid solutions across their whole concentration ranges in the phase diagram using static pair-potential calculations and molecular dynamics simulations. Between pure CeO2 (and UO2) and pure ThO2, oxygen diffusivity is higher in CeO2 (and UO2) due to lower oxygen migration barriers. With the addition of Th to CeO2 (and UO2) in the phase diagram, the diffusivity decreases due to the increase in the migration barriers introduced by a larger ionic radius of Th. On the other side of the phase diagram, with the addition of Ce to ThO2 oxygen diffusion decreases due to oxygen vacancy binding with Ce, even though the migration barriers decrease due to the smaller size of Ce than the host Th. Using these calculations, we provide a schematic of high oxygen diffusivity regions in the phase diagram. We also compare the impact of tetravalent dopants (e.g. actinides) on oxygen vacancy energetics to that of trivalent dopants (e.g. lanthanides). We find that trivalent dopants bind much more strongly with oxygen vacancy than the tetravalent dopants. We also find that the tetravalent dopants that have larger radii than the host cation have negative oxygen vacancy binding energy, whereas all trivalent dopants have positive binding energy irrespective of their ionic radii. This work thus highlights key differences in the oxygen vacancy energetics between the trivalent and tetravalent cations.