Using a combination of experimental techniques such as optical absorption, Raman scattering, continuous wave and pulse Electron Spin Resonance (ESR), we characterize a set of γ-irradiated Yb(3+) doped silica glass preforms with different contents of phosphorous and aluminum. We demonstrate that when P is introduced in excess compared to Al, nearly no radiodarkening is induced by γ-rays. On the other hand, when Al>P, a large absorption band is induced by radiation. Thermal annealing experiments reveal the correlation between the decrease of the optical absorption band and the decrease of the Al-Oxygen Hole Center (AlOHC) ESR signal, demonstrating the main role of AlOHC defects in the fiber darkening. HYSCORE (HYperfine Sublevel CORElation) pulse-ESR experiments show a high Al-P nuclear spin coupling when P>Al and no coupling when Al>P. This result suggests that both AlOHC and POHC creation is inhibited by Al-O-P linkages. Confronting our data with previous works, we show that the well-known photodarkening process, meaning losses induced by the IR pump, can also be explained in this framework.