The gain of erbium-doped fiber amplifiers is damaged by irradiation partly because of creation of color centers responsible of excess absorption at pump and signal wavelengths. Based on the combination of thermally stimulated luminescence and spectrophotometry, this Letter demonstrates that a part of the gain loss should be associated with the reduction of the density of Er3+ ions by irradiation.