Chemical and energetic interactions between broadband infrared intrinsic emission centers (IECs) of bismuthates and extrinsic emission centers (EECs) of Nd2O3 dopants were optically and electronically investigated. Although no visible absorption from the IEC was found in untreated Bi2O3-B2O3 glass, it was clearly observed after a moderate thermal treatment of <200 °C, indicating chemical activity of O-deficient sites as the origin of IECs. On the other hand, Nd2O3 doping chemically stabilized the Bi2O3-B2O3 glass and suppressed IEC formation. By using a microwave measurement sensitive to electric dipoles, we found a 'switching' in local energy balance resulting from the Nd2O3 doping. This was explained by metallization of the O-deficient sites in the Bi2O3-B2O3 glass and multi-phonon excitation of IEC and EEC complexes in the Nd2O3-Bi2O3-B2O3 glass phosphor. Although the electric dipole observed by the microwave measurement was not necessarily caused by IEC, emission properties of the IEC and EEC complexes were consistent with energy balance switching; emissions from IECs after thermal treatment were quenched by EECs with multi-phonon excitation.