The tough and self-healing hydrogels composed of polyampholytes (PA gels) are drawing great attention due to their multiscale structures and the resultant multiple mechanical properties. This work studies the stress relaxation behavior of PA gels and reveals the underlying multiscale structure evolutions by combining birefringence and small-angle X-ray scattering measurements. The PA gels show a fast and strong stress relaxation that obeys the stress-optical rule, which could be associated with relaxation of chain segment orientation by the breaking of ionic bonds. A slow and weak relaxation of phase structure (∼100 nm) is also observed, which tells that the stress redistributes and local strain amplification gradually builds in the phase network at long relaxation times as a result of synergetic breaking of multiple ionic bonds. This work gives insight into exploring the formation of the crack precursor that is important in the fracture and fatigue of self-healing hydrogels.