We report a novel singularity in the hysteresis of spin glasses, the reversal-field memory effect, which creates a nonanalyticity in the magnetization curves at a particular point related to the history of the sample. The origin of the effect is due to the existence of a macroscopic number of "symmetric clusters" of spins associated with a local spin-reversal symmetry of the Hamiltonian. We use first order reversal curve (FORC) diagrams to characterize the effect and compare to experimental results on thin magnetic films. We contrast our results on spin glasses to random magnets and show that the FORC technique is an effective "magnetic fingerprinting" tool.