We investigate the phase noise performance of optoelectronic oscillators when the optical energy storage elements are in the following three configurations: a high-Q whispering gallery mode resonator, an optical delay-line and a combination of both elements. The stability properties of these various optical elements are first characterized, and then systematically compared in the optical and in the microwave frequency domains. Subsequently, the spectral purity of the oscillator is theoretically and experimentally examined for each case. When the resonator is used as both delay and filtering element inside the delay-line based oscillator, the generated spurious modes are highly rejected. A spur rejection by more than 53 dB has been demonstrated for the first-neighboring spur.