We propose low-coherence wavelength shifting interferometry and demonstrate its application to quantitative phase imaging of dynamic specimens. By shifting the source wavelength, multiple interferograms of the sample can be acquired at different spectral bands. A sample phase is thus encoded in the phase step between consecutive acquisitions. For the particular case of four-band imaging, we show that the phase can be extracted with a modified Carré algorithm. We describe signal demodulation in detail and discuss its implication on system implementation. A swept laser-based Mach-Zehnder interferometer is used to demonstrate the technique for real-time imaging of live sperm cells at 62.5 Hz. The dynamic dry mass of the sperm head is measured with a full-scale error of ±2%, validating the technique's capability for high-sensitivity, high-speed quantitative phase imaging.