An ultrafast and Doppler-free optical ranging system based on dispersive frequency-modulated interferometry is demonstrated. The principle is similar to the conventional frequency-modulated continuous-wave interferometry where the range information is derived from the beat frequency between the object signal and the reference signal. However, a passive and static frequency scanning is performed based on the chromatic dispersion of a transform-limited femtosecond pulse in the time domain. We point out that the unbalanced dispersion introduced in the Mach-Zehnder interferometer can be optimized to eliminate the frequency chirp in the temporal interferograms pertaining to the third order dispersion of the all-fiber system, if the dynamic range being considered is small. Some negative factors, such as the polarization instability of the femtosecond pulse, the power fluctuation of the optical signal and the nonuniform gain spectrum of the erbium-doped fiber amplifier lead to an obvious envelope deformation of the temporal interferograms from the Gaussian shape. Thus a new data processing method is proposed to guarantee the range resolution. In the experiment, the vibration of a speaker is measured. A range resolution of 1.59 microm is achieved with an exposure time of 394 fs at a sampling rate of 48.6 MHz.