From a single-frame dual-wavelength spatial carrier-frequency interferogram (SCFI), we propose a novel phase retrieval method of dual-wavelength interferometry (DWI). First, by continuously moving the intercepted area pixel-by-pixel in a single-frame SCFI along the horizontal and vertical directions, we construct a sequence of phase-shifting sub-interferograms. Second, the wrapped phases of each single wavelength can be retrieved from those phase-shifting sub-interferograms via the least-squares iteration algorithm. Third, the phase of synthetic wavelength can be obtained by subtraction between the wrapped phases of single wavelengths. Both the numerical simulation and the experimental result demonstrate that the proposed method reveals greater accuracy and convenience. Furthermore, because only single-frame SCFI can perform the phase retrieval of DWI, the proposed method offers better ability in resisting external vibration and disturbance, which will greatly facilitate the application of DWI in the dynamic phase measurement.