In phase-shifting fringe projection profilometry, fringe harmonics caused by device nonlinearities as well as other factors may badly ruin the measurement results. Generally, the used phase-shifting algorithm enables restraint of effects of harmonics below a certain order depending on the number of phase shifts. When reducing the number of phase shifts for efficiency, high order harmonics will affect the phase-measuring results because of aliasing caused by insufficient sampling rate. To overcome this issue, this paper suggests a non-filtering technique operating in frequency domain, that enables improvement of measurement accuracy by eliminating effects of high order harmonics. With this technique, the phase-shifting algorithm is restated as a process of retrieving the fundamental complex fringes from the phase-shifted fringe patterns. Implementing a Fourier transform to this calculated complex fringe pattern, the actual fundamental signals and the aliased harmonics have their own lobes with separated peaks in the frequency domain. We reconstruct each order of the aliased harmonics by exploiting their relations with the fundamental signals and then estimate their magnitudes by using the spectral peaks. Instead of directly filtering the fringe spectrum, we subtract spectra of the harmonics from Fourier transform of the just calculated complex fringes, so that the Fourier spectrum of the fundamental fringes without harmonics is recovered through an iterative operation. Further, the phase map is measured accurately. Simulation and experimental results confirm that this proposed method can significantly suppress effects of fringe harmonics. Meanwhile, by taking advantage of non-filtering, it effectively preserves the edges and details of the measured surfaces from being blurred.