Raising the speed and accuracy of displacement retrieval by overcoming the weakness of time-consuming parameter estimation and a complex displacement retrieval algorithm is of key importance in improving the real-time quality of optical feedback self-mixing-based displacement measurement sensor. This paper proposes a high-speed and accurate simultaneous estimation method of displacement, optical feedback factor, and linewidth enhancement factor based on data fitting near a half-fringe cooperating with the fringe-counting method. The displacement measurement accuracy of the proposed method is about 16 nm. The high estimation speed ensured by the principle of the reported method constitutes the main character of the method; besides, the method can be applied to any motion, as well as a harmonic vibration.