Accurately lateral displacement measurement is essential for a vast of non-contact sensing technologies. Here, we introduce a high-precision lateral displacement measurement method based on differential wavefront sensing (DWS). Compared to the conventional differential power sensing (DPS) method, the DWS method based on phase readout has the potential to achieve a higher resolution. The beam lateral displacement can be obtained by the curvature distribution of the wavefront on the surface of the detector. According to the theoretical model of the DWS method, the sensitivity of the lateral displacement can be greatly improved by increasing the wavefront curvature of the measured laser beam by means of lenses. An optical system for measuring the lateral displacement of the laser beam is built and calibrated by a high-precision hexapod. The experimental results show that the DWS-based lateral displacement measurement achieves a resolution of 40 pm/Hz1/2 (at 1-10 Hz) with a linear range of about 40 µm, which is consistent with the theoretical model. This technique can be applied to high-precision multi-degree-of-freedom interferometers.