The precise measurement of the distance of fast laterally moving rough surfaces is important in several applications such as lathe monitoring. A nonincremental interferometer based on two tilted interference fringe systems and a precise phase-difference estimation has been realized for this task. However, due to the speckle effect, the two scattered light signals exhibit different phase jumps and random envelopes causing small correlation coefficients and high uncertainties of the phase difference as well as the distance. In this Letter we present for the first time a method to enhance the signal correlation coefficient significantly. The interference signals are generated by scattered light of a rough surface from two different directions. A matching of illumination and receiving optic is performed. By this novel method, distance measurements with an uncertainty down to 1.2 μm at about 10 m/s lateral moving velocity have been achieved. Together with the simultaneously measured lateral velocity, the shape of rotating objects can be precisely determined.