With well-known speckle measurement techniques, the root mean square height as well as the autocorrelation length of isotropic surfaces can be determined quickly and over a large area of interest. Beyond that, the present article studies the speckle-based measurement of anisotropic surfaces. For this purpose, a measurement setup and evaluation algorithm are presented that enable the characterization of unidirectionally anisotropic surfaces machined by grinding. As a result, four measurands are obtained from one speckle image: the machining direction, the autocorrelation length perpendicular to the machining direction, as well as two root mean square roughness parameters parallel and perpendicular to the machining direction. The first two measurands are obtained from a two-dimensional fast Fourier transform of the diffraction pattern resulting from the unidirectional tool marks and the latter two by a bidirectional evaluation of the speckle contrast. In addition to measurements on physical reference samples, a spatial light modulator is used to create a large number of surface topographies with known model parameters in order to quantify the measurement uncertainty.