The primate visual system is arranged hierarchically, starting from the retina and continuing through a series of extrastriate visual areas. Selectivity for motion is first found in individual neurons in the primate visual cortex (V1), in which many simple cells respond selectively to the direction and speed of moving stimuli. Beyond simple cells, most studies of direction selectivity have focused on either V1 complex cells or neurons in the middle temporal area (MT/V5). To understand how visual information is transferred along this pathway, we have studied all three types of neurons, using a reverse correlation procedure to obtain high spatial and temporal resolution maps of activity for different motion stimuli. Most complex and MT cells showed strong second-order interactions, indicating that they were tuned for particular displacements of an apparent motion stimulus. The spatiotemporal structure of these interactions showed a high degree of similarity between the populations of V1 complex cells and MT cells, in terms of the spatiotemporal limits and preferences for motion and their two-dimensional spatial structure. Much of the structure in the V1 and MT second-order kernels could be accounted for on the basis of the first-order responses of V1 simple cells, under the assumption of a Reichardt or motion-energy type of computation.