Impairments in visual motion perception and use of visual motion information to guide behavior have been reported in autism, but the brain alterations underlying these abnormalities are not well characterized. We performed functional magnetic resonance imaging (fMRI) studies to investigate neural correlates of impairments related to visual motion processing. Sixteen high-functioning individuals with autism and 14 age and IQ-matched typically developing individuals completed two fMRI tasks using passive viewing to examine bottom-up responses to visual motion and visual pursuit tracking to assess top-down modulation of visual motion processing during sensorimotor control. The autism group showed greater activation and faster hemodynamic decay in V5 during the passive viewing task and reduced frontal and V5 activation during visual pursuit. The observations of increased V5 activation and its faster decay during passive viewing suggest alterations in local V5 circuitries that may be associated with reduced GABAergic tone and inhibitory modulation. Reduced frontal and V5 activation during active pursuit suggest reduced top-down modulation of sensory processing. These results suggest that both local intrinsic abnormalities in V5 and more widely distributed network level abnormalities are associated with visual motion processing in autism.