Perception of visual motion includes a first-order mechanism sensitive to luminance changes and a second-order motion mechanism sensitive to contrast changes. We studied neural substrates for these motion types in 142 subjects with visual cortex lesions, 68 normal controls and 28 brain lesion controls. On first-order motion, the visual cortex lesion group performed significantly worse than normal controls overall and in each hemifield, but second-order motion did not differ. Only one individual showed a selective second-order motion deficit. Motion deficits were seen with lesions outside the small occipito-temporal region thought to contain a human homolog of motion processing area MT (V5), suggesting that many areas of human brain process visual motion.