Several studies have shown that the motor system is involved in action perception, suggesting that action concepts are represented through sensory-motor processes. Such conclusions imply that motor system impairments should diminish action perception. To test this hypothesis, a group of 10 brain-damaged patients with hemiplegia (specifically, a lesion at the motor system that affected the contralesional arm) viewed point-light displays of arm gestures and attempted to name each gesture. To create the dynamic stimuli, patients individually performed simple gestures with their unaffected arm while being videotaped. The videotapes were converted into point-light animations. Each action was presented as it had been performed, that is, as having been produced by the observer's unaffected arm, and in its mirror reversed orientation, that is, as having been produced by the observer's hemiplegic arm. Action recognition accuracy by patients with hemiplegia was compared with that by 8 brain-damaged patients without any motor deficit and by 10 healthy controls. Overall, performance was better in control observers than in patients. Most importantly, performance by hemiplegic patients, but not by nonhemiplegic patients and controls, varied systematically as a function of the observed limb. Action recognition was best when hemiplegic patients viewed actions that appeared to have been performed by their unaffected arm. Action recognition performance dropped significantly when hemiplegic patients viewed actions that appeared to have been produced with their hemiplegic arm or the corresponding arm of another person. The results of a control study involving the recognition of point-light defined animals in motion indicate that a generic deficit to visual and cognitive functions cannot account for this laterality-specific deficit in action recognition. Taken together, these results suggest that motor cortex impairment decreases visual sensitivity to human action. Specifically, when a cortical lesion renders an observer incapable of performing an observed action, action perception is compromised, possibly by a failure to map the observed action onto the observer's contralesional hemisoma.