The effect of stroke topography on the recovery of hemiparetic upper limb (HUL) function is unclear due to limitations in previous studies-examination of lesion effects only in one point of time, or grouping together patients with left and right hemispheric damage (LHD, RHD), or disregard to different lesion impact on proximal and distal operations. Here we used voxel-based lesion symptom mapping (VLSM) to investigate the impact of stroke topography on HUL function taking into consideration the effects of (a) assessment time (subacute, chronic phases), (b) side of damaged hemisphere (left, right), (c) HUL part (proximal, distal). HUL function was examined in 3 groups of patients-Subacute (n = 130), Chronic (n = 66), and Delta (n = 49; patients examined both in the subacute and chronic phases)-using the proximal and distal sub-divisions of the Fugl-Meyer (FM) and the Box and Blocks (B&B) tests. HUL function following LHD tended to be affected in the subacute phase mainly by damage to white matter tracts, the putamen and the insula. In the chronic phase, a similar pattern was shown for B&B performance, whereas FM performance was affected by damage only to the white matter tracts. HUL function following RHD was affected in both phases, mainly by damage to the basal ganglia, white matter tracts and the insula, along with a restricted effect of damage to other cortical structures. In the chronic phase HUL function following RHD was affected also by damage to the thalamus. In the small Delta groups the following trends were found: In LHD patients, delayed motor recovery, captured by the B&B test, was affected by damage to the sensory-motor cortex, white matter association fibers and parts of the perisilvian cortex. In the RHD patients of the Delta group, delayed motor recovery was affected by damage to white matter projection fibers. Proximal and distal HUL functions examined in LHD patients (both in the subacute and chronic phases) tended to be affected by similar structures-mainly white matter projection tracts. In RHD patients, a distinction between proximal and distal HUL functions was found in the subacute but not in the chronic phase, with proximal and distal HUL functions affected by similar subcortical and cortical structures, except for an additional impact of damage to the superior temporal cortex and the retro-lenticular internal capsule only on proximal HUL function. The current study suggests the existence of important differences between the functional neuroanatomy underlying motor recovery following left and right hemisphere damage. A trend for different lesion effects was shown for residual proximal and distal HUL motor control. The study corroborates earlier findings showing an effect of the time after stroke onset (subacute, chronic) on the results of VLSM analyses. Further studies with larger sample size are required for the validation of these results.