Cerebral small vessel disease (CSVD) occurs often in memory clinic patients. Apart from cognitive deficits, these patients can express physical decline, which predicts adverse health outcomes. In this study, we investigated the cooccurrence of clinically relevant impairments in physical performance and CSVD in memory clinic patients. We included 131 patients with vascular brain injury, mild cognitive impairment or Alzheimer disease with available 3T MRI and physical performance scores. CSVD was visually rated according to 3 subtypes and as a total burden score, composed of the presence of white matter hyperintensities (WMH), lacunar infarcts (LI), and cerebral microbleeds (MB). Physical performance was assessed with the Short Physical Performance Battery (SPPB), covering gait speed, balance, and chair stand performance. CSVD markers and impaired physical performance both occurred often. High total CSVD burdens cooccurred with impaired chair stand performances [odds ratio (OR) 2.67; 95% confidence interval (CI) (1.12-6.34)]. WMH cooccurred with impaired SPPB scores (OR, 3.76; 95% CI, 1.68-8.44), impaired gait speeds (OR, 4.11; 95% CI, 1.81-9.31) and impaired chair stand performances (OR, 5.62; 95% CI, 2.29-13.80). In memory clinic patients, high burdens of CSVD, particularly WMH, often cooccur with impairments in physical performance. The presence of WMH should alert clinicians to the presence of these, clinically relevant, physical impairments.