Cerebral autosomal dominant arteriopathy with subcortical infarcts and leukoencephalopathy (CADASIL) is thought to be a pure genetic form of subcortical vascular cognitive impairment (SVCI). The aim of this study was to compare white matter integrity and cortical thickness between typical CADASIL, a genetic form, and two sporadic forms of SVCI (with NOTCH3 and without NOTCH3 variants). We enrolled typical CADASIL patients (N = 11) and SVCI patients [with NOTCH3 variants (N = 15), without NOTCH3 variants (N = 101)]. To adjust the age difference, which reflects the known difference in clinical and radiologic courses between typical CADASIL patients and SVCI patients, we constructed a W-score of measurement for diffusion tensor image and cortical thickness. Typical CADASIL patients showed more frequent white matter hyperintensities in the bilateral posterior temporal region compared to SVCI patients (p < 0.001, uncorrected). We found that SVCI patients, regardless of the presence of NOTCH3 variants, showed significantly greater microstructural alterations (W-score, p < 0.05, FWE-corrected) and cortical thinning (W-score, p < 0.05, FDR-corrected) than typical CADASIL patients. In this study, typical CADASIL and SVCI showed distinct anatomic vulnerabilities in the cortical and subcortical structures. However, there was no difference between SVCI with NOTCH3 variants and SVCI without NOTCH3 variants.