We previously reported that pathologic measures of arteriosclerosis (AS), cerebral infarction, and Alzheimer’s disease (AD) are independently correlated with cortical gray matter (CGM) atrophy measured by in vivo magnetic resonance imaging (MRI). Here, we use path analyses to model the associations between these three pathology measures and cognitive impairment, as mediated by CGM atrophy, after controlling for age and education. In this sample of 116 elderly persons followed longitudinally to autopsy (ischemic vascular disease (IVD) program project), differential patterns were observed between AS and atrophy/cognition versus AD and atrophy/cognition. The total effect of AD pathology on global cognition (β = -0.61, s.e. = 0.06) was four times stronger than that of AS (β = -0.15, s.e. = 0.08). The effect of AS on cognition appears to occur through cerebral infarction and CGM atrophy (β = -0.13, s.e. = 0.04). In contrast, the effects of AD pathology on global cognition (β = -0.50, s.e. = 0.07) occur through a direct pathway that is five times stronger than the indirect pathway acting through CGM atrophy (β = -0.09, s.e. = 0.03). The strength of this direct AD pathway was not significantly mitigated by adding hippocampal volume to the model. AD pathology affects cognition not only through brain atrophy, but also via an unmeasured pathway that could be related to synaptic dysfunction before the development of cortical atrophy.