Mitochondrial dysfunction is a prominent hallmark of Alzheimer's disease (AD). Mitochondrial DNA (mtDNA) damage may be a major cause of abnormal reactive oxidative species production in AD or increased neuronal susceptibility to oxidative injury during aging. The purpose of this study was to assess the influence of mtDNA sequence variation on clinically significant cognitive impairment and dementia risk in the population-based Health, Aging, and Body Composition (Health ABC) Study. We first investigated the role of common mtDNA haplogroups and individual variants on dementia risk and 8-year change on the Modified Mini-Mental State Examination (3MS) and Digit Symbol Substitution Test (DSST) among 1,631 participants of European genetic ancestry. Participants were free of dementia at baseline and incidence was determined in 273 cases from hospital and medication records over 10-12 follow-up years. Participants from haplogroup T had a statistically significant increased risk of developing dementia (OR = 1.86, 95% CI = 1.23, 2.82, p = 0.0008) and haplogroup J participants experienced a statistically significant 8-year decline in 3MS (β = -0.14, 95% CI = -0.27, -0.03, p = 0.0006), both compared with common haplogroup H. The m.15244A>G, p.G166G, CytB variant was associated with a significant decline in DSST score (β = -0.58, 95% CI -0.89, -0.28, p = 0.00019) and the m.14178T>C, p.I166V, ND6 variant was associated with a significant decline in 3MS score (β = -0.87, 95% CI -1.31, -3.86, p = 0.00012). Finally, we sequenced the complete ~16.5 kb mtDNA from 135 Health ABC participants and identified several highly conserved and potentially functional nonsynonymous variants unique to 22 dementia cases and aggregate sequence variation across the hypervariable 2-3 regions that influences 3MS and DSST scores.