The association between HIV-associated neurocognitive impairment (NCI) and health-related quality of life (HRQoL) is not well known. We investigated this association among the CNS (Central Nervous System) HIV Antiretroviral Therapy Effects Research (CHARTER) study participants. We performed factor analysis to distinguish physical and mental HRQoL, followed by general linear models. We analyzed 1,340 HIV participants, including 35.6% with NCI, 77.2% males, 70.5% unemployed, and 42.2% with depression. Impaired participants had lower (worse) mental and physical HRQoL mean scores compared to unimpaired participants. NCI was negatively associated with mental HRQoL in crude (mean difference: -4.38; 95% CI: -6.70 to -2.06) and adjusted analysis (-2.56, -4.83 to -0.30). NCI was also negatively associated with physical HRQoL in unadjusted analysis (-4.62, -7.45 to -1.78), though the association weakened in the adjusted analysis (-2.20, -4.81 to 0.40). The association between NCI and HRQoL was confounded mainly by employment and was partially mediated by depression. These findings suggest that future strategies aimed at improving HRQoL among HIV-infected patients with NCI might benefit from concurrent management of depression.