In developed countries, the elderly population is growing rapidly. Self-rated health is an important indicator of health and has been the focus of a number of aging studies. However, little is known about the possible differences in determinants of self-rated health according to age. Self-rated health and its possible determinants were examined in 542 community-dwelling Japanese women aged 40 to 91 years. The proportion of women with poor self-rated health increased with age. Approximately half of the women had poor self-rated health among women aged 80 years and over. We used multiple logistic regression analysis to examine the association with poor self-rated health in women aged 40 to 64 years (n = 248) and 65 years and over (n = 294). Lower physical activity and a greater number of comorbidities were associated with poor self-rated health for ages 40 to 64 and 65 years and over. Furthermore, the risk of poor self-rated health increased by 1.5-fold with every increase in the number of painful joints in the lower extremities for the 40- to 64-year-old age group, and increased by 2-fold with back pain for those 65 years and over. The determinants of self-rated health were similar in both middle-aged and elderly Japanese women. Our results could contribute to effective interventions to improve quality of life among Japanese women.