Currently, the growing population of the elderly is one of biggest problems in terms of increase in geriatric diseases. Lack of data from large prospective studies on geriatric breast cancer patients often makes it difficult for clinicians to make treatments decisions for them. Because both benefit and risk of treatment should be taken into account, treatment is usually determined considering life expectancy or comorbidities in elderly patients. Treatment of breast cancer is differentiated according to histologic classifications, and hormone therapy is even adopted for patients with metastatic breast cancer if tumor tissue expresses hormone receptors. Endocrine therapy can offer great benefit to elderly patients considering its equivalent efficacy to chemotherapy with fewer toxicities if it is appropriately used. Aromatase inhibitors are usually prescribed agents in hormone therapy for elderly breast cancer patients due to their physiology after menopause. Here, endocrine therapy for elderly patients with breast cancer in neoadjuvant, adjuvant, and palliative setting is reviewed along with predictive adverse events resulting from the use of hormone agents.