Aromatase, the key enzyme in estrogen biosynthesis, converts androstenedione to estrone and testosterone to estradiol. The enzyme is expressed in various tissues such as ovary, placenta, bone, brain, skin, and adipose tissue. Aromatase enzyme is encoded by a single gene CYP 19A1 and its expression is controlled by tissue-specific promoters. Aromatase mRNA is primarily transcribed from promoter I.4 in normal breast tissue and physiological levels of aromatase are found in breast adipose stromal fibroblasts. Under the conditions of breast cancer, as a result of the activation of a distinct set of aromatase promoters (I.3, II, and I.7) aromatase expression is enhanced leading to local overproduction of estrogen that promotes breast cancer. Aromatase is considered as a potential target for endocrine treatment of breast cancer but due to nonspecific reduction of aromatase activity in other tissues, aromatase inhibitors (AIs) are associated with undesirable side effects such as bone loss, and abnormal lipid metabolism. Inhibition of aromatase expression by inactivating breast tumor-specific aromatase promoters can selectively block estrogen production at the tumor site. Although several synthetic chemical compounds and nuclear receptor ligands are known to inhibit the activity of the tumor-specific aromatase promoters, further development of more specific and efficacious drugs without adverse effects is still warranted. Plants are rich in chemopreventive agents that have a great potential to be used in chemotherapy for hormone dependent breast cancer which could serve as a source for natural AIs. In this brief review, we summarize the studies on phytochemicals such as biochanin A, genistein, quercetin, isoliquiritigenin, resveratrol, and grape seed extracts related to their effect on the activation of breast cancer-associated aromatase promoters and discuss their aromatase inhibitory potential to be used as safer chemotherapeutic agents for specific hormone-dependent breast cancer.