The aim is to study the association of estrogen receptor α (ERα) and estrogen receptor β (ERβ) gene polymorphisms and diplotypes with ovarian response to follicle-stimulating hormone (FSH) stimulation and the hormone levels [FSH, luteinizing hormone (LH), E2] at the third day of the menstrual cycle. Three hundred women undergoing medically assisted reproduction and 300 women with at least one spontaneous pregnancy participated in the study. ERα PvuII and XbaI polymorphisms as well as ERβ AluI polymorphism were genotyped. The FSH, LH, and E2 levels were determined at the third day of the menstrual cycle, while the follicular size, the follicle, and oocyte numbers were recorded during oocyte retrieval. PvuII CC, XbaI GG, and AluI GG women were characterized by increased amounts of large follicles compared to PvuII TT, XbaI AA, and AluI AA women (p = 0.045, 0.01, and 0.033, respectively). The PvuII CC/XbaI GG diplotype was also significantly associated with higher large follicle numbers compared to the PvuII TT/XbaI AA diplotype (p = 0.024). However, no associations were observed between ER gene polymorphisms and the hormonal profile, the follicle/oocyte numbers, and the pregnancy rates. ERα and ERβ genetic variants were associated with ovarian response to standard gonadotropin stimulation of women undergoing in-vitro fertilization affecting mainly the follicular growth.