The estrogen receptors (ERs) are a group of versatile receptors. They regulate an enormity of processes starting in early life and continuing through sexual reproduction, development, and end of life. This review provides a background and structural perspective for the ERs as part of the nuclear receptor superfamily and discusses the ER versatility and promiscuity. The wide repertoire of ER actions is mediated mostly through ligand-activated transcription factors and many DNA response elements in most tissues and organs. Their versatility, however, comes with the drawback of promiscuous interactions with structurally diverse exogenous chemicals with potential for a wide range of adverse health outcomes. Even when interacting with endogenous hormones, ER actions can have adverse effects in disease progression. Finally, how nature controls ER specificity and how the subtle differences in receptor subtypes are exploited in pharmaceutical design to achieve binding specificity and subtype selectivity for desired biological response are discussed. The intent of this review is to complement the large body of literature with emphasis on most recent developments in selective ER ligands.