In the cell nucleus, each chromosome is confined to a chromosome territory. This spatial organization of chromosomes plays a crucial role in gene regulation and genome stability. An additional level of organization has been discovered at the chromosome scale: the spatial segregation into open and closed chromatins to form two genome-wide compartments. Although considerable progress has been made in our knowledge of chromatin organization, a fundamental issue remains the understanding of its dynamics, especially in cancer. To address this issue, we performed genome-wide mapping of chromatin interactions (Hi-C) over the time after estrogen stimulation of breast cancer cells. To biologically interpret these interactions, we integrated with estrogen receptor α (ERα) binding events, gene expression and epigenetic marks. We show that gene-rich chromosomes as well as areas of open and highly transcribed chromatins are rearranged to greater spatial proximity, thus enabling genes to share transcriptional machinery and regulatory elements. At a smaller scale, differentially interacting loci are enriched for cancer proliferation and estrogen-related genes. Moreover, these loci are correlated with higher ERα binding events and gene expression. Taken together these results reveal the role of a hormone--estrogen--on genome organization, and its effect on gene regulation in cancer.