The genome is spatially organized inside nuclei, with chromosomes and genes occupying preferential positions relative to each other and to various nuclear landmarks. What drives this organization is unclear, but recent findings suggest there are extensive intra- and inter-chromosomal communications between various genomic regions that appear to play important roles in genome function. Here we review transcription factories, distinct sub-nuclear foci where nascent transcription occurs. We argue that the spatially restricted, limited number of transcription sites compels transcribed regions of the genome to dynamically self-organize into tissue-specific conformations, thus playing a major role in the three-dimensional interphase organization of the genome.