Recent studies have elucidated cell-lineage-specific three-dimensional genome organization; however, how such specific architecture is established or maintained is unclear. We hypothesized that lineage-defining transcription factors maintain cell identity via global control of genome organization. These factors bind many genomic sites outside of the genes that they directly regulate and thus are potentially implicated in three-dimensional genome organization. Using chromosome-conformation-capture techniques, we show that the transcription factor Paired box 5 (Pax5) is critical for the establishment and maintenance of the global lineage-specific architecture of B cells. Pax5 was found to supervise genome architecture throughout B cell differentiation, until the plasmablast stage, in which Pax5 is naturally silenced and B cell-specific genome structure is lost. Crucially, Pax5 did not rely on ongoing transcription to organize the genome. These results implicate sequence-specific DNA-binding proteins in global genome organization to establish and maintain lineage fidelity.