Transcription factors (TF) regulate gene expression acting as DNA sequence-specific binding factors, orchestrating cofactor recruitment and assembly of the transcriptional machinery. Nuclear receptors, a ligand-inducible TF class, regulate a large proportion of the genome, yet achieve highly cell-specific and context-dependent transcription, despite their widespread expression. High-throughput genome-wide profiling of TF binding reveals a startling proportion of colocalized cell- and context-specific TF-binding patterns, implying TF interactions play a critical role in transcription. These interactions depend on the chromatin architecture, that predominantly acts to predetermine accessibility of TF-binding sites at regulatory elements. Here, we summarize recent findings that highlight the importance of combinatorial TF interactions in determining diverse nuclear receptor-mediated transcriptional responses, emphasizing the significance of chromatin structure in directing TF and nuclear receptor recruitment. Interactions between TFs are likely to be a general mechanism of regulatory factors, contributing to transcriptional control in health and disease.