The packaging of eukaryotic DNA into nucleosomal arrays permits cells to tightly regulate and fine-tune gene expression. The ordered disassembly and reassembly of these nucleosomes allows RNA polymerase II (RNAPII) conditional access to the underlying DNA sequences. Disruption of nucleosome reassembly following RNAPII passage results in spurious transcription initiation events, leading to the production of non-coding RNA (ncRNA). We review the molecular mechanisms involved in the suppression of these cryptic initiation events and discuss the role played by ncRNAs in regulating gene expression.