The vast majority of the mammalian genome has the potential to express noncoding RNA (ncRNA). The 11-subunit RNA exosome complex is the main source of cellular 3'-5' exoribonucleolytic activity and potentially regulates the mammalian noncoding transcriptome. Here we generated a mouse model in which the essential subunit Exosc3 of the RNA exosome complex can be conditionally deleted. Exosc3-deficient B cells lack the ability to undergo normal levels of class switch recombination and somatic hypermutation, two mutagenic DNA processes used to generate antibody diversity via the B-cell mutator protein activation-induced cytidine deaminase (AID). The transcriptome of Exosc3-deficient B cells has revealed the presence of many novel RNA exosome substrate ncRNAs. RNA exosome substrate RNAs include xTSS-RNAs, transcription start site (TSS)-associated antisense transcripts that can exceed 500 base pairs in length and are transcribed divergently from cognate coding gene transcripts. xTSS-RNAs are most strongly expressed at genes that accumulate AID-mediated somatic mutations and/or are frequent translocation partners of DNA double-strand breaks generated at Igh in B cells. Strikingly, translocations near TSSs or within gene bodies occur over regions of RNA exosome substrate ncRNA expression. These RNA exosome-regulated, antisense-transcribed regions of the B-cell genome recruit AID and accumulate single-strand DNA structures containing RNA-DNA hybrids. We propose that RNA exosome regulation of ncRNA recruits AID to single-strand DNA-forming sites of antisense and divergent transcription in the B-cell genome, thereby creating a link between ncRNA transcription and overall maintenance of B-cell genomic integrity.