In addition to the conserved RNA polymerases I to III (Pols I to III) in eukaryotes, two atypical polymerases, Pols IV and V, specifically produce noncoding RNA in the RNA-directed DNA methylation pathway in plants. Here, we report on the structures of cauliflower Pol V in the free and elongation conformations. A conserved tyrosine residue of NRPE2 stacks with a double-stranded DNA branch of the transcription bubble to potentially attenuate elongation by inducing transcription stalling. The nontemplate DNA strand is captured by NRPE2 to enhance backtracking, thereby increasing 3'-5' cleavage, which likely underpins Pol V's high fidelity. The structures also illuminate the mechanism of Pol V transcription stalling and enhanced backtracking, which may be important for Pol V's retention on chromatin to serve its function in tethering downstream factors for RNA-directed DNA methylation.