Histone modifications regulate transcription by RNA polymerase II and maintain a balance between active and repressed chromatin states. The conserved Paf1 complex (Paf1C) promotes specific histone modifications during transcription elongation, but the mechanisms by which it facilitates these marks are undefined. We previously identified a 90-amino acid region within the Rtf1 subunit of Paf1C that is necessary for Paf1C-dependent histone modifications in Saccharomyces cerevisiae. Here we show that this histone modification domain (HMD), when expressed as the only source of Rtf1, can promote H3 K4 and K79 methylation and H2B K123 ubiquitylation in yeast. The HMD can restore histone modifications in rtf1Δ cells whether or not it is directed to DNA by a fusion to a DNA binding domain. The HMD can facilitate histone modifications independently of other Paf1C subunits and does not bypass the requirement for Rad6-Bre1. The isolated HMD localizes to chromatin, and this interaction requires residues important for histone modification. When expressed outside the context of full-length Rtf1, the HMD associates with and causes Paf1C-dependent histone modifications to appear at transcriptionally inactive loci, suggesting that its function has become deregulated. Finally, the Rtf1 HMDs from other species can function in yeast. Our findings suggest a direct and conserved role for Paf1C in coupling histone modifications to transcription elongation.