Our understanding of the ruminal epithelial tissue-associated bacterial (defined as epimural bacteria in this study) community is limited. In this study, we aimed to determine whether diet influences the diversity of the epimural bacterial community in the bovine rumen. Twenty-four beef heifers were randomly assigned to either a rapid grain adaptation (RGA) treatment (n = 18) in which the heifers were allowed to adapt from a diet containing 97% hay to a diet containing 8% hay over 29 days or to the control group (n = 6), which was fed 97% hay. Rumen papillae were collected when the heifers were fed 97%, 25%, and 8% hay diets. PCR-denaturing gradient gel electrophoresis (DGGE) and quantitative real-time PCR analysis were used to characterize rumen epimural bacterial diversity and to estimate the total epimural bacterial population (copy numbers of the 16S rRNA gene). The epimural bacterial diversity from RGA heifers changed (P = 0.01) in response to the rapid dietary transition, whereas it was not affected in control heifers. A total of 88 PCR-DGGE bands were detected, and 44 were identified from phyla including Firmicutes, Bacteroidetes, and Proteobacteria. The bacteria Treponema sp., Ruminobacter sp., and Lachnospiraceae sp. were detected only when heifers were fed 25% and 8% hay diets, suggesting the presence of these bacteria is the result of adaptation to the high-grain diets. In addition, the total estimated population of rumen epimural bacteria was positively correlated with molar proportions of acetate, isobutyrate, and isovalerate, suggesting that they may play a role in volatile fatty acid metabolism in the rumen.