Fermentation differs between the proximal and distal gut but little is known regarding how the bacterial communities differ or how they are influenced by diet. In order to investigate this, we compared community diversity in the cecum and feces of rats by 16S rRNA gene content and DNA shot gun metagenomics after feeding purified diets containing different fermentable substrates. Gut community composition was dependent on the source of fermentable substrate included in the diet. Cecal communities were dominated by Firmicutes, and contained a higher abundance of Lachnospiraceae compared to feces. In feces, community structure was shifted by varying degrees depending on diet towards the Bacteroidetes, although this change was not always evident from 16S rRNA gene data. Multi-dimensional scaling analysis (PCoA) comparing cecal and fecal metagenomes grouped by location within the gut rather than by diet, suggesting that factors in addition to substrate were important for community change in the distal gut. Differentially abundant genes in each environment supported this shift away from the Firmicutes in the cecum (e.g., motility) towards the Bacteroidetes in feces (e.g., Bacteroidales transposons). We suggest that this phylum level change reflects a shift to ammonia as the primary source of nitrogen used to support continued microbial growth in the distal gut.