A polyphenol-rich diet has been associated with various health benefits. This study assessed the effects of polyphenol/anthocyanin isolated from a purple sweet potato (Ipomoea batatas cv. Ayamurasaki) on colonic fermentation in cellulose- or inulin-fed rats. Male Fischer-344 rats were assigned to one of these experimental diets: 5% cellulose (CEL), 5% CEL + 1% purple sweet potato polyphenol extract (CELP), 5% inulin (INU), and 5% INU + 1% purple sweet potato polyphenol extract (INUP) in each diet. The purple sweet potato polyphenol extract (PSPP) increased the relative abundance of Dorea and reduced the relative abundances of Oscillospira and Bacteroides in cellulose- or inulin-fed rats, respectively. Besides, PSPP reduced the caecal iso-butyrate and pH in the cellulose-fed rats. Further, PSPP triggered an increase in the caecal mucin level when combined with cellulose and increased the caecal IgA level while reducing the indole production in both the cellulose- or inulin-fed rats. Finally, PSPP may have different effects on the intestinal fermentation properties depending on the fermentability of dietary fiber associated with it. Therefore, this study demonstrated that dietary inclusion of polyphenol/anthocyanin from purple sweet potato might confer positive health attributes to the host gut.