This study investigated the effects of ensiled mulberry leaves (EML) and sun-dried mulberry fruit pomace (SMFP) on the ruminal bacterial and archaeal community composition of finishing steers. Corn grain- and cotton meal-based concentrate was partially replaced with EML or SMFP. The diets had similar crude protein (CP), neutral detergent fiber (NDF), and metabolizable energy. Following the feeding trial, the steers were slaughtered and ruminal liquid samples were collected to study the ruminal microbiome. Extraction of DNA, amplification of the V4 region of the 16S rRNA gene, and Illumina MiSeq pyrosequencing were performed for each sample. Following sequence de-noising, chimera checking, and quality trimming, an average of 209,610 sequences were generated per sample. Quantitative real-time PCR was performed to examine the selected bacterial species in the rumen. Our results showed that the predominant phyla were Bacteroidetes (43.90%), Firmicutes (39.06%), Proteobacteria (4.31%), and Tenericutes (2.04%), and the predominant genera included Prevotella (13.82%), Ruminococcus (2.51%), Butyrivibrio (2.38%), and Succiniclasticum (2.26%). Compared to the control group, EML and SMFP groups had a higher abundance of total bacteria (p < 0.001); however, the bacterial community composition was similar among the three groups. At the phylum level, there were no significant differences in Firmicutes (p = 0.7932), Bacteroidetes (p = 0.2330), Tenericutes (p = 0.2811), or Proteobacteria (p = 0.0680) levels among the three groups; however, Fibrobacteres decreased in EML (p = 0.0431). At the genus level, there were no differences in Prevotella (p = 0.4280), Ruminococcus (p = 0.2639), Butyrivibrio (p = 0.4433), or Succiniclasticum (p = 0.0431) levels among the groups. Additionally, the dietary treatments had no significant effects on the archaeal community composition in the rumen. Therefore, EML and SMFP supplementation had no significant effects on the ruminal bacterial or archaeal community composition of finishing steers.