The objective of the present study was to characterize the rumen bacterial and archaeal communities in dairy cows fed different ratios of maize silage (MS) and grass silage (GS), and place the findings in the context of ruminal fermentation as well as previously reported methane (CH4) emissions. Rumen fluid from 12 rumen cannulated dairy cows was collected after 10 and 17 days of feeding one of four diets, all of which had the same roughage to concentrate ratio of 80:20 based on dry matter (DM). Roughage in the four diets (GS100, GS0, GS67, GS33) consisted of either 1000 g/kg DM GS (GS100), 1000 g/kg DM MS (GS0), or a mixture of both silages in different proportions [667 g/kg DM GS and 333 g/kg DM MS (GS67); 333 g/kg DM GS and 677 g/kg DM MS (GS33)]. Total volatile fatty acid (VFA) concentrations and the molar proportions of the ruminal VFA were not affected by diet. Only the molar proportion of isovalerate was affected by time, being lower on day 17 than on day 10. Bacterial and archaeal concentrations were not affected by diet but increased from day 10 to day 17. The bacterial community composition was affected by diet, time and diet × time, whereas the archaeal community composition was only affected by diet. Several bacterial and archaeal genus level groups were associated with diet, but not with time. Analysis indicated the increased use of hydrogen by succinate and lactate producing bacteria is likely to at least partially explain the previously reported lower CH4 emissions from MS fed dairy cows. Furthermore, time had a significant effect on both bacterial and archaeal concentrations, and also bacterial community composition. This indicates that the rumen microbiota had not stabilized after 10 days of feeding the experimental diets.