To study how a vegetarian or low meat diet influences the risk of colorectal cancer compared to a high meat diet, and to assess the explanatory role of factors associated with these diets. In the Netherlands Cohort Study - Meat Investigation Cohort (NLCS-MIC) (cohort of 10,210 individuals including 1040 self-defined vegetarians), subjects completed a baseline questionnaire in 1986, based on which they were classified into vegetarians (n = 635), pescetarians (n = 360), 1 day/week- (n = 1259), 2-5 day/week- (n = 2703), and 6-7 day/week meat consumers (n = 5253). After 20.3 years of follow-up, 437 colorectal cancer cases (307 colon, 92 rectal) were available. A non-significantly decreased risk of CRC for vegetarians, pescetarians, and 1 day/week compared to 6-7 day/week meat consumers was observed (age/sex adjusted Hazard Ratios (HR): 0.73(0.47-1.13), 0.80(0.47-1.39), and 0.72(0.52-1.00), respectively). Most of the differences in HR between these groups could be explained by intake of dietary fiber and soy products. Other (non-)dietary factors characteristic for a vegetarian or low meat diet had negligible individual effects, but attenuated the HRs towards the null when combined. Vegetarians, pescetarians, and 1 day/week meat eaters showed a non-significantly decreased risk of colorectal cancer compared to 6-7 day/week meat consumers, mainly due to differences in dietary pattern other than meat intake.