Diet is a major modulator of gut microbiota, which plays a key role in the health status, including colorectal cancer (CRC) development. Several studies and meta-analyses have evidenced an association of certain dietary factors and xenobiotic intake with the incidence of CRC. Nevertheless, how these dietary factors impact the first stages of intestinal mucosa damage is still uncertain. This study aimed at exploring the associations of relevant dietary factors with the gut microbiota of control individuals and subjects diagnosed with intestinal polyps. A total of 60 volunteers were recruited, clinically classified according to colonoscopy criteria and interviewed using food frequency questionnaires (FFQs). The nutritional status of each volunteer was determined and the intake of dietary xenobiotics was quantified. The relative abundance of faecal microbiota taxonomic groups was obtained through 16S rRNA gene sequencing. The association of dietary factors and xenobiotics with faecal microbiota composition showed differences according to the clinical diagnosis group. Our results showed that the intake of red meat (≥50 g day-1) and total polycyclic aromatic hydrocarbons (PAHs) (≥0.75 μg day-1) was associated with a decreased abundance of the family Bacteroidaceae and an increased abundance of Coriobacteriaceae in control subjects. The intake of the heterocyclic amines 2-amino-1-methyl-6-phenylimidazo(4,5-b)pyridine (PhIP) (≥40 ng day-1) and 2-amino-3,8 dimethylimidazo(4,5,f) quinoxaline (MeIQx) (≥50 ng day-1) was associated with a decreased abundance of Akkermansiaceae in the control diagnosis group. Moreover, N-nitroso compounds (NOCs), nitrites (≥1.69 mg day-1) and N-nitrosodimethylamine (NDMA) (≥0.126 μg day-1) were associated with a decreased abundance of Bifidobacteriaceae. The intake of ethanol (≥12 g day-1) in the polyps group was associated with an increased abundance of Peptostreptococcaceae and a decreased abundance of Veillonellaceae. Moreover, linear regression analyses allowed us to identify ethanol, calcium, bioactive compounds such as flavonoids, stilbenes, cellulose, phenolic acids or total polyphenols, and dietary xenobiotics such as PhIP and MeIQx, the NOC N-nitrosopyrrolidine (NPYR) or the total PAHs as potential predictors of faecal microbiota group abundances. These results indicated that the consumption of milk, red meat, processed meat and ethanol and the intake of polyphenols, dietary PAHs, HAs and NOCs are associated with specific groups of the intestinal microbiota, depending on the clinical diagnosis group.