Evidence suggests that sex influences the effect of diet on the gut bacterial composition, yet, no studies have been performed assessing dietary fatty acid composition (i.e., fat quality) in this context. This study examined the effect of dietary fat quality on colonic bacterial composition in an aged, genetically-diverse mouse population. CD-1 mice were fed isoenergetic diets consisting of (1) control fat (CO; "Western-style" fat blend), (2) CO supplemented with 30% fish oil, (3) CO supplemented with 30% dairy fat, or (4) CO supplemented with 30% echium oil. Fecal samples were collected at mid-life and aged (reproductively senescent) time points. Overall, the abundance of Bacteroidetes was greater in mice fed echium oil compared to mice fed the control fat. Examination of colonic bacterial relative abundance also revealed sex differences, with 73 bacterial taxa being differentially expressed in males and females. Notably, results showed a strong interactive effect among the diet, sex, and age of mice which influenced colonic bacterial relative abundance and alpha diversity. In males, supplementation of the diet with dairy fat or echium oil caused the abundance of Bacteroidetes and Bacteroides to change with age. Additionally, supplementation of the diet with fish oil induced sex-dependent changes in the alpha diversity of aged mice compared to mid-life. This work supports that sex is a critical factor in colonic bacterial composition of an aged, genetically-heterogenous population. Moreover, this study establishes that the effectiveness of dietary interventions for health maintenance and disease prevention via direct or indirect manipulation of the gut microbiota is likely dependent on an individual's sex, age, and genetic background.