An emerging paradigm suggests that gut glycosylation is a key force in maintaining the homeostatic relationship between the gut and its microbiota. Nevertheless, it is unclear how gut glycosylation contributes to the HIV-associated microbial translocation and inflammation that persist despite viral suppression and contribute to the development of several comorbidities. We examined terminal ileum, right colon, and sigmoid colon biopsies from HIV-infected virally-suppressed individuals and found that gut glycomic patterns are associated with distinct microbial compositions and differential levels of chronic inflammation and HIV persistence. In particular, high levels of the pro-inflammatory hypo-sialylated T-antigen glycans and low levels of the anti-inflammatory fucosylated glycans were associated with higher abundance of glycan-degrading microbial species (in particular, Bacteroides vulgatus), a less diverse microbiome, higher levels of inflammation, and higher levels of ileum-associated HIV DNA. These findings are linked to the activation of the inflammasome-mediating eIF2 signaling pathway. Our study thus provides the first proof-of-concept evidence that a previously unappreciated factor, gut glycosylation, is a force that may impact the vicious cycle between HIV infection, microbial translocation, and chronic inflammation.