Previous research studies have shown that sulfated polysaccharides can inhibit food allergy, but the detailed mechanism remains largely unknown. In this study, RBL-2H3 cells were used to compare the anti-allergic activities of four sulfated polysaccharides, and an ovalbumin (OVA)-sensitized allergic mouse experiment was used to explore their desensitization effect, with regard to the alteration in gut microbiota and immune cell differentiation. Compared with the shark, bovine and porcine chondroitin sulfate, sea cucumber chondroitin sulfate (SCCS) significantly inhibited the degranulation of RBL-2H3 cells. SCCS reduced allergic symptoms and protected the jejunum from injury in mice. Furthermore, SCCS increased the relative abundance of Lachnospiraceae NK4A136, decreased the relative proportion of Prevotellaceae NK3B31, and up-regulated the secretion of short chain fatty acids such as butyric acid in the feces, resulting in an increase in the mucin 2 (MUC2) secretion by goblet cells HT-29. Meanwhile, SCCS induced the differentiation of regulatory T cells in the mesenteric lymph nodes of mice. This study provides a deeper understanding of the functioning mechanism of SCCS in alleviating food allergy and may guide the development and production of anti-allergy active ingredients.