Onchidium struma polysaccharides (OsPs) are natural biologically active compounds, and our previous work showed that they can inhibit the activity of α-glucosidase in vitro, showing potential hypoglycemic activity. However, the effects of OsPs on type 2 diabetes mellitus (T2DM) in vivo remain unknown. Thus, the anti-diabetic activity of OsPs was evaluated in the present study in diabetic mice. The results showed that OsPs can significantly ameliorate the features of T2DM in mice by improving the levels of fasting blood glucose (FBG), oral glucose tolerance test (OGTT), and pro-inflammatory factors, and ameliorating insulin resistance. Furthermore, OsPs can significantly improve biochemical indicators, decrease the contents of total cholesterol (TC) and triglyceride (TG), and reduce lipid accumulation in the liver. The possible mechanism of the prevention and treatment of T2DM by OsPs may involve the phosphoinositide 3-kinase (PI3K)/protein kinase B (AKT-1) signaling pathway. OsPs can regulate the dysbiosis of gut microbiota and reverse the abundance of Lactobacillus in mice with T2DM. Moreover, OsPs significantly increased the concentration of short-chain fatty acids (SCFAs) in mice with T2DM. Our results indicate that OsPs can be used as a novel food supplement for the prevention and treatment of T2DM.