In the present study, the hypoglycemic effects of glucomannan (AGM) and its enzyme-degraded products from Amorphophallus albus were investigated. Four degraded products were prepared through ultrafiltration of β-glucanase-degraded products of AGM. The hypoglycemic activities were evaluated in HFD-STZ-induced type 2 diabetes mellitus (T2DM) mice, and the diversity of gut bacteria was analyzed by 16S rRNA gene sequencing; the fecal short chain fatty acids (SCFAs) and endogenous metabolites were determined by UPLC-QTOF-MS/MS. It was found that AGM and its enzyme-degraded products, though with different molecular weights, had similar β-glycosidic bonds and monosaccharide compositions, exerted similar strength of hypoglycemic effects, and reinstated with a similar extent the disordered gut microbiota and the contents of SCFAs and endogenous metabolites. It was speculated that the hypoglycemic activity of AGM is decided by not the molecular weight but the glycosidic bonds/monosaccharide composition of AGM, which might be structurally specific to the gut bacteria, and thus certain SCFAs and endogenous metabolites that are related to the occurrence and therapy of T2DM. This study provides a scientific basis for using AGM as potential prebiotics beneficial for prevention or therapeutic treatment of T2DM.