In this study, the two enzymatic low molecular weight chitosan oligosaccharides (LMW-COSs), LMW-COS-H and LMW-COS-L, were prepared with average MWs of 879.6 Da and 360.9 Da, respectively. Compared to LMW-COS-L, the LMW-COS-H was more effective in improving high-fat diet (HFD)-induced metabolic abnormalities, such as obesity, hyperlipidemia, low-grade inflammation and insulin resistance. The subsequent analysis of gut microbiota showed that the supplement of LMW-COSs caused overall structural and genus/species-specific changes in the gut microbiota, which were significantly correlated with the metabolic parameters. Specifically, both of the LMW-COSs significantly decreased the relative abundance of inflammatory bacteria such as Erysipelatoclostridium and Alistipes, whereas that of the beneficial intestinal bacteria (such as Akkermansia and Gammaproteobacteria) increased significantly. This study suggested that there were potential prebiotic functions of LMW-COSs in HFD fed mice, which regulated the dysfunctional gut microbiota, alleviated low-grade inflammation and maintained the intestinal epithelial barrier.