Type 2 diabetes mellitus (T2DM) occurs in 95% of the diabetic populations. Management of T2DM is a challenge. Traditional Chinese medicines (TCM) are usually served as adjuvants used to improve diabetic syndromes in combination of routine antidiabetic drugs. For single-herb prescriptions, Ginseng, Bitter melon, Golden Thread, Fenugreek, Garlic, and Cinnamon might have antidiabetic effects in T2DM patients. Among 30 antidiabetic formulas approved by the State Food and Drugs Administrator of China, top 10 of the most frequently prescribed herbs are Membranous Milkvetch Root, Rehmannia Root, Mongolian Snakegourd Root, Ginseng, Chinese Magnoliavine Fruit, Kudzuvine Root, Dwarf Lilyturf Tuber, Common Anemarrhena Rhizome, Barbary Wolfberry Fruit, and India Bread, which mainly guided by the theory of TCM. Their action mechanisms are related to improve insulin sensitivity, stimulate insulin secretion, protect pancreatic islets, and even inhibit intake of intestinal carbohydrates. However, it is very difficult to determine antihyperglycemic components of TCM. Nevertheless, TCM are becoming popular complementary and alternative medicine in treatment of syndromes of T2DM. In the future, it requires further validation of phytochemical, pharmacological, and clinical natures of TCM in T2DM in the future studies, especially for those herbs with a high prescription frequency.