An attempt has been made to compare the hypoglycemic effect of the organic constituents and the inorganic constituents in traditional Chinese medicine, Jinqi compound recipe. Alloxan-induced hyperglycemic mice were used in the study. The body weights, blood glucose, HbA1c, insulin secretion, and glycogen synthesis of the mice were analyzed respectively. After the mice were administered (oral) with organic constituents, the blood glucose and the HbA1c of alloxan-induced hyperglycemic mice decreased (p < 0.05, p < 0.01) and the glycogen synthesis of alloxan-induced hyperglycemic mice elevated (p < 0.05). Also, the body weight of the alloxan-induced hyperglycemic mice was increased gradually. However, the same result did not occur in the inorganic constituent-treated groups. It is noted that neither organic constituents nor inorganic constituents could elevate the level of serum insulin in the alloxan-induced hyperglycemic mice significantly. It is implied that the hypoglycemic effect for type 2 diabetes was caused by the organic constituents of Jinqi recipe. The results can be used to set new standards to control the quality of the Jinqi recipe.