The disease burden of diabetes mellitus is increasing throughout the world. The need for more potent drugs to complement the present anti-diabetic drugs has become an imperative. Astragalus membranaceus, a key component of most Chinese herbal anti-diabetic formulas, has been an important prospect for lead anti-diabetic compounds. It has been progressively studied for its anti-diabetic properties. Ethnopharmacological studies have established its potential to alleviate diabetes mellitus. Recent studies have sought to relate its chemical constituents to types 1 and 2 diabetes mellitus. Its total polysaccharides, saponins, and flavonoids fractions and several isolated compounds have been the most studied. The total polysaccharides fraction demonstrated activity to both types 1 and 2 diabetes mellitus. This paper discusses the anti-diabetic effects and pharmacological action of the chemical constituents in relation to types 1 and 2 diabetes mellitus.