Ginseng, a perennial plant belonging to the Panax genus of the Araliaceae family, has been used in China, Korea, and Japan as a traditional herbal medicine for thousands of years. Ginseng is recorded to have exhibited a wide variety of beneficial pharmacological effects and has become a popular and worldwide known health supplement and drug. The protective effects of ginseng on central nervous system are discussed in this review. Ginseng species and ginsenosides and their intestinal metabolism and bioavailability are concisely introduced. The molecular mechanisms of the effects of ginseng on central nervous system, mainly focused on the neuroprotection properties of ginseng, memory, and learning enhanced properties, and the effects on neurodegenerative disorders are presented. Thus, ginseng and its constituents are of potential merits in the treatment of cerebral disorders.