Osteoporosis is a common disease with wide prevalence, especially in the elderly population. Osteoporosis induced fractures not only decrease the patient's life quality, but also cause heavy financial burden to the society. Although current medications for osteoporosis are effective, numerous adverse effects have been observed accompanying their clinical applications. Effective prevention and therapy strategies with high safety are critical, which benefit both individual patients and the whole society. Traditional Chinese medicines have been used for thousands of years to treat bone related diseases in China and a number of modern preparations have been developed that are currently commercially available. In addition, several medicinal herbs demonstrated therapeutic effects against osteoporosis in animal models. This paper reviewed the anti-osteoporotic effects of traditional Chinese formulas, medicinal herbs and bioactive constituents based on clinical trials and in vivo animal studies. Due to the lack of rigorous studies to compare the effectiveness with conventional interventions, traditional formulas are recommended as alternative medications or supplements to treat osteoporosis at the current stage. Although there are abundant natural resources with anti-osteoporotic effects, either in the form of medicinal herbs or bioactive components, much work need to be accomplished before they are developed into potential drugs.