With its advantages of ecological safety, environmental affinity, and high selectivity, allelopathic technology has been widely developed for algae inhibition. However, obtaining effective allelochemicals and realizing their mechanism are difficult. In this paper, a Chinese herbal medicine, namely, Rheum palmatum L. (Chinese rhubarb), was utilized as a source of allelopathic substances for the first time. Four units of rhubarb organic extracts were collected to study the inhibition of growth, photosynthesis, proteins, and algal toxin of Microcystis aeruginosa. Results showed that the ethyl acetate, n-butanol, and aqueous phases of the rhubarb extracts have notable inhibitory effects. After a 16-day treatment, the four extracts reduced M. aeruginosa by 64.1%, 59.3%, 61.9%, and 7.2% with disruption of algal photosynthesis and protein synthesis and reduction of algal toxin.