Since microglia possess both neuroprotective and neurotoxic potential, they play a crucial role in the central nervous system (CNS). Excessive microglial activation induces inflammation-mediated neuronal damage and degeneration. At present, numerous herbal compounds are able to suppress neurotoxicity via inhibiting microglial activation. Therefore, many researchers focus on pharmacological inhibitors of microglial activation to ameliorate neurodegenerative disorders. Further work should concentrate on the exploration of new herbal compounds, which characteristically inhibit microglial neurotoxicity, rather than modulating neuroprotection alone. In this review, we summarize these herbal compounds, which in the past several years have been shown to exert potential neuroprotective activity by inhibiting microglial activation. The therapeutic targets and pharmacological mechanisms of these compounds have also been discussed.