Medicinal plants have been used world-wide for thousands of years and are widely recognized as having high healing but minor toxic side effects. The scarcity and increasing demand for medicinal plants and their products have promoted the development of artificial cultivation of medicinal plants. Currently, one of the prominent issues in medicinal cultivation systems is the unstable quality of the products. Arbuscular mycorrhiza (AM) affects secondary metabolism and the production of active ingredients of medicinal plants and thus influence the quality of herbal medicines. In this review, we have assembled, analyzed, and summarized the effects of AM symbioses on secondary metabolites of medicinal plants. We conclude that symbiosis of AM is conducive to favorable characteristics of medicinal plants, by improving the production and accumulation of important active ingredients of medicinal plants such as terpenes, phenols, and alkaloids, optimizing the composition of different active ingredients in medicinal plants and ultimately improving the quality of herbal materials. We are convinced that the AM symbiosis will benefit the cultivation of medicinal plants and improve the total yield and quality of herbal materials. Through this review, we hope to draw attention to the status and prospects of, and arouse more interest in, the research field of medicinal plants and mycorrhiza.