Colorectal cancer (CRC) is among the most common malignant diseases with high morbidity and mortality rates. Ginseng and its major extracts, ginsenosides, have been used in medical fields for thousands of years. In particular, their huge anti-cancer potential has drawn a great deal of attention in recent years. There is a large body of evidence that has shown that ginseng and its extracts could significantly inhibit tumor development and progression by suppressing cell proliferation, tumor growth, invasion and metastasis, inducing tumor cell apoptosis, regulating tumor-associated immune responses, and improving the therapeutic effect of chemotherapy. Notably, different subtypes of ginsenosides, even those extracted from the same ginseng, have exhibited distinct anti-cancer functions through different mechanisms. Over the past few years, a large number of studies have focused on how ginseng or various ginsenosides influence CRC development. Therefore, the roles and the potential of ginseng and ginsenosides in the treatment of CRC are summarized in this review. In addition, the biochemical properties of ginseng and ginsenosides are also briefly described.