Jiedu Xiaozheng Yin (JXY), a polyherbal formula of traditional Chinese medicine (TCM), has been used to treat various kinds of cancer in China. However, the mechanism of its anticancer activity has yet to be elucidated. Air-dried herbs were extracted with reagents of different polarity. HepG2 cells were treated with different doses of ethyl acetate extract (EE-JXY) and chloroform extract (CE-JXY) for 24 h. Cell viability was detected by MTT assay. Colony formation ability was also evaluated. Cell cycle was evaluated by FACS. Tumor bearing BALB/c nude mice was treated with EE-JXY (0.06 g/kg) for 20 days. Tumor volume and weight were monitored. The percentage of PCNA-positive cells and the level of G1 phase proteins [cyclin-dependent kinase2 (CDK2), cyclin‑dependent kinase4 (CDK4), cyclin D and cyclin E and G2 phase proteins [cyclin-dependent kinase1 (CDK1), cyclin A and cyclin B] were detected by immunohistochemistry and western blotting. EE-JXY and CE-JXY dose-dependently inhibited the growth of HepG2 cells (P<0.01 for both). Furthermore, EE-JXY inhibited the formation of cell colonies and blocked the cell cycle to G1 phase in a dose-dependent manner (P<0.01 for all). EE-JXY showed an obviously antitumor effect in vivo (P<0.05). Further investigation showed that EE-JXY decreased the proliferation index of tumors (P<0.01) through increasing the expression of G1-related proteins (cyclin D and cyclin E, P<0.05 and P<0.01). These results suggested that JXY inhibits the growth of HepG2 cells at least via arresting the cell cycle at the G0/G1 phase.