The prenylated flavonoid icaritin (ICT, 1), a new drug for treating advanced hepatocellular carcinoma (HCC), was selected as a template to develop more potent inhibitors. An initial semisynthetic modification of ICT was performed to obtain a structure-activity relationship (SAR), which indicated that the cytotoxicity is enhanced by OH-3 rhamnosylation and that OH-7 is an important modification site. Based on the results of the SAR study, 46 N-containing ICT derivatives were synthesized and evaluated as the anti-HCC inhibitors. The results showed that most of the derivatives produced inhibited three HCC cell lines used (Hep3B, HepG2 and SMMC-7721). The modification strategy was validated by 3D-QSAR, which provided information for the further design and optimization of ICT. The most potent compound, 11c, exhibited IC50 values of 7.6 and 3.1 μM against HepG2 and SMMC-7721 cells, respectively, which were more potent than those of ICT and sorafenib, respectively. Further mechanistic studies indicated that 11c caused arrest at the G0/G1 phase in the cell cycle and induced cell apoptosis in HepG2 and SMMC-7721 cells.