Previous studies have shown that 5,2',4'-trihydroxy-6,7,5'-trimethoxyflavone (TTF1) is the primary anticancer constituent of the traditional Chinese medicinal plant Sorbaria sorbifolia (SS), which has been applied to treat cancer in China. In this study, we investigated the in vitro and in vivo antitumor effects and biological mechanisms of small-molecule TTF1 nanoparticles (TTF1-NPs). The effects of TTF1-NPs on cell growth and apoptosis were investigated using human hepatoma cells. The molecular changes associated with the effects of TTF1-NPs were analyzed by immunocytochemistry and Western blot analysis. The in vivo effect of TTF1-NPs was investigated using the HepG2 tumor xenograft model. We found that TTF1-NPs exhibited antitumor effects in vitro accompanied by induction of apoptosis in human hepatoma cells. Mechanistically, our data showed that TTF1-NPs induced apoptosis via endoplasmic reticulum stress (ERS) pathway in hepatoma cells. Moreover, inhibition of ERS activation blocked TTF1-NP-induced apoptosis in HepG2 cells. Finally, TTF1-NPs inhibited the growth of HepG2 xenograft tumors. Taken together, our results demonstrated that TTF1-NP-induced apoptosis was mediated at least in part by the ERS pathway and thus inhibited hepatoma tumor growth.