Amino-terminal enhancer of split (AES) is a member of the Groucho/TLE family. Although it has no DNA-binding site, AES can regulate transcriptional activity by interacting with transcriptional factors. Emerging evidence indicates that AES may play an important role in tumor metastasis, but the molecular mechanism is still poorly understood. In this study, we found that knockdown of AES by RNA interference (RNAi) downregulated RND3 expression at the mRNA and protein levels in MDA-MB-231 and HepG2, two cancer cell lines. Furthermore, luciferase assays showed that overexpression of AES significantly enhanced RND3 promoter activity. Moreover, inhibition of AES both in MDA-MB-231 and HepG2 cells by RNAi significantly promoted cell proliferation, cell cycle progression and invasion, consistent with the effects of RNAi-mediated RND3 knockdown in these cells. For the first time, data are presented showing that alteration of the malignant behavior of cancer cells by AES is related to RND3 regulation, and these findings also provide new insights into the mechanism of AES action in regulating tumor malignancy.