While poly(ADP-ribosyl)ation (PARylation) plays an important role in DNA repair, the role of dePARylation in DNA repair remains elusive. Here, we report that a novel small molecule identified from the NCI database, COH34, specifically inhibits poly(ADP-ribose) glycohydrolase (PARG), the major dePARylation enzyme, with nanomolar potency in vitro and in vivo. COH34 binds to the catalytic domain of PARG, thereby prolonging PARylation at DNA lesions and trapping DNA repair factors. This compound induces lethality in cancer cells with DNA repair defects and exhibits antitumor activity in xenograft mouse cancer models. Moreover, COH34 can sensitize tumor cells with DNA repair defects to other DNA-damaging agents, such as topoisomerase I inhibitors and DNA-alkylating agents, which are widely used in cancer chemotherapy. Notably, COH34 also efficiently kills PARP inhibitor-resistant cancer cells. Together, our study reveals the molecular mechanism of PARG in DNA repair and provides an effective strategy for future cancer therapies.