Sirtuins are NAD+-dependent protein lysine deacylases implicated in aging-related diseases. Mammalian Sirtuin 4 (Sirt4) is located in mitochondria and a potential therapeutic target for cancer and metabolic diseases, but no potent and selective Sirt4 inhibitors have been reported. Here, we describe the identification of potent Sirt4-specific small-molecule inhibitors. Testing hits from a target-based virtual screen revealed 12 active compounds. A focused screen based on two top compounds, followed by structure-assisted design of derivatives, yielded four first-in-class potent Sirt4 inhibitors. Kinetic analyses indicate compound competition with the acyl peptide substrate, consistent with the docking models and implicating Sirt4's unique acyl binding site. The compounds indeed show preference for Sirt4 over other isoforms, with one of them (69) being highly isoform selective, and they are active in cells. Our results provide first lead compounds and mechanistic insights for optimization toward Sirt4-specific inhibitors useful as experimental tools and potential therapeutics.