USP5 is a deubiquitinase that has been implicated in a range of diseases, including cancer, but no USP5-targeting chemical probe has been reported to date. Here, we present the progression of a chemical series that occupies the C-terminal ubiquitin-binding site of a poorly characterized zinc-finger ubiquitin binding domain (ZnF-UBD) of USP5 and competitively inhibits the catalytic activity of the enzyme. Exploration of the structure-activity relationship, complemented with crystallographic characterization of the ZnF-UBD bound to multiple ligands, led to the identification of 64, which binds to the USP5 ZnF-UBD with a KD of 2.8 μM and is selective over nine proteins containing structurally similar ZnF-UBD domains. 64 inhibits the USP5 catalytic cleavage of a di-ubiquitin substrate in an in vitro assay. This study provides a chemical and structural framework for the discovery of a chemical probe to delineate USP5 function in cells.