Owing to its roles in human health and disease, the modification of nuclear, cytoplasmic, and mitochondrial proteins with O-linked N-acetylglucosamine residues (O-GlcNAc) has emerged as a topic of great interest. Despite the presence of O-GlcNAc on hundreds of proteins within cells, only two enzymes regulate this modification. One of these enzymes is O-GlcNAcase (OGA), a dimeric glycoside hydrolase that has a deep active site cleft in which diverse substrates are accommodated. Chemical tools to control OGA are emerging as essential resources for helping to decode the biochemical and cellular functions of the O-GlcNAc pathway. Here we describe rationally designed bicyclic thiazolidine inhibitors that exhibit superb selectivity and picomolar inhibition of human OGA. Structures of these inhibitors in complex with human OGA reveal the basis for their exceptional potency and show that they extend out of the enzyme active site cleft. Leveraging this structure, we create a high affinity chemoproteomic probe that enables simple one-step purification of endogenous OGA from brain and targeted proteomic mapping of its post-translational modifications. These data uncover a range of new modifications, including some that are less-known, such as O-ubiquitination and N-formylation. We expect that these inhibitors and chemoproteomics probes will prove useful as fundamental tools to decipher the mechanisms by which OGA is regulated and directed to its diverse cellular substrates. Moreover, the inhibitors and structures described here lay out a blueprint that will enable the creation of chemical probes and tools to interrogate OGA and other carbohydrate active enzymes.