Reversible glycosylation of nuclear and cytoplasmic proteins is an important regulatory mechanism across metazoans. One enzyme, O-linked N-acetylglucosamine transferase (OGT), is responsible for all nucleocytoplasmic glycosylation and there is a well-known need for potent, cell-permeable inhibitors to interrogate OGT function. Here we report the structure-based evolution of OGT inhibitors culminating in compounds with low nanomolar inhibitory potency and on-target cellular activity. In addition to disclosing useful OGT inhibitors, the structures we report provide insight into how to inhibit glycosyltransferases, a family of enzymes that has been notoriously refractory to inhibitor development.