O-GlcNAc transferase (OGT) attaches a GlcNAc moiety on specific substrate proteins using UDP-GlcNAc as the sugar donor. This modification can alter protein function by regulating cellular signaling and transcription pathways in response to altered nutrient availability and stress. Specific inhibitors of OGT would be valuable tools for biological studies and lead structures for therapeutics. The existing OGT inhibitors are mainly derived from the sugar donor substrate, but poor cell permeability and off-target effects limit their use. Here, we describe our progress on OGT inhibition based on substrate peptides identified by array screening. Subsequently, bisubstrate inhibitors were prepared by conjugating these peptides to uridine in various ways. In parallel, an in silico fragment screening was conducted to obtain small molecules targeting the UDP binding pocket. After evaluation of the initial hits, one of these small molecules was elaborated into a novel OGT hybrid inhibitor, as the replacement of uridine. The novel compounds inhibit OGT activity with IC50 values in the micromolar range.