Viral macrodomains that can bind to or hydrolyze protein adenosine diphosphate ribosylation (ADP-ribosylation) have emerged as promising targets for antiviral drug development. Many inhibitor development efforts have been directed against the severe acute respiratory syndrome coronavirus 2 macrodomain 1 (SARS-CoV-2 Mac1). However, potent inhibitors for viral macrodomains are still lacking, with the best inhibitors still in the micromolar range. Based on GS-441524, a remdesivir precursor, and our previous studies, we have designed and synthesized potent binders of SARS-CoV-2 Mac1 and other viral macrodomains including those of Middle East respiratory syndrome coronavirus (MERS-CoV), Venezuelan equine encephalitis virus (VEEV), and Chikungunya virus (CHIKV). We show that the 1'-CN group of GS-441524 promotes binding to all four viral macrodomains tested while capping the 1″-OH of GS-441524-diphosphate-ribose with a simple phenyl ring further contributes to binding. Incorporating these two structural features, the best binders show 20- to 6000-fold increases in binding affinity over ADP-ribose for SARS-CoV-2, MERS-CoV, VEEV, and CHIKV macrodomains. Moreover, building on these potent binders, we have developed two highly sensitive fluorescence polarization tracers that only require nanomolar proteins and can effectively resolve the binding affinities of nanomolar inhibitors. Our findings and probes described here will facilitate future development of more potent viral macrodomain inhibitors.