Histone deacetylases (HDACs) are enzymes pursued as drug targets in various cancers and several non-oncological conditions, such as inflammation and neurodegenerative disorders. In the past decade, HDAC inhibitors (HDACi) have emerged as relevant pharmaceuticals, with many efforts devoted to the development of new representatives. However, the growing safety concerns regarding the established hydroxamic acid-based HDAC inhibitors tend to drive current research more toward the design of inhibitors bearing alternative zinc-binding groups (ZBGs). This Perspective presents an overview of all non-hydroxamic acid ZBGs that have been incorporated into the clinically approved prototypical HDACi, suberoylanilide hydroxamic acid (vorinostat). This provides the unique opportunity to compare the inhibition potential and biological effects of different ZBGs in a direct way, as the compounds selected for this Perspective differ only in their ZBG. To that end, different strategies used to select a ZBG, its properties, activity, and liabilities are discussed.