Histone deacetylases (HDACs) are a family of conserved metalloproteases which play a key role in the development of cancer. They can be divided into 18 subtypes according to their structural diversity. Histone deacetylase inhibitors are considered as potential anti-cancer agents and a lot of pan-HDAC inhibitors have entered clinical trials. Selective HDAC inhibitors targeting only one member or one class subtype are less exploited at present and regarded less toxic as well as more tolerable than pan-HDAC inhibitors. Certain structural modifications or new moieties may help to acquire isoform selectivity. In this review, we will focus on each member of HDACs and selective HDAC inhibitors as well as the relationship between structure and selectivity.