Inhibitor of apoptosis (IAP) proteins are characterized by the presence of the conserved baculoviral IAP repeat (BIR) domain that is involved in protein-protein interactions. IAPs were initially thought to be mainly responsible for caspase inhibition, acting as negative regulators of apoptosis, but later works have shown that IAPs also control a plethora of other different cellular pathways. As X-linked IAP (XIAP), and other IAP, levels are often deregulated in cancer cells and have been shown to correlate with patients' prognosis, several approaches have been pursued to inhibit their activity in order to restore apoptosis. Many small molecules have been designed to target the BIR domains, the vast majority being inspired by the N-terminal tetrapeptide of Second Mitochondria-derived Activator of Caspases/Direct IAp Binding with Low pI (Smac/Diablo), which is the natural XIAP antagonist. These compounds are therefore usually referred to as Smac mimetics (SMs). Despite the fact that SMs were intended to specifically target XIAP, it has been shown that they also interact with cellular IAP-1 (cIAP1) and cIAP2, promoting their proteasome-dependent degradation. SMs have been tested in combination with several cytotoxic compounds and are now considered promising immune modulators which can be exploited in cancer therapy, especially in combination with immune checkpoint inhibitors. In this review, we give an overview of the structural hot-spots of BIRs, focusing on their fold and on the peculiar structural patches which characterize the diverse BIRs. These structures are exploited/exploitable for the development of specific and active IAP inhibitors.