The presence of large and near-identical inverted repeat sequences (called palindromes) is a common feature of the constitutively haploid sex chromosomes of different species. Despite the fact palindromes originated in a non-recombining context, they have evolved a strong recombinational activity in the form of abundant arm-to-arm gene conversion. Their independent appearance in different species suggests they can have a profound biological significance that has yet to be fully clarified. It has been theorized that natural selection may have favored palindromic organization of male-specific genes and that the establishment of intra-palindrome gene conversion has strong adaptive significance. Arm-to-arm gene conversion allows the efficient removal of deleterious mutations, increases the fixation rate of beneficial mutations and has played an important role in modulating the equilibrium between gene loss and acquisition during Y chromosome evolution. Additionally, a palindromic organization of duplicates could favor the formation of unusual chromatin structures and could optimize the use of gene conversion as a mechanism to maintain the structural integrity of male-specific genes. In this review, we describe the structural features of palindromes on mammalian sex chromosomes and summarize different hypotheses regarding palindrome evolution and the functional benefits of arm-to-arm gene conversion on the unique haploid portion of the nuclear genome.