Animals display remarkable variation in social behaviour. However, outside of rodents, little is known about the neural mechanisms of social variation, and whether they are shared across species and sexes, limiting our understanding of how sociality evolves. Using coral reef butterflyfishes, we examined gene expression correlates of social variation (i.e. pair bonding versus solitary living) within and between species and sexes. In several brain regions, we quantified gene expression of receptors important for social variation in mammals: oxytocin (OTR), arginine vasopressin (V1aR), dopamine (D1R, D2R) and mu-opioid (MOR). We found that social variation across individuals of the oval butterflyfish, Chaetodon lunulatus, is linked to differences in OTR,V1aR, D1R, D2R and MOR gene expression within several forebrain regions in a sexually dimorphic manner. However, this contrasted with social variation among six species representing a single evolutionary transition from pair-bonded to solitary living. Here, OTR expression within the supracommissural part of the ventral telencephalon was higher in pair-bonded than solitary species, specifically in males. These results contribute to the emerging idea that nonapeptide, dopamine and opioid signalling is a central theme to the evolution of sociality across individuals, although the precise mechanism may be flexible across sexes and species.
Keywords: butterflyfish; nonapeptides; pair bond; sex differences; species differences.