Serotonin (5-HT) inhibits aggression and modulates aspects of sexual behaviour in many species, but the mechanisms responsible are not well understood. Here, we exploited the social dominance hierarchy of Astatotilapia burtoni to understand the role of the serotonergic system in long-term maintenance of social status. We identified three populations of 5-HT cells in dorsal and ventral periventricular pretectal nuclei (PPd, PPv), the nucleus of the paraventricular organ (PVO) and raphe. Dominant males had more 5-HT cells than subordinates in the raphe, but the size of these cells did not differ between social groups. Subordinates had higher serotonergic turnover in the raphe and preoptic area (POA), a nucleus essential for hypothalamic-pituitary-gonadal (HPG) axis function. The relative abundance of mRNAs for 5-HT receptor (5-HTR) subtypes 1A and 2A (htr1a, htr2a) was higher in subordinates, a difference restricted to the telencephalon. Because social status is tightly linked to reproductive capacity, we asked whether serotonin turnover and the expression of its receptors correlated with testes size and circulating levels of 11-ketotestosterone (11-KT). We found negative correlations between both raphe and POA serotonin turnover and testes size, as well as between htr1a mRNA levels and circulating 11-KT. Thus, increased serotonin turnover in non-aggressive males is restricted to specific brain nuclei and is associated with increased expression of 5-HTR subtypes 1A and 2A exclusively in the telencephalon.
Keywords: Aggression; Behavioral plasticity; Dominance hierarchy; Serotonin.
© 2014. Published by The Company of Biologists Ltd.