Transformations of neural representations in a social behaviour network

Abstract

Mating and aggression are innate social behaviours that are controlled by subcortical circuits in the extended amygdala and hypothalamus^1-4. The bed nucleus of the stria terminalis (BNSTpr) is a node that receives input encoding sex-specific olfactory cues from the medial amygdala^5,6, and which in turn projects to hypothalamic nuclei that control mating^7-9 (medial preoptic area (MPOA)) and aggression^9-14 (ventromedial hypothalamus, ventrolateral subdivision (VMHvl)), respectively¹⁵. Previous studies have demonstrated that male aromatase-positive BNSTpr neurons are required for mounting and attack, and may identify conspecific sex according to their overall level of activity¹⁶. However, neural representations in BNSTpr, their function and their transformations in the hypothalamus have not been characterized. Here we performed calcium imaging^17,18 of male BNSTpr^Esr1 neurons during social behaviours. We identify distinct populations of female- versus male-tuned neurons in BNSTpr, with the former outnumbering the latter by around two to one, similar to the medial amygdala and MPOA but opposite to VMHvl, in which male-tuned neurons predominate^6,9,19. Chemogenetic silencing of BNSTpr^Esr1 neurons while imaging MPOA^Esr1 or VMHvl^Esr1 neurons in behaving animals showed, unexpectedly, that the male-dominant sex-tuning bias in VMHvl was inverted to female-dominant whereas a switch from sniff- to mount-selective neurons during mating was attenuated in MPOA. Our data also indicate that BNSTpr^Esr1 neurons are not essential for conspecific sex identification. Rather, they control the transition from appetitive to consummatory phases of male social behaviours by shaping sex- and behaviour-specific neural representations in the hypothalamus.