Although hundreds of plant lineages have independently evolved dioecy (that is, separation of the sexes), the underlying genetic basis remains largely elusive1. Here we show that diverse poplar species carry partial duplicates of the ARABIDOPSIS RESPONSE REGULATOR 17 (ARR17) orthologue in the male-specific region of the Y chromosome. These duplicates give rise to small RNAs apparently causing male-specific DNA methylation and silencing of the ARR17 gene. CRISPR-Cas9-induced mutations demonstrate that ARR17 functions as a sex switch, triggering female development when on and male development when off. Despite repeated turnover events, including a transition from the XY system to a ZW system, the sex-specific regulation of ARR17 is conserved across the poplar genus and probably beyond. Our data reveal how a single-gene-based mechanism of dioecy can enable highly dynamic sex-linked regions and contribute to maintaining recombination and integrity of sex chromosomes.