This study analyzed differences between men and women regarding the use of psychotropic drugs and associated factors in a population of adults and seniors in the city of Campinas, Brazil. A population-based, cross-sectional study was conducted using data from the ISACamp 2014/2015 health survey in the city of Campinas. The sample was composed of 1999 individuals aged 20 years or older. For each sex, prevalence rates and prevalence ratios were estimated for the use of psychotropic drugs according to demographic characteristics, socioeconomic characteristics, health problems, degree of limitation and type of emotional/mental problem. The most used classes of medications were also determined. The prevalence of the use of psychotropic drugs was 11.7% (7.3% among men and 15.8% among women). The most common therapeutic classes were antidepressants (38.2%) and benzodiazepines (24.0%). The frequency of antidepressant use was higher among women (44.3%) than men (25.5%). Regarding associated factors, reports of emotional/mental problems were associated with the greater use of this type of drug in both sexes. Among the men, white skin color, a lack of an occupational activity, a greater number of complaints of health problems and the occurrence of insomnia were associated with the use psychotropic drugs. Among the women, a significant increase in the use of these drugs was found with the increase in age and higher prevalence rates were found among those with a higher level of schooling, those with a greater number of diagnosed chronic diseases and those with a common mental disorder. The present results confirm the greater use of psychotropic agents, especially antidepressants, in the female sex and reveal that the pattern of associated factors differs between sexes. It is therefore necessary to understand the peculiarities of each sex that exert an influence on the perception of health problems and the desire to seek care, which, in turn, affect the use of psychotropic agents.