Although research has shown a connection between minority stressors and internalizing mental health problems, the role of minority stress has mainly been neglected in the assessment of sexual problems among non-heterosexual men. Using online samples of heterosexual (n = 933) and non-heterosexual participants (n = 561) aged 18 to 50 years, this study aimed to comparatively assess sexual difficulties and problems and explore the role of minority stress in non-heterosexual men's sexual problems. Although the age-adjusted odds of reporting rapid ejaculation, delayed ejaculation, and sex-related anxiousness significantly differed between the two groups, the overall prevalence of sexual difficulties and the associated levels of distress did not significantly differ between the samples. In multivariate assessment, anxiety and depression significantly increased the odds of reporting distressing sexual difficulties among both heterosexual and non-heterosexual participants. In the non-heterosexual sample, positive body image significantly decreased the odds of experiencing sexual problems. Pointing to a role of minority stress, highest levels of victimization related to sexual orientation increased the risk of sexual problems. This association was partially mediated by negative emotions. Our findings offer some support for a recent call to include sexual orientation among the social determinants of health recognized by the World Health Organization.