Human facial attractiveness and facial sexual dimorphism (masculinity-femininity) are important facets of mate choice and are hypothesized to honestly advertise genetic quality. However, it is unclear whether genes influencing facial attractiveness and masculinity-femininity have similar, opposing, or independent effects across sex, and the heritability of these phenotypes is poorly characterized. To investigate these issues, we assessed facial attractiveness and facial masculinity-femininity in the largest genetically informative sample (n = 1,580 same- and opposite-sex twin pairs and siblings) to assess these questions to date. The heritability was ~0.50-0.70 for attractiveness and ~0.40-0.50 for facial masculinity-femininity, indicating that, despite ostensible selection on genes influencing these traits, substantial genetic variation persists in both. Importantly, we found evidence for intralocus sexual conflict, whereby alleles that increase masculinity in males have the same effect in females. Additionally, genetic influences on attractiveness were shared across the sexes, suggesting that attractive fathers tend to have attractive daughters and attractive mothers tend to have attractive sons.