High cholesterol levels are associated with prostate cancer development. Androgens promote cholesterol accumulation by activating the sterol-regulatory element-binding protein isoform 2 (SREBP-2) transcription factor. However, SREBP-2 is in balance with the liver X receptor (LXR; NR1H2/NR1H3), a transcription factor that prevents cholesterol accumulation. Here, we show that LXR activity is down-regulated by the androgen receptor (AR; NR3C4). In turn, this reduces LXR target gene expression. This antagonism on LXR is also exerted by other steroid hormone receptors, including the estrogen, glucocorticoid, and progesterone receptors. This suggests a generalizable mechanism, but the AR does not affect LXR mRNA levels, protein degradation, or DNA binding. We also found that the AR does not require protein synthesis to influence LXR, suggesting a direct antagonism. However, the AR does not directly bind LXR. The AR N-terminal domain (involved in transactivation), but not its DNA-binding domain, is required to suppress LXR activity, suggesting coactivator competition. Overall, this androgen-mediated antagonism of LXR complements SREBP-2 activation, providing a more complete picture as to how androgens increase cellular cholesterol levels in a prostate cancer setting. Given the cross-talk between other steroid hormone receptors and LXR, hormonal regulation of cholesterol via LXR may occur in a variety of cellular contexts.