Androgen receptor (AR) plays important roles in the development of prostate cancer (PCa). The antagonistic drugs, which suppress the activity of AR, are widely used in the treatment of PCa. However, the molecular mechanism of antagonism about how ligands affect the structures of AR remains elusive. To better understand the conformational variability of ARs bound with agonists or antagonists, we performed long time unbiased molecular dynamics (MD) simulations and enhanced sampling simulations for the ligand binding domain of AR (AR-LBD) in complex with various ligands. Based on the simulation results, we proposed an allosteric pathway linking ligands and helix 12 (H12) of AR-LBD, which involves the interactions among the ligands and the residues W741, H874, and I899. The interaction pathway provides an atomistic explanation of how ligands affect the structure of AR-LBD. A repositioning of H12 was observed, but it is facilitated by the C-terminal of H12, instead of by the loop between helix 11 (H11) and H12. The bias-exchange metadynamics simulations further demonstrated the above observations. More importantly, the free energy profiles constructed by the enhanced sampling simulations revealed the transition process between the antagonistic form and agonistic form of AR-LBD. Our results would be helpful for the design of more efficient antagonists of AR to combat PCa.