Androgen receptor (AR) is a steroid hormone nuclear receptor which upon binding its endogenous androgenic ligands (agonists), testosterone and dihydrotestosterone (DHT), alters gene transcription, producing a diverse range of biological effects. Antiandrogens, such as the pharmaceuticals bicalutamide and hydroxyflutamide, act as agonists in the absence of androgens and as antagonists in their presence or in high concentration. The atomic level mechanism of action by agonists and antagonists of AR is less well characterized. Therefore, in this study, multiple 1 μs molecular dynamics (MD), docking simulations, and perturbation-response analyses were performed to more fully explore the nature of interaction between agonist or antagonist and AR and the conformational changes induced in the AR upon interaction with different ligands. We characterized the mechanism of the ligand entry/exit and found that helix-12 and nearby structural motifs respond dynamically in that process. Modeling showed that the agonist and antagonist/agonist form a hydrogen bond with Thr877/Asn705 and that this interaction is absent for antagonists. Agonist binding to AR increases the mobility of residues at allosteric sites and coactivator binding sites, while antagonist binding decreases mobility at these important sites. A new site was also identified as a potential surface for allosteric binding. These results shed light on the effect of agonists and antagonists on the structure and dynamics of AR.