Crucial to the function of Hsp70 chaperones is the nucleotide-regulated transition between two conformational states, the ATP bound state with high association and dissociation rates for substrates and the ADP bound state with two and three orders of magnitude lower association and dissociation rates. The spontaneous transition between the two states is extremely slow, indicating a high energy barrier for the switch that regulates the transition. Here we provide evidence that a universally conserved proline in the ATPase domain constitutes the switch that assumes alternate conformations in response to ATP binding and hydrolysis. The conformation of the proline, acting through an invariant arginine as relay, determines and stabilizes the opened and closed conformation of the substrate binding domain and thereby regulates the chaperone activity of Hsp70.