AMPA glutamate receptors (AMPARs) mediate fast excitatory synaptic transmission. Upon fast consecutive synaptic stimulation, transmission can be depressed. Recuperation from fast synaptic depression has been attributed solely to recovery of transmitter release and/or AMPAR desensitization. We show that AMPAR lateral diffusion, observed in both intact hippocampi and cultured neurons, allows fast exchange of desensitized receptors with naïve functional ones within or near the postsynaptic density. Recovery from depression in the tens of millisecond time range can be explained in part by this fast receptor exchange. Preventing AMPAR surface movements through cross-linking, endogenous clustering, or calcium rise all slow recovery from depression. Physiological regulation of postsynaptic receptor mobility affects the fidelity of synaptic transmission by shaping the frequency dependence of synaptic responses.