The flow of information through the epidermal growth factor receptor (EGFR) is shaped by molecular interactions in the plasma membrane. The EGFR is associated with lipid rafts, but their role in modulating receptor mobility and subsequent interactions is unclear. To investigate the role of nanoscale rafts in EGFR dynamics, we used single-molecule fluorescence imaging to track individual receptors and their dimerization partner, human epidermal growth factor receptor 2 (HER2), in the membrane of human mammary epithelial cells. We found that the motion of both receptors was interrupted by dwellings within nanodomains. EGFR was significantly less mobile than HER2. This difference was likely due to F-actin because its depolymerization led to similar diffusion patterns between the EGFR and HER2. Manipulations of membrane cholesterol content dramatically altered the diffusion pattern of both receptors. Cholesterol depletion led to almost complete confinement of the receptors, whereas cholesterol enrichment extended the boundaries of the restricted areas. Interestingly, F-actin depolymerization partially restored receptor mobility in cholesterol-depleted membranes. Our observations suggest that membrane cholesterol provides a dynamic environment that facilitates the free motion of EGFR and HER2, possibly by modulating the dynamic state of F-actin. The association of the receptors with lipid rafts could therefore promote their rapid interactions only upon ligand stimulation.