A translating ribosome is typically thought to follow the reading frame defined by the selected start codon. Using super-resolution ribosome profiling, here we report pervasive out-of-frame translation immediately from the start codon. The start codon-associated ribosome frameshifting (SCARF) stems from the slippage of ribosomes during the transition from initiation to elongation. Using a massively paralleled reporter assay, we uncovered sequence elements acting as SCARF enhancers or repressors, implying that start codon recognition is coupled with reading frame fidelity. This finding explains thousands of mass spectrometry spectra unannotated from human proteome. Mechanistically, we find that the eukaryotic initiation factor 5B (eIF5B) maintains the reading frame fidelity by stabilizing initiating ribosomes. Intriguingly, amino acid starvation induces SCARF by proteasomal degradation of eIF5B. The stress-induced SCARF protects cells from starvation by enabling amino acid recycling and selective mRNA translation. Our findings illustrate a beneficial effect of translational "noise" in nutrient stress adaptation.