The endosome-to-Golgi or endocytic retrograde trafficking pathway is an important post-Golgi recycling route. Here we show that amino acids (AAs) can stimulate the retrograde trafficking and regulate the cell surface localization of certain Golgi membrane proteins. By testing components of the AA-stimulated mTORC1 signaling pathway, we demonstrate that SLC38A9, v-ATPase and Ragulator, but not Rag GTPases and mTORC1, are essential for the AA-stimulated trafficking. Arl5, an ARF-like family small GTPase, interacts with Ragulator in an AA-regulated manner and both Arl5 and its effector, the Golgi-associated retrograde protein complex (GARP), are required for the AA-stimulated trafficking. We have therefore identified a mechanistic connection between the nutrient signaling and the retrograde trafficking pathway, whereby SLC38A9 and v-ATPase sense AA-sufficiency and Ragulator might function as a guanine nucleotide exchange factor to activate Arl5, which, together with GARP, a tethering factor, probably facilitates the endosome-to-Golgi trafficking.