The mTOR complex 1 (mTORC1) pathway promotes cell growth in response to many cues, including amino acids, which act through the Rag guanosine triphosphatases (GTPases) to promote mTORC1 translocation to the lysosomal surface, its site of activation. Although progress has been made in identifying positive regulators of the Rags, it is unknown if negative factors also exist. Here, we identify GATOR as a complex that interacts with the Rags and is composed of two subcomplexes we call GATOR1 and -2. Inhibition of GATOR1 subunits (DEPDC5, Nprl2, and Nprl3) makes mTORC1 signaling resistant to amino acid deprivation. In contrast, inhibition of GATOR2 subunits (Mios, WDR24, WDR59, Seh1L, and Sec13) suppresses mTORC1 signaling, and epistasis analysis shows that GATOR2 negatively regulates DEPDC5. GATOR1 has GTPase-activating protein (GAP) activity for RagA and RagB, and its components are mutated in human cancer. In cancer cells with inactivating mutations in GATOR1, mTORC1 is hyperactive and insensitive to amino acid starvation, and such cells are hypersensitive to rapamycin, an mTORC1 inhibitor. Thus, we identify a key negative regulator of the Rag GTPases and reveal that, like other mTORC1 regulators, Rag function can be deregulated in cancer.