Fragile X syndrome (FXS) is the most common inherited cause of intellectual disability. Although language delays are frequently observed in FXS, neither the longitudinal course of language development nor its cognitive predictors are well understood. The present study investigated whether phonological and working memory skills are predictive of growth in vocabulary and syntax in individuals with FXS during adolescence. Forty-four individuals with FXS (mean age = 12.61 years) completed assessments of phonological memory (nonword repetition and forward digit recall), verbal working memory (backward digit recall), vocabulary, syntax, and nonverbal cognition. Vocabulary and syntax skills were reassessed at a 2-year follow-up. In a series of analyses that controlled for nonverbal cognitive ability and severity of autism symptoms, the relative contributions of phonological and working memory to language change over time were investigated. These relationships were examined separately for boys and girls. In boys with FXS, phonological memory significantly predicted gains in vocabulary and syntax skills. Further, verbal working memory was uniquely associated with vocabulary gains among boys. In girls with FXS, phonological and working memory skills showed no relationship with language change across the 2-year time period. Our findings indicate that, for adolescent boys with FXS, acquisition of vocabulary and syntax may be constrained by the ability to maintain and manipulate phonological representations online. Implications for the identification and treatment of language disorders in this population are discussed. The present study is the first to identify specific cognitive mechanisms contributing to language growth over time in individuals with FXS.