DDX3X is a DEAD-box RNA helicase that has been implicated in multiple aspects of RNA metabolism including translation initiation and the assembly of stress granules (SGs). Recent genomic studies have reported recurrent DDX3X mutations in numerous tumors including medulloblastoma (MB), but the physiological impact of these mutations is poorly understood. Here we show that a consistent feature of MB-associated mutations is SG hyper-assembly and concomitant translation impairment. We used CLIP-seq to obtain a comprehensive assessment of DDX3X binding targets and ribosome profiling for high-resolution assessment of global translation. Surprisingly, mutant DDX3X expression caused broad inhibition of translation that impacted DDX3X targeted and non-targeted mRNAs alike. Assessment of translation efficiency with single-cell resolution revealed that SG hyper-assembly correlated precisely with impaired global translation. SG hyper-assembly and translation impairment driven by mutant DDX3X were rescued by a genetic approach that limited SG assembly and by deletion of the N-terminal low complexity domain within DDX3X. Thus, in addition to a primary defect at the level of translation initiation caused by DDX3X mutation, SG assembly itself contributes to global translation inhibition. This work provides mechanistic insights into the consequences of cancer-related DDX3X mutations, suggesting that globally reduced translation may provide a context-dependent survival advantage that must be considered as a possible contributor to tumorigenesis.