Early auditory deprivation may drive the auditory cortex into cross-modal processing of non-auditory sensory information. In a recent study, we had shown that early deaf subjects exhibited increased activation in the superior temporal gyrus (STG) bilaterally during visual spatial working memory; however, the changes in the organization of the STG related spontaneous functional network, and their cognitive relevance are still unknown. To clarify this issue, we applied resting state functional magnetic resonance imaging on 42 early deafness (ED) and 40 hearing controls (HC). We also acquired the visual spatial and numerical n-back working memory (WM) information in these subjects. Compared with hearing subjects, the ED exhibited faster reaction time of visual WM tasks in both spatial and numerical domains. Furthermore, ED subjects exhibited significantly increased functional connectivity between the STG (especially of the right hemisphere) and bilateral anterior insula and dorsal anterior cingulated cortex. Finally, the functional connectivity of STG could predict visual spatial WM performance, even after controlling for numerical WM performance. Our findings suggest that early auditory deprivation can strengthen the spontaneous functional connectivity of STG, which may contribute to the cross-modal involvement of this region in visual working memory.