Navigated transcranial magnetic stimulation (TMS) combined with diffusion-weighted magnetic resonance imaging (DW-MRI) and tractography allows investigating functional anatomy of the human brain with high precision. Here we demonstrate that working memory (WM) processing of tactile temporal information is facilitated by delivering a single TMS pulse to the middle frontal gyrus (MFG) during memory maintenance. Facilitation was obtained only with a TMS pulse applied to a location of the MFG with anatomical connectivity to the primary somatosensory cortex (S1). TMS improved tactile WM also when distractive tactile stimuli interfered with memory maintenance. Moreover, TMS to the same MFG site attenuated somatosensory evoked responses (SEPs). The results suggest that the TMS-induced memory improvement is explained by increased top-down suppression of interfering sensory processing in S1 via the MFG-S1 link. These results demonstrate an anatomical and functional network that is involved in maintenance of tactile temporal WM.