Recent functional neuroimaging studies have shown that multiple cortical areas are involved in memory encoding and retrieval. However, the underlying anatomical connections among these memory-related areas in humans remain elusive due to methodological limitations. Diffusion tensor imaging (DTI) is a technique based on detecting the diffusion of water molecules from magnetic resonance images. DTI allows non-invasive mapping of anatomical connections and gives a comprehensive picture of connectivity throughout the entire brain. By combining functional magnetic resonance imaging (fMRI) and DTI, we show that memory-related areas in the left dorsolateral prefrontal cortex (DLPFC) and the left ventrolateral prefrontal cortex (VLPFC) each connect with memory-related areas in the left temporal cortex. This result suggests there are two pathways between prefrontal cortex and temporal cortex related to the human memory system.