We explored mechanisms of cross-modal priming between visual and haptic modalities. Specifically, we investigated a mechanism of the visual-to-haptic transfer (Experiment 1) and vice versa (Experiment 2). In Experiment 1, three experimental groups, presented visual prime stimuli using novel three-line patterns, were asked to form visual images matching only with shape, haptic images matching only with shape, or haptic images matching with both shape and texture of haptic targets. Priming occurred only when induced haptic images of prime stimuli coincided with the actual texture of haptic targets. In Experiment 2, two experimental groups, presented haptic prime stimuli, were asked to form visual images matching only with shape, or visual images matching with both shape and material (i.e., monochromatic contrast between foreground and background) of visual targets. Priming occurred regardless of experimental conditions, including the control group. Thus, both shape and material representations significantly contributed to the visual-to-haptic transfer. Contrastingly, only shape representation played a significant role in the haptic-to-visual transfer.