Human brains have a remarkable ability to separate streams of visual input into distinct memory-traces. It is unclear, however, how this ability relates to the way these inputs are explored via unique gaze-patterns. Moreover, it is yet unknown how motivation to forget or remember influences the link between gaze similarity and memory. In two experiments, we used a modified directed-forgetting paradigm and either showed blurred versions of the encoded scenes (Experiment 1) or pink noise images (Experiment 2) during attempted memory control. Both experiments demonstrated that higher levels of across-stimulus gaze similarity relate to worse future memory. Although this across-stimulus interference effect was unaffected by motivation, it depended on the perceptual overlap between stimuli and was more pronounced for different scene comparisons, than scene-pink noise comparisons. Intriguingly, these findings echo the pattern similarity effects from the neuroimaging literature and pinpoint a mechanism that could aid the regulation of unwanted memories.