An emerging literature has suggested that self-relevance automatically enhances stimulus processing (i.e., the self-prioritization effect). Specifically, during shape-label matching tasks, geometric shapes associated with the self are identified more rapidly than comparable stimuli paired with other targets (e.g., friend, stranger). Replicating and extending work that challenges the putative automaticity of this effect, here we hypothesized that self-relevance facilitates stimulus processing only when task sets draw attention to previously formed shape-label associations in memory. The results of a shape-classification task confirmed this prediction. Compared to shapes associated with a friend, those paired with the self were classified more rapidly when participants were required to report who the stimulus denoted (i.e., self or friend). In contrast, self-relevance failed to facilitate performance when participants judged either what the shape was (i.e., triangle or square, diamond or circle) or where it was located on the screen (i.e., above or below fixation). These findings further elucidate the conditions under which self-relevance does-and does not-influence stimulus processing.