Cognitive control processes that enable purposeful behavior are often context-specific. A teenager, for example, may inhibit the tendency to daydream at work but not in the classroom. However, the nature of contextual boundaries for cognitive control processes remains unclear. Therefore, we revisited an ongoing controversy over whether such boundaries reflect (a) an attentional reset that occurs whenever a context-defining (e.g., sensory) feature changes or (b) a disruption of episodic memory retrieval that occurs only when the updated context-defining feature is linked to a different task set. To distinguish between these hypotheses, we used a cross-modal distractor-interference task to determine precisely when changing a salient context-defining feature-the sensory modality in which task stimuli appear-bounds control processes underlying the congruency sequence effect (CSE). Consistent with the task set hypothesis, but not with the attentional reset hypothesis, Experiments 1 and 2 revealed that changing the sensory modality in which task stimuli appear eliminates the CSE only when the task structure enables participants to form modality-specific task sets. Experiment 3 further revealed that such "modality-specific" CSEs are associated with orienting attention to the sensory modality in which task stimuli appear, which may facilitate the formation of a modality-specific task set. These findings support the view that task sets serve as boundaries for the CSE. (PsycInfo Database Record (c) 2020 APA, all rights reserved).