When compatible and incompatible mappings of a location-relevant task are mixed, or a location-relevant task is mixed with a task for which stimulus location is irrelevant, the benefit of the compatible mapping is eliminated for physical locations and enhanced for location words. Two experiments examined the influence of presenting the location information for the mixed conditions in different stimulus modes (physical location or word). Experiment 1 showed that the effects of mixing location-relevant and location-irrelevant tasks on the spatial compatibility and Simon effects are reduced when the location information is presented in different modes for the two tasks. Experiment 2 showed, in contrast, that the mode distinction had little influence on the effects of mixed compatible and incompatible mappings for location-relevant tasks: The compatibility effect was eliminated for physical locations and enhanced for words, as when there is no mode distinction. Thus, when location is relevant for one task and colour for the other, the task-defined associations of locations to responses are mode specific, but when location is relevant for both tasks, the associations are mode independent.