In 1998, Tucker and Ellis found that keypress responses are faster when the task-irrelevant orientation of a graspable object's handle corresponds to response hand location. Over the past 20 years, researchers have disagreed over the extent to which grasping affordance or spatial compatibility contributes to the effect. One of the causes behind the conflicting findings and interpretations may be that studies advocating the grasping affordance view have tended to overlook the contributions of low-level perceptual characteristics to the observed correspondence effects. The present study evaluated the role of visual salience and bottom-up attention in the occurrence of the effect. Experiment 1 involved a vertical orientation task (bimanual keypresses) using photographs of graspable objects, centered based on object width or pixel area. The same procedure was performed using a color discrimination task on solid-colored silhouettes, large (Experiment 2) and small-sized (Experiment 3), as well as silhouette outlines (Experiment 4). Similar result patterns across Experiments 1-3 were observed and discussed in the context of diverging findings in Experiment 4, prompting us to introduce the notion of asymmetry-based Simon effects, whereby location is coded at stimulus onset on the basis of asymmetric changes in luminance between both hemifields, coupled with object-center eccentricity relative to fixation. These low-level factors were critical in modulating the temporal dynamics, as well as the direction of compatibility effects (toward handles or bodies), irrespective of grasping affordance, task, object identity, or stimulus size. The present findings provide further evidence that the Tucker and Ellis paradigm for studying variable affordances is extremely vulnerable to location-coding, which may arise on the basis of exogenous deployments of attention. This problem is only exacerbated in a large portion of the relevant literature, whereby visually complex stimuli are primarily discussed in terms of their graspable nature and relation to task, rather than their low-level, attention-capturing features.