How does visual perception shape the way we coordinate movements? Recent studies suggest that the brain organizes movements based on minimizing reaching errors in the presence of motor and sensory noise. We present an alternative hypothesis in which movement trajectories also result from acquired knowledge about the geometrical properties of the object that the brain is controlling. To test this hypothesis, we asked human subjects to control a simulated kinematic linkage by continuous finger motion, a completely novel experience. This paradigm removed all biases arising from influences of limb dynamics and past experience. Subjects were exposed to two different types of visual feedback; some saw the entire simulated linkage and others saw only the moving extremity. Consistent with our hypothesis, subjects learned to move the simulated linkage along geodesic lines corresponding to the geometrical structure of the observed motion. Thus, optimizing final accuracy is not the unique determinant of trajectory formation.