The sense of agency refers to the feeling that one is controlling events through one's own behavior. This study examined how task performance and the delay of events influence one's sense of agency during continuous action accompanied by a goal. The participants were instructed to direct a moving dot into a square as quickly as possible by pressing the left and right keys on a keyboard to control the direction in which the dot traveled. The interval between the key press and response of the dot (i.e., direction change) was manipulated to vary task difficulty. Moreover, in the assisted condition, the computer ignored participants' erroneous commands, resulting in improved task performance but a weaker association between the participants' commands and actual movements of the dot relative to the condition in which all of the participants' commands were executed (i.e., self-control condition). The results showed that participants' sense of agency increased with better performance in the assisted condition relative to the self-control condition, even though a large proportion of their commands were not executed. We concluded that, when the action-feedback association was uncertain, cognitive inference was more dominant relative to the process of comparing predicted and perceived information in the judgment of agency.