When people observe others performing actions similar to their own while dancing or playing musical instruments, they sometimes feel as if their actions were subsumed into others' actions or others' actions led their own actions. Many studies have been conducted to investigate agency attribution. However, these studies have mainly examined agency attribution in cases where people do not know the true agent. Few studies have focused on how people attribute agency to others despite knowing that they themselves are actual agents. This study investigates agency attribution to others performing actions similar to one's own when one knows who the actual agent is. We evaluated agency attribution when participants manipulated a mouse to control a cursor while observing another person performing similar actions. Our findings demonstrated that participants could attribute agency to others despite knowing that they themselves were actual agents. We refer to this illusory sense as "illusory agency attribution to others." We suggest that illusory agency attribution to others is determined by multiple factors including a bottom-up process with a subjective feeling of agency in addition to a top-down process with an interpretative judgement of agency.