Abacus experts are able to mentally calculate multi-digit numbers rapidly. Some behavioral and neuroimaging studies have suggested a visuospatial and visuomotor strategy during abacus mental calculation. However, no study up to now has attempted to dissociate temporally the visuospatial neural process from the visuomotor neural process during abacus mental calculation. In the present study, an abacus expert performed the mental addition tasks (8-digit and 4-digit addends presented in visual or auditory modes) swiftly and accurately. The 100% correct rates in this expert's task performance were significantly higher than those of ordinary subjects performing 1-digit and 2-digit addition tasks. ERPs, EEG source localizations, and fMRI results taken together suggested visuospatial and visuomotor processes were sequentially arranged during the abacus mental addition with visual addends and could be dissociated from each other temporally. The visuospatial transformation of the numbers, in which the superior parietal lobule was most likely involved, might occur first (around 380 ms) after the onset of the stimuli. The visuomotor processing, in which the superior/middle frontal gyri were most likely involved, might occur later (around 440 ms). Meanwhile, fMRI results suggested that neural networks involved in the abacus mental addition with auditory stimuli were similar to those in the visual abacus mental addition. The most prominently activated brain areas in both conditions included the bilateral superior parietal lobules (BA 7) and bilateral middle frontal gyri (BA 6). These results suggest a supra-modal brain network in abacus mental addition, which may develop from normal mental calculation networks.