As one of the most common synergies between the exteroceptors and proprioceptors, the synergy between visual and vestibule enables the human brain to judge the state of human motion, which is essential for motion recognition and human self-protection. Hence, in this work, an artificial motion sensory system (AMSS) based on artificial vestibule and visual is developed, which consists of a tribo-nanogenerator (TENG) as a vestibule that can sense rotation and synaptic transistor array as retina. The principle of temporal congruency has been successfully realized by multisensory input. In addition, pattern recognition results show that the accuracy of multisensory integration is more than 15% higher than that of single sensory. Moreover, due to the rotation recognition and visual recognition functions of AMSS, we realized multimodal information recognition including angles and numbers in the spiking correlated neural network (SCNN), and the accuracy rate reached 89.82%. Besides, the rapid self-protection of a human was successfully realized by AMSS in the case of simulated amusement rides, and the reaction time of multiple motion sensory integration is only one-third of that of a single vestibule. The development of AMSS based on the synergy of simulated vision and vestibule will show great potential in neural robot, artificial limbs, and soft electronics.
Keywords: TENG; artificial vestibule; multimodal information recognition; multiple motion sensory integration; self-protection; synaptic transistor.