Whole-Body Teleoperation Control of Dual-Arm Robot Using Sensor Fusion

Biomimetics (Basel). 2023 Dec 5;8(8):591. doi: 10.3390/biomimetics8080591.

Abstract

As human-robot interaction and teleoperation technologies advance, anthropomorphic control of humanoid arms has garnered increasing attention. However, accurately translating sensor-detected arm motions to the multi-degree freedom of a humanoid robotic arm is challenging, primarily due to occlusion issues with single-sensor setups, which reduce recognition accuracy. To overcome this problem, we propose a human-like arm control strategy based on multi-sensor fusion. We defined the finger bending angle to represent finger posture and employed a depth camera to capture arm movement. Consequently, we developed an arm movement tracking system and achieved anthropomorphic control of the imitation human arm. Finally, we verified our proposed method's effectiveness through a series of experiments, evaluating the system's robustness and real-time performance. The experimental results show that this control strategy can control the motion of the humanoid arm stably, and maintain a high recognition accuracy in the face of complex situations such as occlusion.

Keywords: humanoid arm; sensor fusion; teleoperation.

Grants and funding

This research was partially funded by the Project of Shandong High-end Medical Device Innovation and Entrepreneurship Community, grant number 2022-SGTTXM-018; the Open Project Program of Key Laboratory for Cross-Scale Micro and Nano Manufacturing, Ministry of Education, Changchun University of Science and Technology, grant number CMNM-KF202103.