GestureMoRo: an algorithm for autonomous mobile robot teleoperation based on gesture recognition

Lei Chen; Chunxu Li; Ashraf Fahmy; Johann Sienz

doi:10.1038/s41598-024-54488-w

GestureMoRo: an algorithm for autonomous mobile robot teleoperation based on gesture recognition

Sci Rep. 2024 Mar 14;14(1):6199. doi: 10.1038/s41598-024-54488-w.

Authors

Lei Chen¹, Chunxu Li^{2

3}, Ashraf Fahmy⁴, Johann Sienz⁴

Affiliations

¹ College of Mechanical and Electrical Engineering, Hohai University, Changzhou, 213200, China.
² Department of Machenical Engineering, Faculty of Science and Engineering, Swansea University, Swansea, SA1 8EN, UK. chunxu.li@swansea.ac.uk.
³ College of Mechanical and Electrical Engineering, Hohai University, Changzhou, 213200, China. chunxu.li@swansea.ac.uk.
⁴ Department of Machenical Engineering, Faculty of Science and Engineering, Swansea University, Swansea, SA1 8EN, UK.

Abstract

Gestures are a common way people communicate. Gesture-based teleoperation control systems tend to be simple to operate and suitable for most people's daily use. This paper employed a LeapMotion sensor to develop a mobile robot control system based on gesture recognition, which mainly established connections through a client/server structure. The principles of gesture recognition in the system were studied and the relevant self-investigated algorithms-GestureMoRo, for the association between gestures and mobile robots were designed. Moreover, in order to avoid the unstably fluctuated movement of the mobile robot caused by palm shaking, the Gaussian filter algorithm was used to smooth and denoise the collected gesture data, which effectively improved the robustness and stability of the mobile robot's locomotion. Finally, the teleoperation control strategy of the gesture to the WATER2 mobile robot was realized, and the effectiveness and practicability of the designed system were verified through multiple experiments.

Keywords: Gaussian filter; Gesture recognition; GestureMoRo; LeapMotion; Mobile robot; Teleoperation.