An Intuitive End-to-End Human-UAV Interaction System for Field Exploration

Ran Jiao; Zhaowei Wang; Ruihang Chu; Mingjie Dong; Yongfeng Rong; Wusheng Chou

doi:10.3389/fnbot.2019.00117

An Intuitive End-to-End Human-UAV Interaction System for Field Exploration

Front Neurorobot. 2020 Feb 14:13:117. doi: 10.3389/fnbot.2019.00117. eCollection 2019.

Authors

Ran Jiao¹, Zhaowei Wang¹, Ruihang Chu¹, Mingjie Dong², Yongfeng Rong¹, Wusheng Chou^{1

3}

Affiliations

¹ School of Mechanical Engineering and Automation, Beihang University, Beijing, China.
² College of Mechanical Engineering and Applied Electronics Technology, Beijing University of Technology, Beijing, China.
³ The State Key Laboratory of Virtual Reality, Technology and Systems, Beihang University, Beijing, China.

Abstract

This paper presents an intuitive end-to-end interaction system between a human and a hexacopter Unmanned Aerial Vehicle (UAV) for field exploration in which the UAV can be commanded by natural human poses. Moreover, LEDs installed on the UAV are used to communicate the state and intents of the UAV to the human as feedback throughout the interaction. A real time multi-human pose estimation system is built that can perform with low latency while maintaining competitive performance. The UAV is equipped with a robotic arm, kinematic and dynamic attitude models for which are provided by introducing the center of gravity (COG) of the vehicle. In addition, a super-twisting extended state observer (STESO)-based back-stepping controller (BSC) is constructed to estimate and attenuate complex disturbances in the attitude control system of the UAV, such as wind gusts, model uncertainties, etc. A stability analysis for the entire control system is also presented based on the Lyapunov stability theory. The pose estimation system is integrated with the proposed intelligent control architecture to command the UAV to execute an exploration task stably. Additionally, all the components of this interaction system are described. Several simulations and experiments have been conducted to demonstrate the effectiveness of the whole system and its individual components.

Keywords: UAV; back-stepping; extended state observer; intuitive interaction; pose estimation; super-twisting.