A human-centered safe robot reinforcement learning framework with interactive behaviors

Shangding Gu; Alap Kshirsagar; Yali Du; Guang Chen; Jan Peters; Alois Knoll

doi:10.3389/fnbot.2023.1280341

A human-centered safe robot reinforcement learning framework with interactive behaviors

Front Neurorobot. 2023 Nov 9:17:1280341. doi: 10.3389/fnbot.2023.1280341. eCollection 2023.

Authors

Shangding Gu^#¹, Alap Kshirsagar^#², Yali Du^#³, Guang Chen⁴, Jan Peters², Alois Knoll¹

Affiliations

¹ Department of Computer Science, Technical University of Munich, Munich, Germany.
² Department of Computer Science, Technical University of Darmstadt, Darmstadt, Germany.
³ Department of Informatics, King's College London, London, United Kingdom.
⁴ College of Electronic and Information Engineering, Tongji University, Shanghai, China.

^# Contributed equally.

Abstract

Deployment of Reinforcement Learning (RL) algorithms for robotics applications in the real world requires ensuring the safety of the robot and its environment. Safe Robot RL (SRRL) is a crucial step toward achieving human-robot coexistence. In this paper, we envision a human-centered SRRL framework consisting of three stages: safe exploration, safety value alignment, and safe collaboration. We examine the research gaps in these areas and propose to leverage interactive behaviors for SRRL. Interactive behaviors enable bi-directional information transfer between humans and robots, such as conversational robot ChatGPT. We argue that interactive behaviors need further attention from the SRRL community. We discuss four open challenges related to the robustness, efficiency, transparency, and adaptability of SRRL with interactive behaviors.

Keywords: bi-direction information; interactive behaviors; safe collaboration; safe exploration; value alignment.

Grants and funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was partially supported by the European Union's Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreement No. 945539 (Human Brain Project SGA3); this work was also supported by The Adaptive Mind, funded by the Excellence Program of the Hessian Ministry of Higher Education, Science, Research and Art.