Neural Networks Trained via Reinforcement Learning Stabilize Walking of a Three-Dimensional Biped Model With Exoskeleton Applications

Chujun Liu; Musa L Audu; Ronald J Triolo; Roger D Quinn

doi:10.3389/frobt.2021.710999

Neural Networks Trained via Reinforcement Learning Stabilize Walking of a Three-Dimensional Biped Model With Exoskeleton Applications

Front Robot AI. 2021 Aug 6:8:710999. doi: 10.3389/frobt.2021.710999. eCollection 2021.

Authors

Chujun Liu¹, Musa L Audu^{2

3}, Ronald J Triolo^{2

3}, Roger D Quinn^{1

2}

Affiliations

¹ Department of Mechanical Engineering, Case Western Reserve University, Cleveland, OH, United States.
² Advanced Platform Technology Center, Louis Stokes VA Medical Center, Cleveland, OH, United States.
³ Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States.

Abstract

Our group is developing a cyber-physical walking system (CPWS) for people paralyzed by spinal cord injuries (SCI). The current CPWS consists of a functional neuromuscular stimulation (FNS) system and a powered lower-limb exoskeleton for walking with leg movements in the sagittal plane. We are developing neural control systems that learn to assist the user of this CPWS to walk with stability. In a previous publication (Liu et al., Biomimetics, 2019, 4, 28), we showed a neural controller that stabilized a simulated biped in the sagittal plane. We are considering adding degrees of freedom to the CPWS to allow more natural walking movements and improved stability. Thus, in this paper, we present a new neural network enhanced control system that stabilizes a three-dimensional simulated biped model of a human wearing an exoskeleton. Results show that it stabilizes human/exoskeleton models and is robust to impact disturbances. The simulated biped walks at a steady pace in a range of typical human ambulatory speeds from 0.7 to 1.3 m/s, follows waypoints at a precision of 0.3 m, remains stable, and continues walking forward despite impact disturbances and adapts its speed to compensate for persistent external disturbances. Furthermore, the neural network controller stabilizes human models of different statures from 1.4 to 2.2 m tall without any changes to the control parameters. Please see videos at the following link: 3D biped walking control.

Keywords: biped stability; exoskeleton; gait; neural network; reinforcement learning.