Velocity control in car-following behavior with autonomous vehicles using reinforcement learning

Abstract

Car-following behavior is a common driving behavior. It is necessary to consider the following vehicle in the car-following model of autonomous vehicle (AV) under the background of the vehicle-to-vehicle transportation system. In this study, a safe velocity control method for AV based on reinforcement learning with considering the following vehicle is proposed. First, the mixed driving environment of AVs and human-driven vehicles is constructed, and the trajectories of the leading and following vehicles are extracted from the naturalistic High D driving dataset. Next, the soft actor-critic (SAC) algorithm is used as the velocity control algorithm, in which the agent is AV, the action is acceleration, and the state is the relative distance and relative speed between the AV and the leading and following vehicles. Then, a reward function based on state and corresponding action is designed to guide AV to choose acceleration without collision between the leading and following vehicles. Furthermore, AVs are gradually able to learn to avoid collisions between the leading and following vehicles after training the model. The test result of the trained model shows that the SAC agent can achieve complete collision avoidance, resulting in zero collision. Finally, the driving performance of the SAC agent and that of human driving are compared and analyzed for safety and efficiency. The results of this study are expected to improve the safety of the car-following process..

Keywords: Autonomous driving; Car-following considering the following vehicle; High D; Reinforcement learning; Soft actor-critic.