Dual-flow network with attention for autonomous driving

Lei Yang; Weimin Lei; Wei Zhang; Tianbing Ye

doi:10.3389/fnbot.2022.978225

Dual-flow network with attention for autonomous driving

Front Neurorobot. 2023 Jan 9:16:978225. doi: 10.3389/fnbot.2022.978225. eCollection 2022.

Authors

Lei Yang^{1

2}, Weimin Lei^{1

3}, Wei Zhang¹, Tianbing Ye²

Affiliations

¹ School of Computer Science and Engineering, Northeastern University, Shenyang, China.
² DAMO Academy, Alibaba Group, Hangzhou, China.
³ Engineering Research Center of Security Technology of Complex Network System Ministry of Education, Shenyang, China.

Abstract

We present a dual-flow network for autonomous driving using an attention mechanism. The model works as follows: (i) The perception network extracts red, blue, and green (RGB) images from the video at low speed as input and performs feature extraction of the images; (ii) The motion network obtains grayscale images from the video at high speed as the input and completes the extraction of object motion features; (iii) The perception and motion networks are fused using an attention mechanism at each feature layer to perform the waypoint prediction. The model was trained and tested using the CARLA simulator and enabled autonomous driving in complex urban environments, achieving a success rate of 74%, especially in the case of multiple dynamic objects.

Keywords: CARLA simulator; artificial intelligence; attention; autonomous driving; deep neural network; network architecture; visual navigation.

Grants and funding

This work was supported by the Fundamental Research Funds for the Central Universities of China (No. N2216010) and the National Key Research and Development Program of China (No. 2018YFB1702000).