Dynamic Gesture Recognition Model Based on Millimeter-Wave Radar With ResNet-18 and LSTM

Yongqiang Zhang; Lixin Peng; Guilei Ma; Menghua Man; Shanghe Liu

doi:10.3389/fnbot.2022.903197

Dynamic Gesture Recognition Model Based on Millimeter-Wave Radar With ResNet-18 and LSTM

Front Neurorobot. 2022 Jun 7:16:903197. doi: 10.3389/fnbot.2022.903197. eCollection 2022.

Authors

Yongqiang Zhang^{1

2}, Lixin Peng², Guilei Ma¹, Menghua Man¹, Shanghe Liu¹

Affiliations

¹ National Key Laboratory on Electromagnetic Environment Effects, Army Engineering University, Shijiazhuang, China.
² School of Information Science and Engineering, Hebei University of Science and Technology, Shijiazhuang, China.

Abstract

In this article, a multi-layer convolutional neural network (ResNet-18) and Long Short-Term Memory Networks (LSTM) model is proposed for dynamic gesture recognition. The Soli dataset is based on the dynamic gesture signals collected by millimeter-wave radar. As a gesture sensor radar, Soli radar has high positional accuracy and can recognize small movements, to achieve the ultimate goal of Human-Computer Interaction (HCI). A set of velocity-range Doppler images transformed from the original signal is used as the input of the model. Especially, ResNet-18 is used to extract deeper spatial features and solve the problem of gradient extinction or gradient explosion. LSTM is used to extract temporal features and solve the problem of long-time dependence. The model was implemented on the Soli dataset for the dynamic gesture recognition experiment, where the accuracy of gesture recognition obtained 92.55%. Finally, compare the model with the traditional methods. The result shows that the model proposed in this paper achieves higher accuracy in dynamic gesture recognition. The validity of the model is verified by experiments.

Keywords: Human-Computer Interaction; LSTM; ResNet-18; gesture recognition; millimeter-wave radar.