Eye-Gaze Controlled Wheelchair Based on Deep Learning

Jun Xu; Zuning Huang; Liangyuan Liu; Xinghua Li; Kai Wei

doi:10.3390/s23136239

Eye-Gaze Controlled Wheelchair Based on Deep Learning

Sensors (Basel). 2023 Jul 7;23(13):6239. doi: 10.3390/s23136239.

Authors

Jun Xu¹, Zuning Huang², Liangyuan Liu², Xinghua Li², Kai Wei²

Affiliations

¹ School of Automation, Harbin University of Science and Technology, Harbin 150080, China.
² School of Electrical and Electronic Engineering, Harbin University of Science and Technology, Harbin 150080, China.

Abstract

In this paper, we design a technologically intelligent wheelchair with eye-movement control for patients with ALS in a natural environment. The system consists of an electric wheelchair, a vision system, a two-dimensional robotic arm, and a main control system. The smart wheelchair obtains the eye image of the controller through a monocular camera and uses deep learning and an attention mechanism to calculate the eye-movement direction. In addition, starting from the relationship between the trajectory of the joystick and the wheelchair speed, we establish a motion acceleration model of the smart wheelchair, which reduces the sudden acceleration of the smart wheelchair during rapid motion and improves the smoothness of the motion of the smart wheelchair. The lightweight eye-movement recognition model is transplanted into an embedded AI controller. The test results show that the accuracy of eye-movement direction recognition is 98.49%, the wheelchair movement speed is up to 1 m/s, and the movement trajectory is smooth, without sudden changes.

Keywords: CBAM attention; deep learning; eye-tracking; wheelchair acceleration model.

MeSH terms

Deep Learning*
Fixation, Ocular
Humans
Motion
Movement
Wheelchairs*

Grants and funding

Heilongjiang Provincial Education Department Scientific Research Fund Project “Intelligent Eye-Movement Wheelchair Based on Deep Learning”, Project No. S202210214067.