Unusual Driver Behavior Detection in Videos Using Deep Learning Models

Hamad Ali Abosaq; Muhammad Ramzan; Faisal Althobiani; Adnan Abid; Khalid Mahmood Aamir; Hesham Abdushkour; Muhammad Irfan; Mohammad E Gommosani; Saleh Mohammed Ghonaim; V R Shamji; Saifur Rahman

doi:10.3390/s23010311

Unusual Driver Behavior Detection in Videos Using Deep Learning Models

Sensors (Basel). 2022 Dec 28;23(1):311. doi: 10.3390/s23010311.

Authors

Affiliations

¹ Computer Science Department, College of Computer Science and Information Systems, Najran University, Najran 61441, Saudi Arabia.
² Department of Computer Science and Information Technology, University of Sargodha, Sargodha 40100, Pakistan.
³ Department of Computer Science, University of Management & Technology, Lahore 54770, Pakistan.
⁴ Nautical Science Department, Faculty of Maritime Studies, King Abdulaziz University, Jeddah 22254, Saudi Arabia.
⁵ Faculty of Computer Science and Information Technology, Virtual University of Pakistan, Lahore 54000, Pakistan.
⁶ Electrical Engineering Department, College of Engineering, Najran University, Najran 61441, Saudi Arabia.
⁷ Department of Hydrographic Surveying, Faculty of Maritime Studies, King Abdulaziz University, P.O. Box 80401, Jeddah 21589, Saudi Arabia.

Abstract

Anomalous driving behavior detection is becoming more popular since it is vital in ensuring the safety of drivers and passengers in vehicles. Road accidents happen for various reasons, including health, mental stress, and fatigue. It is critical to monitor abnormal driving behaviors in real time to improve driving safety, raise driver awareness of their driving patterns, and minimize future road accidents. Many symptoms appear to show this condition in the driver, such as facial expressions or abnormal actions. The abnormal activity was among the most common causes of road accidents, accounting for nearly 20% of all accidents, according to international data on accident causes. To avoid serious consequences, abnormal driving behaviors must be identified and avoided. As it is difficult to monitor anyone continuously, automated detection of this condition is more effective and quicker. To increase drivers' recognition of their driving behaviors and prevent potential accidents, a precise monitoring approach that detects abnormal driving behaviors and identifies abnormal driving behaviors is required. The most common activities performed by the driver while driving is drinking, eating, smoking, and calling. These types of driver activities are considered in this work, along with normal driving. This study proposed deep learning-based detection models for recognizing abnormal driver actions. This system is trained and tested using a newly created dataset, including five classes. The main classes include Driver-smoking, Driver-eating, Driver-drinking, Driver-calling, and Driver-normal. For the analysis of results, pre-trained and fine-tuned CNN models are considered. The proposed CNN-based model and pre-trained models ResNet101, VGG-16, VGG-19, and Inception-v3 are used. The results are compared by using the performance measures. The results are obtained 89%, 93%, 93%, 94% for pre-trained models and 95% by using the proposed CNN-based model. Our analysis and results revealed that our proposed CNN base model performed well and could effectively classify the driver's abnormal behavior.

Keywords: abnormal behaviors; deep learning; driver; drowsiness; human activity; surveillance.

MeSH terms

Accidents, Traffic / prevention & control
Automobile Driving*
Deep Learning*
Problem Behavior*
Safety

Grants and funding

(NU/RG/SERC/11/3)/The authors acknowledge the support from the Deanship of Scientific Research, Najran Univer-sity. Kingdom of Saudi Arabia, for funding this work under the Research Groups funding pro-gram grant code number (NU/RG/SERC/11/3).