PaperlayerView exact revision on arXiv

Source-linked AI summary

Driver Drowsiness Detection Model Using Convolutional Neural Networks Techniques for Android Application

Rateb Jabbar, Mohammed Shinoy, Mohamed Kharbeche, Khalifa Al-Khalifa, Moez Krichen, Kamel Barkaoui

arXiv:2002.03728v1cs.CV

TL;DR

Drowsy-driving microsleeps can leave drivers unable to respond to hazards. This paper detects drowsiness by passing camera-detected facial landmarks to a CNN. It reports 83.3% overall accuracy and a 75 KB maximum model size, while noting limitations from sunglasses and poor lighting.

  • Problem

    Microsleeps leave sleepy drivers unable to perceive visual information or react to lane departures or harsh braking.

    Source: §I · p. 1

  • Method

    The model uses facial landmark coordinates extracted from images as input to a CNN that classifies drowsiness.

    Source: B. Drowsiness Detection based on CNN… · pp. 3–4

  • Results

    83.3% overall accuracy was reported for D2CNN-FLD, a 2.5% improvement over the prior MLP method.

    Source: §IV · p. 4

  • Takeaways & Limitations

    The proposed model's maximum size is 75 KB, and the authors identify it as suitable for integration into dashboards or mobile devices for driver-sleepiness intervention.

    Source: §IV · p. 5, §V · p. 6

    Sunglasses that obstruct facial features and bad lighting conditions limit the technology.

    Source: §V · p. 6

Abstract

from arXiv · show

A sleepy driver is arguably much more dangerous on the road than the one who is speeding as he is a victim of microsleeps. Automotive researchers and manufacturers are trying to curb this problem with several technological solutions that will avert such a crisis. This article focuses on the detection of such micro sleep and drowsiness using neural network based methodologies. Our previous work in this field involved using machine learning with multi-layer perceptron to detect the same. In this paper, accuracy was increased by utilizing facial landmarks which are detected by the camera and that is passed to a Convolutional Neural Network (CNN) to classify drowsiness. The achievement with this work is the capability to provide a lightweight alternative to heavier classification models with more than 88% for the category without glasses, more than 85% for the category night without glasses. On average, more than 83% of accuracy was achieved in all categories. Moreover, as for model size, complexity and storage, there is a marked reduction in the new proposed model in comparison to the benchmark model where the maximum size is 75 KB. The proposed CNN based model can be used to build a real-time driver drowsiness detection system for embedded systems and Android devices with high accuracy and ease of use.

I. INTRODUCTION

Drowsy driving is dangerous because microsleeps leave drivers unable to perceive visual information or react. The paper proposes a deep-learning detection algorithm that may support drowsiness interventions on mobile and embedded devices.

I. INTRODUCTION

  • Existing driver-assistance technologies include cruise control, automatic braking, lane warnings, steering assistance, and attention tracking, but many systems are proprietary and limited to high-end vehicles.The introduction notes that Samsung partnered with Eyesight to track attention using facial features and patterns.Source: §I · p. 1
  • Android Auto and Apple Car are increasingly built into vehicles, including lower-end cars, offering platforms for camera-based driver-behavior detection.The passage also identifies paired embedded devices or mobile phones as possible platforms.Source: §I · p. 1
  • Microsleeps leave drivers unable to perceive visual information, making them unable to react to lane departures or harsh braking.The introduction cites NHTSA statistics attributing about 2.5% of crash fatalities to drowsy driving and reports over 72,000 related crashes in 2015.Source: §I · p. 1
  • Many current machine-vision models are bulky, require dedicated hardware, and run inefficiently on low-computational-power devices.The paper describes its deep-learning algorithm as potentially enabling an intervention and as simple to configure on mobile or embedded devices.Source: §I · p. 2
  • The paper presents a CNN and facial-landmark-based drowsiness algorithm, with later sections covering Android implementation, accuracy, efficiency, and comparisons.The introduction identifies the methodology as D2CNN-FLD.Source: §I · p. 2

II. LITERATURE REVIEW

Prior drowsiness-detection work uses vehicle behavior, physiological signals, computer vision, or combinations of these inputs. The paper motivates a CNN-based alternative by noting the storage and computational costs of larger models and comparing its prior MLP model as a baseline.

II. LITERATURE REVIEW

  • Vehicle-behavior methods use steering, speed, and accelerator data in a Bayesian Network; one study reported lower false positives than PERCLOS and emphasized the importance of recent driving context.The cited method uses data from a previous 10-second period to assess drowsiness-related lane-departure risk.Source: §II · p. 2
  • Physiological approaches include EEG, IMU, and heart-rate sensing, with reported trade-offs involving comfort, drowsiness-stage coverage, and power use.One EEG-and-IMU study detected five drowsiness levels at about 95% accuracy, while a PULP implementation extended battery life to almost 46 hours.Source: §II · p. 2
  • Vision-based approaches analyze facial cues; reported examples include 95% accuracy for eyelid-curvature detection, 0.84 mean average precision for MobileNet-SSD, and 84% accuracy for an Android facial-landmark application.The eyelid-curvature method was limited by poor performance at night, while other cited work used multiple facial features or sequential video learning.Source: §II · p. 2
  • Combining blink, eye-tracking, and vital-sign inputs can support drowsiness intervention, but multiple sensors and proprietary solutions can be expensive.The cited smart-glass system beeps to ask drivers to rest.Source: §II · p. 2
  • Larger deep-learning models bring greater computational requirements; the authors' earlier D2MLP-FLD model achieved 81% accuracy at 100KB, motivating a lighter model with improved performance and storage use.The literature review labels Fig. 1 as NTHU dataset sample images and Fig. 2 as a video preprocessing outline; it does not describe their visual contents.Source: §II · pp. 2–3, Figure 1 · p. 3, Figure 2 · p. 3

A. Dataset and Preprocessing

The study uses videos from the NTHU Driver Drowsiness Detection Dataset and preprocesses them into images.

A. Dataset and Preprocessing

  • The deep-learning model is trained on videos from the National Tsing Hua University Driver Drowsiness Detection Dataset.The videos are preprocessed to create images for the study.Source: A. Dataset and Preprocessing · p. 3
  • Video preprocessing creates images from the dataset videos for use in the study.The passage identifies images as the resulting study material but does not specify the preprocessing steps.Source: A. Dataset and Preprocessing · p. 3
  • The dataset videos provide the training source for the study's deep-learning model.The passage does not report dataset size or composition here.Source: A. Dataset and Preprocessing · p. 3

1) NTHU Dataset:

The NTHU dataset comprises simulated driving videos featuring 22 subjects, recorded in day and night conditions.

1) NTHU Dataset:

  • The dataset includes 22 subjects from different ethnicities recorded under day and night conditions.The recordings use infrared cameras to obtain night-time videos.Source: 1) NTHU Dataset · p. 3
  • The simulated scenarios include conversations, yawning, slow blinking, sleepy head movements, and drowsy eyes.The passage refers to Fig. 1 as showing examples of these scenarios.Source: 1) NTHU Dataset · p. 3
  • The dataset contains 9.5 hours of 640*480 video recorded at 30 frames per second.Subjects enact regular and drowsy driving behaviors.Source: 1) NTHU Dataset · p. 3

2) Preprocessing the dataset:

The dataset is prepared by extracting video frames, augmenting the resulting images, and locating facial landmarks. These steps create image data and facial coordinates for the detection model.

2) Preprocessing the dataset:

  • About 600,000 images are produced by converting video frames recorded at 30 frames per second.The paper notes that images, rather than video data, are fed into the neural network.Source: 2) Preprocessing the dataset · p. 3
  • Codebox generates additional training images by applying augmentation operations to images extracted from video frames.Figure 3 illustrates the augmentation operations.Source: 2) Preprocessing the dataset · p. 3, Figure 3 · p. 3
  • Dlib identifies 68 facial landmarks as x,y coordinates to map salient facial features, including the eyes, mouth, nose, and jawline.The passage says OpenCV Haar Cascades are used to detect faces before landmark extraction.Source: 2) Preprocessing the dataset · p. 3, Figure 4 · p. 3

B. Drowsiness Detection based on CNN and Facial Landmark Detection (D2CNN-FLD)

D2CNN-FLD takes Dlib facial-landmark coordinates and labels as input and produces two-class probabilities for drowsy driving. Its CNN uses normalization, convolutional layers, pooling, dropout, and a softmax output.

B. Drowsiness Detection based on CNN and Facial Landmark Detection (D2CNN-FLD)

  • The model has five layers, with a first convolutional layer of 100 neurons receiving 134 input nodes and three subsequent convolutional layers of 1,024 neurons each.The algorithm describes the final layer as fully connected with a softmax function.Source: B. Drowsiness Detection based on CNN… · pp. 3–4
  • The algorithm takes Dlib facial-landmark positions and labels as input and outputs two-class probabilities for drowsy driving.The dataset is min-max scaled to the range 0 to 1.Source: B. Drowsiness Detection based on CNN… · pp. 3–4
  • Leaky ReLU permits a small gradient for inactive units, while max pooling reduces parameters and dropout is used to prevent overfitting.The text gives a 25.4% dropout rate after testing rates from 20% to 30%; the algorithm also specifies 20% dropout in its repeated layers.Source: B. Drowsiness Detection based on CNN… · p. 4

C. Android Implementation and Architecture

The Android application captures driver images and passes them between Java and the C++ Dlib library through JNI. Dlib extracts facial landmarks for the CNN, and the application displays results in real time and sends visual and audio notifications when drowsiness is detected.

C. Android Implementation and Architecture

IV. EXPERIMENTAL RESULTS

D2CNN-FLD achieved 83.3% overall accuracy, a 2.5% improvement over the prior MLP method. Experiments also compared accuracy, model size, detection time, and speed across simulated driving scenarios and benchmarks.

IV. EXPERIMENTAL RESULTS

  • The dataset used 200 videos across five simulated driving scenarios, with 18 training subjects and four evaluation subjects.The videos were converted to images, and facial-landmark coordinates were extracted from generated images.Source: §IV · p. 4, Table I · p. 5
  • 83.3% overall accuracy was achieved by D2CNN-FLD, a 2.5% improvement over the prior MLP method.The experiments used a Dell workstation with an NVIDIA QUADRO P4000.Source: §IV · p. 4
  • D2CNN-FLD outperformed D2MLP-FLD in validation and test accuracy, with a 0.075 MB model size, 25% smaller than D2MLP-FLD at about 0.1 MB.Table II summarizes accuracy, compression, drowsiness detection time, and overall speed.Source: Table II · p. 5, §IV · p. 5
  • On the NVIDIA Quadro P4000, D2CNN-FLD detection time was 38.45 ms versus 41.6 ms for D2MLP-FLD; on Samsung Galaxy S8 plus, it was 142 ms versus 154 ms.Overall speed was 6.87 fps versus 7.4 fps on the NVIDIA Quadro P4000, and 234.25 fps versus 267.56 fps on the Samsung Galaxy S8 plus.Source: §IV · p. 5
  • D2CNN-FLD's maximum model size was 75 KB, compared with 547 MB for VGG16-based RCNN and 236 MB for AlexNet-based Faster-RCNN.Compressed Faster-RCNN models still occupy approximately 12 MB.Source: §IV · p. 5
  • D2CNN-FLD and D2MLP-FLD require 44.1 MB and 38.9 MB of RAM, respectively, versus more than 3180 MB for Faster-RCNN, supporting possible integration into smaller embedded systems.The paper also reports that D2CNN-FLD averaged 83.33% accuracy while maintaining a very small model size.Source: §IV · pp. 4–5

V. CONCLUSION

The system detects facial landmarks from mobile-device images and passes them to a CNN to classify drowsy driving, aiming for lightweight embedded implementation. It reports 83.33% average accuracy across categories with a model no larger than 75 KB, while noting sunglasses and poor lighting as limitations.

V. CONCLUSION

  • The system detects facial landmarks from mobile-device images and passes them to a trained CNN to classify drowsy driving behavior.The stated objective is a lightweight system for embedded implementation while maintaining high performance.Source: §V · p. 5
  • 83.33% average accuracy across all categories was achieved with a model whose maximum size did not exceed 75 KB.Source: §V · p. 6
  • The authors identify sunglasses-obstructed facial features and bad lighting as limitations, with room to improve feature detection in poor lighting.Source: §V · p. 6

Related papers

Loading 2002.03728v1…