Source-linked AI summary
Real-time Multi-Class Helmet Violation Detection Using Few-Shot Data Sampling Technique and YOLOv8
Armstrong Aboah, Bin Wang, Ulas Bagci, Yaw Adu-Gyamfi
TL;DR
Helmet-usage violations remain difficult to detect in real time, despite their importance for traffic safety and enforcement. The study combines few-shot data sampling with YOLOv8 to develop a robust detector using fewer annotations, achieving an overall mAP score of 0.5861 at 95 fps on the experimental test dataset and ranking 7th on the public leaderboard.
Problem
Helmet-usage violations remain a significant problem, while most existing helmet-detection systems are not real-time despite the need for real-time traffic surveillance and enforcement.
Method
The system combines few-shot data sampling and data augmentation with the single-stage YOLOv8 detector to develop a robust real-time model using fewer annotations.
Results
0.5861 overall mAP at 95 fps was achieved by YOLOv8+TTA on the experimental test dataset, which ranked 7th on the public leaderboard.
Takeaways & Limitations
The proposed system is reported as effective, robust, and well-suited for practical real-time helmet-violation detection.
Abstract
from arXiv · showhide
Traffic safety is a major global concern. Helmet usage is a key factor in preventing head injuries and fatalities caused by motorcycle accidents. However, helmet usage violations continue to be a significant problem. To identify such violations, automatic helmet detection systems have been proposed and implemented using computer vision techniques. Real-time implementation of such systems is crucial for traffic surveillance and enforcement, however, most of these systems are not real-time. This study proposes a robust real-time helmet violation detection system. The proposed system utilizes a unique data processing strategy, referred to as few-shot data sampling, to develop a robust model with fewer annotations, and a single-stage object detection model, YOLOv8 (You Only Look Once Version 8), for detecting helmet violations in real-time from video frames. Our proposed method won 7th place in the 2023 AI City Challenge, Track 5, with an mAP score of 0.5861 on experimental validation data. The experimental results demonstrate the effectiveness, efficiency, and robustness of the proposed system.
1. Introduction
The study targets real-time helmet-violation detection that remains reliable across weather and lighting conditions. It combines few-shot data sampling, augmentation, and YOLOv8 to build a robust system with fewer annotations.
- Helmet violations remain a significant problem, while many existing automatic detection systems cannot operate in real time.
- Varying weather, lighting, and time of day make helmet visibility and violation detection challenging.
- YOLOv8 serves as the single-stage object detector for real-time helmet-violation detection.
- The proposed system uses few-shot data sampling to select representative images and augmentation to develop a robust model with fewer annotations.
- The study compares YOLOv5, YOLOv7, and YOLOv8 to identify an effective model for helmet-violation detection.
- The experiments report an effective, robust, fast, and efficient system for detecting helmets in real-world scenarios.
2. Related works
Prior helmet-enforcement research has used computer vision and deep learning to improve accuracy and speed. The literature nevertheless identifies a need for systems that are more accurate, faster, and robust across conditions.
- Helmet violation detection is motivated by helmet usage’s role in reducing injuries and fatalities in road accidents.
- Earlier approaches used color and texture features or deep neural networks for real-time helmet enforcement.
- Reported accuracy rates in prior studies ranged from 89.5% to 97.5%, with processing times below 30, 50, or 100 milliseconds per frame.
- The reviewed studies show that deep learning approaches can achieve high accuracy and processing speed for real-time helmet enforcement.
- Further research is needed to improve accuracy and processing times while increasing robustness across conditions and environments.
3. Data
The study uses videos spanning varied camera, lighting, and weather conditions, then applies few-shot sampling and augmentation to build a representative training set with fewer annotation demands.
- Data Overview: The dataset contains 100 training and 100 testing videos, averaging 20 seconds at 10 fps and 1920x1080 resolution.Videos include varied camera angles, lighting, and weather conditions; the task is identifying motorcycles, riders, and helmet usage.
- Data Processing: The preprocessing framework combines representative frame selection with augmentation to improve variability and real-world generalization.The two main steps are few-shot data sampling and data augmentation.
- Few-shot Data Sampling: Few-shot sampling addresses missing annotations by selecting representative frames instead of manually reviewing all 20,000 frames.The framework was developed because the challenge annotations contained omissions that could affect training.
- Few-shot Data Sampling: Videos are categorized as day, night, or foggy using background-based pixel-frequency analysis, supporting balanced sampling across conditions.The categorization algorithm uses a maximum pixel-frequency threshold and skewness to distinguish conditions.
- Few-shot Data Sampling: Frame sampling calculates category-specific rates to select more frames from underrepresented video types.Rates use the number of videos in each category and their frame rates.
- Data Augmentation: Augmentation varies viewpoint, scale, occlusion, lighting, and appearance through flipping, rotation, scaling, cropping, blurring, and color manipulation.These transformations target viewpoint variation, partial obstruction, poor lighting, and differing image conditions.
- Training and Validation Dataset: Training used 4,500 examples with a 0.7:0.3 training-validation split, while SCAN removed highly similar frames and prevented sequential overlap across splits.The validation set was selected to reduce potential evaluation bias.
4. Helmet Detection Models
The study compares YOLOv5, YOLOv7, and YOLOv8 as single-stage detectors, emphasizing YOLOv8 for real-time object detection and using test-time augmentation to improve predictions.
- Model Comparison: The experiment compares YOLOv5, YOLOv7, and YOLOv8, all of which perform single-stage object detection.The comparison evaluates these three YOLO-family models for helmet-violation detection.
- YOLOv5: YOLOv5 uses a CSPDarknet53 backbone, a feature-combining neck, and a prediction head for object classes and bounding boxes.Its architecture also includes SPP and PANet components.
- YOLOv7: YOLOv7 introduces compound scaling, EELAN, and specialized convolution and loss-design changes relative to YOLOv5.EELAN is described as supporting effective learning while maintaining the original gradient route.
- YOLOv8: YOLOv8 uses an updated backbone, neck, and detection head with anchor-free bounding-box prediction for real-time object detection.The model also supports instance segmentation and feature-pyramid processing for objects of varying sizes.
- Model Training: All three models were trained for 400 epochs with batch size 16 and 832x832 images using genetically optimized hyperparameters.Training was conducted with hyperparameters summarized in Table 1.
- Test Time Augmentation: Test-time augmentation averages predictions across augmented test inputs, potentially improving accuracy at additional computational cost.The technique can use rotation, flipping, or cropping and may be implemented with parallel processing or batching.
5. Results and Discussion
The evaluation compares YOLO detectors with and without test-time augmentation on validation and test data, using mAP for leaderboard assessment. YOLOv8 with TTA achieved the strongest reported test performance and ranked seventh.
- Test dataset: The 2023 NVIDIA AI City Challenge Task 5 test set contains 100 unannotated 20-second videos at 1920 × 1080 resolution.Models detect motorcycles and riders with or without helmets from the test videos.
- Submission format: Submissions encode video and frame identifiers, bounding-box coordinates, class identifiers, and confidence values.Confidence is represented as a probability between 0 and 1.
- Evaluation: The evaluation uses mean Average Precision across test-video frames, and leaderboard ranking is determined by the mAP score.Equation 1 defines the reported mAP calculation, with N described as the number of queries.
- Validation results: YOLOv7 and YOLOv8 outperform YOLOv5 on mAP.05 and mAP.05- .95, while TTA further improves model performance.YOLOv8+TTA achieves the highest validation mAP.05- .95 score of 0.647.
- Test results: 0.5861 overall mAP and 95 fps were achieved by YOLOv8+TTA on the experimental test dataset, where it outperformed all other models.The result was ranked 7th on the public leaderboard, and the inference speed supports real-time predictions.
6. Conclusion
The study developed a real-time multi-class helmet violation detection system designed to remain robust across varying weather conditions and times of day. It combines few-shot data sampling with YOLOv8 and reports effective, robust, and efficient real-world helmet detection.
- Conclusion: The proposed system combines few-shot data sampling with YOLOv8 for real-time multi-class helmet violation detection.The system targets robustness across varying weather conditions and times of day.
- Conclusion: The experimental results demonstrate effective, robust, and efficient helmet detection in real-world scenarios.The authors report that the system is well-suited for practical real-time applications.
- Conclusion: The study identifies the proposed system as robust to varying weather conditions and times of day.This robustness is stated as part of the system's practical suitability.