Source-linked AI summary
YOLO-Z: Improving small object detection in YOLOv5 for autonomous vehicles
Aduen Benjumea, Izzeddin Teeti, Fabio Cuzzolin, Andrew Bradley
TL;DR
Small-object detection is challenging because limited resolution and context constrain machine detectors, especially in real-time autonomous-vehicle settings. The paper experimentally modifies YOLOv5’s architecture and parameters, producing YOLO-Z models that improve small-object detection while retaining real-time-compatible inference. It also identifies a need to test the approach across more varied datasets and settings.
Problem
Small-object detection is challenging because limited resolution and context information are available to the model, while autonomous vehicles require real-time processing.
Method
The study experimentally modifies YOLOv5’s structural elements, connections, depth, width, and input-related settings for autonomous-racing small-object detection.
Results
Close to 6% improvement in standard mAP at 50% IoU was achieved by YOLO-Z models with an inference-time increase of around 3 ms.
Takeaways & Limitations
YOLO-Z models outperform YOLOv5 for the study’s smaller objects while retaining inference time compatible with real-time autonomous racing.
Takeaways & Limitations
The consistency and generality of the empirical gains require further investigation with different datasets and more varied circumstances and settings.
Abstract
from arXiv · showhide
As autonomous vehicles and autonomous racing rise in popularity, so does the need for faster and more accurate detectors. While our naked eyes are able to extract contextual information almost instantly, even from far away, image resolution and computational resources limitations make detecting smaller objects (that is, objects that occupy a small pixel area in the input image) a genuinely challenging task for machines and a wide-open research field. This study explores how the popular YOLOv5 object detector can be modified to improve its performance in detecting smaller objects, with a particular application in autonomous racing. To achieve this, we investigate how replacing certain structural elements of the model (as well as their connections and other parameters) can affect performance and inference time. In doing so, we propose a series of models at different scales, which we name `YOLO-Z', and which display an improvement of up to 6.9% in mAP when detecting smaller objects at 50% IOU, at the cost of just a 3ms increase in inference time compared to the original YOLOv5. Our objective is to inform future research on the potential of adjusting a popular detector such as YOLOv5 to address specific tasks and provide insights on how specific changes can impact small object detection. Such findings, applied to the broader context of autonomous vehicles, could increase the amount of contextual information available to such systems.
1 Introduction
Small-object detection is difficult because limited pixels and context lose significance through processing, yet it matters for real-time autonomous racing. The study therefore modifies YOLOv5 structurally to improve small-object detection while preserving real-time operation.
- Small objects are difficult to detect because limited resolution and context reduce the information available to the model.
- Two-stage detectors can improve performance but are less suited to real-time applications because of their inference-time cost.
- YOLOv5 is a popular, fast, accessible single-stage detector whose existing optimizations largely adjust parameters or training data rather than model structure.
- The study targets an autonomous racing vehicle that detects differently coloured cones and proposes modified YOLOv5 models designed to maintain real-time processing speeds.
- The paper contributes a small-object-focused YOLOv5 model and an experimental methodology for analyzing and modifying its structural elements.
2 Related work
Related work spans detector architectures, YOLO applications, image-region and scale-focused processing, backbone changes, and feature-pyramid methods. The paper positions structural YOLOv5 modification as a way to address small-object detection under autonomous-driving latency requirements.
- One-stage vs two-stage object detectors: Two-stage detectors decompose detection into region proposal generation and classification, while related work often prioritizes performance over inference time.
- YOLO detectors: YOLO frames object detection as regression, simplifying networks to support faster real-time models and motivating further study of isolated architectural changes.
- YOLOv5: YOLOv5 emphasizes accessibility through PyTorch and shares a design with YOLOv4, making it a practical basis for adaptation.
- YOLO applications and optimization: YOLO-based applications detect small objects, but several optimize use cases through anchors, augmentation, contextual processing, or limited architectural changes.
- Region, scale, and feature-map processing: Region-focused systems can adjust resolution but may require multiple network passes, whereas feature-pyramid networks aggregate feature maps to enhance backbones.
- Autonomous driving: Autonomous-driving detectors prioritize inference time, while smaller-object detection can provide information about farther-away objects and vehicle surroundings.
3 Methodology
The methodology evaluates structural modifications to YOLOv5 across four model scales using an annotated autonomous-racing cone dataset and controlled comparisons against unmodified baselines. It measures overall and small-object detection performance alongside inference speed while varying model components, feature-map connections, and experimental settings.
- Experimental design: YOLOv5 models at S, M, L, and X scales were individually modified and treated as separate models for evaluation.The scales apply different depth and width multipliers while preserving the overall architecture.
- Experimental design: The experiments first established unmodified YOLOv5 baselines, filtered ineffective individual changes, and then tested combinations of selected techniques.This staged process was repeated while observing accuracy and inference-time effects.
- Evaluation: Performance was evaluated at large, medium, and small object scales plus overall performance using reimplemented COCO-style metrics, with small objects defined as areas below 32 squared pixels.The metric module also used 96 squared pixels as the threshold for large objects.
- Dataset: The dataset contains close to 4,000 annotated images across four cone classes, including digitally augmented images and challenging weather conditions.The data represent views from an autonomous racing car and support path planning based on cone colours.
- Dataset: Over 30,000 labelled objects provide high object density, but blue and yellow cones are overrepresented and class imbalance affects overall results.Evaluation therefore averages scores for the most prominent classes.
- Dataset: The dataset was split into training, validation, and testing sets using a 65:15:20 ratio.The validation set informed training, while the training and test sets were larger.
- Architectural modifications: The study modified YOLOv5 through configurable model instructions, backbone replacements, neck alternatives, and changes to feature-map connections and model width or depth.The tested backbones included ResNet50 and proportionally downscaled DenseNet; the neck alternatives included FPN and BiFPN.
- Architectural modifications: Higher-resolution feature maps were redirected to the neck and head either inclusively by adding a map or exclusively by replacing the lowest-resolution map.These changes target small-object information while considering inference-time and large-object trade-offs.
4 Results
Across experiments, YOLO-Z combines architectural changes with dataset- and scale-dependent settings to improve small-object detection over YOLOv5 while retaining real-time-compatible inference. The strongest effects involve feature-map routing, DenseNet backbones, and selected anchor and training configurations, although gains vary by scale and require broader validation.
- Backbone: DenseNet consistently improved performance with an approximately 3ms fixed inference-time increase, whereas ResNet generally worsened performance and increased inference time.The authors therefore considered DenseNet the better general fit for small-scale object detection.
- Neck architecture: Using only an FPN outperformed bi-FPN at the S scale, while the relative benefit of neck choices varied across model scales.In combined YOLO-Z models, FPN tended to outperform bi-FPN where earlier experiments had shown the opposite, with the X scale gaining less overall.
- Feature maps: Redirecting feature maps to the neck and head had the largest impact, particularly replacing the lowest-resolution map with a higher-resolution one.The authors attribute this to small objects occupying more pixels and influencing processing more strongly.
- Anchors: Dataset-generated anchors improved performance without affecting inference time, but preferred anchor counts depended on model scale.Three anchors performed better at S, while larger L and X models benefited more from five anchors.
- Modified models: YOLO-Z models achieved an average absolute improvement of 5.9 mAP for small objects at 50% IoU across scales, with a 2.6ms average inference-time increase.They also improved absolute mAP for all objects by 2.7 at the same IoU.
- Discussion: The proposed YOLO-Z family outperformed YOLOv5 while retaining inference times compatible with real-time autonomous racing, especially for small objects.Performance remained stable for medium-sized objects, and the techniques may also apply to other small or distant-object applications.
- Limitations: The empirical gains and their generality should be investigated further using different datasets and settings, including challenges such as traffic-sign detection.The authors note that the techniques require testing across varied circumstances for refinement and better understanding.
5 Conclusions
The study develops YOLO-Z models by altering YOLOv5 architectures for small-object detection in autonomous racing. The resulting models improve detection while adding limited inference time.
- YOLO-Z models improve small-object detection by modifying architectural and model elements of YOLOv5.
- Close to 6% improvement in mAP at 50% IoU is achieved for small-object detection.
- Around 3 ms of additional inference time accompanies the reported detection improvement.