Source-linked AI summary
Improved YOLOv5 network for real-time multi-scale traffic sign detection
Junfan Wang, Yi Chen, Mingyu Gao, Zhekang Dong
TL;DR
Traffic sign detection must handle large scale variation while meeting real-time and vehicle-side resource constraints. The paper modifies YOLOv5s with AF-FPN and automatic data augmentation, reporting improved TT100K detection performance with unchanged FLOPs and 95 FPS vehicle-side inference.
Problem
Traffic sign detectors must recognize targets at different scales accurately and quickly, while feature pyramids may lose information across feature levels.
Method
The paper replaces YOLOv5s’s original FPN with AF-FPN and uses automatic learning data augmentation instead of the original mosaic augmentation.
Results
On TT100K, the proposed method achieves 65.14% mAP across 182 traffic sign classes and 95 FPS vehicle-side detection speed.
Takeaways & Limitations
The proposed network improves multi-scale traffic sign detection while remaining suitable for real-time vehicle-side deployment.
Abstract
from arXiv · showhide
Traffic sign detection is a challenging task for the unmanned driving system, especially for the detection of multi-scale targets and the real-time problem of detection. In the traffic sign detection process, the scale of the targets changes greatly, which will have a certain impact on the detection accuracy. Feature pyramid is widely used to solve this problem but it might break the feature consistency across different scales of traffic signs. Moreover, in practical application, it is difficult for common methods to improve the detection accuracy of multi-scale traffic signs while ensuring real-time detection. In this paper, we propose an improved feature pyramid model, named AF-FPN, which utilizes the adaptive attention module (AAM) and feature enhancement module (FEM) to reduce the information loss in the process of feature map generation and enhance the representation ability of the feature pyramid. We replaced the original feature pyramid network in YOLOv5 with AF-FPN, which improves the detection performance for multi-scale targets of the YOLOv5 network under the premise of ensuring real-time detection. Furthermore, a new automatic learning data augmentation method is proposed to enrich the dataset and improve the robustness of the model to make it more suitable for practical scenarios. Extensive experimental results on the Tsinghua-Tencent 100K (TT100K) dataset demonstrate the effectiveness and superiority of the proposed method when compared with several state-of-the-art methods.
2 Related Works
Traffic sign detection requires accurate recognition across widely varying target scales while retaining fast inference. Prior work addresses this through detector architecture changes, multi-scale feature fusion, and data augmentation, but feature pyramids can lose information across levels.
- CNN-based traffic sign detection: Single-stage detectors are commonly used for traffic detection because of their fast detection performance.
- CNN-based traffic sign detection: Traffic signs vary substantially in scale and occupy a small portion of scene images, making scale-invariant representation a major challenge.
- CNN-based traffic sign detection: FPN improves multi-scale recognition through multi-layer feature fusion but can lose information when feature channels are reduced.
- CNN-based traffic sign detection: Reduced channels in high-level features can decrease detection accuracy for large-scale targets.
- Data augmentation: Data augmentation methods include color and geometric transformations that expand and enrich training datasets.
- Data augmentation: AutoAugment automatically searches for improved data augmentation policies from the training data.
3 Proposed Method
The proposed method modifies YOLOv5s for vehicle-side deployment by replacing its original feature pyramid with AF-FPN and replacing mosaic augmentation with automatically selected augmentation policies. The framework retains YOLOv5’s input, backbone, neck, and prediction stages while targeting stronger multi-scale recognition.
- Improved YOLOv5s network framework: YOLOv5s is the smallest YOLOv5 model, with 14.10M memory size, but its accuracy is insufficient for small-scale targets.
- Improved YOLOv5s network framework: YOLOv5 consists of Input, Backbone, Neck, and Prediction components, with CSPDarknet53 extracting features and FPN plus PANet aggregating them.
- Improved YOLOv5s network framework: AF-FPN replaces the original FPN to improve multi-scale recognition and trade off recognition speed against accuracy.
- Improved YOLOv5s network framework: The method removes mosaic augmentation and uses the best operations selected by an automatic learning policy to enrich the dataset.
- Improved YOLOv5s network framework: The proposed YOLOv5s network architecture is presented in Fig. 1.
A B IoU
The method replaces GIoU with CIoU loss and uses AF-FPN components to preserve multi-scale context and enhance feature representations. CIoU adds center-distance and aspect-ratio considerations to bounding-box regression.
- A B IoU: CIoU loss considers bounding-box overlap, central-point distance, and aspect-ratio consistency.
- A B IoU: CIoU penalizes normalized distances between the central points of predicted and ground-truth boxes using the smallest enclosing box.
- A B IoU: The overlap-area factor receives higher priority during regression, especially for non-overlapping boxes.
- A B IoU: AF-FPN adds adaptive attention and feature enhancement modules to reduce high-level context loss and strengthen feature-pyramid representations.
- A B IoU: The feature pipeline applies AAM to C5, fuses M6 with M5, propagates features top-down, and expands receptive fields through FEM.
PANet
AF-FPN combines adaptive attention and feature enhancement modules to preserve multi-scale context and adapt receptive fields in YOLOv5 traffic-sign detection. Its augmentation policy uses randomly selected two-operation sub-policies learned through reinforcement learning.
- AAM: AAM pools context at multiple scales, generates spatial weight maps, and fuses the weighted features into the high-level feature map M6.The resulting M6 contains richer multi-scale context and alleviates information loss from channel reduction.
- FEM: FEM uses parallel dilated convolutions to provide different receptive fields and average pooling to fuse their traffic information for multi-scale prediction.The three branches use the same kernel size with different dilation rates.
- FEM: Dilated convolution expands receptive fields while retaining feature-map resolution, with receptive-field size determined by kernel size, dilation rate, and stride.The cited formulation describes how dilation changes receptive-field coverage relative to standard convolution.
- Data Augmentation: The augmentation search space contains 5 sub-strategies, each applying two sequential image operations with learned probabilities and magnitudes.One sub-policy is randomly selected for each image, and the policy is learned with a controller RNN using PPO.
4 Experiments and Analysis
Experiments on TT100K evaluate the improved YOLOv5 model across multi-scale traffic signs, accuracy, missed detections, deployment size, and inference speed. The method achieves strong small-target and overall detection performance while remaining suitable for mobile real-time use.
- Experimental setting: 42.5% of traffic signs in TT100K are small objects, motivating evaluation of multi-scale detection performance.The dataset contains 182 traffic-sign types with detailed annotations across different traffic environments.
- Efficiency and deployment: 16.3M model size and 17.9G FLOPs support mobile deployment and real-time detection on the vehicle side.The authors report that the model is suitable for mobile platforms and requires less hardware than larger networks.
- Missed-detection analysis: The method has a significantly lower missed-detection rate than other methods across 19 traffic-sign types, although ip, w57, and po remain difficult.The authors identify these categories as having high missed-detection rates requiring future improvement.
- Qualitative and mobile evaluation: Qualitative results show recognition of multi-sized traffic signs in actual scenes with almost no missed or false detections.The trained model was also transplanted to a mobile platform with an external camera for real-time road-scene recognition.
- Ablation study: The complete method reaches 0.6514 mAP versus 0.6018 for YOLOv5s, while augmentation alone and AF-FPN alone reach 0.6131 and 0.6267.The authors describe the complete model's mAP as 4.96% higher than the YOLOv5s baseline, with unchanged FLOPs.
5 Conclusion
The paper concludes that modified YOLOv5s with AF-FPN and automatic data augmentation improves multi-scale traffic-sign recognition while retaining fast inference and mobile deployability. On TT100K, it reports 95 FPS and a 4.96% mAP improvement over the original network.
- Conclusion: AF-FPN improves feature-map information extraction and representation for detecting multi-scale traffic signs.The structure provides feature maps with different receptive fields and fuses receptive-field pyramids without additional calculations.
- Conclusion: The automatic data augmentation strategy enriches the traffic-sign dataset with Noise, Mosaic, and other methods to improve model training.
- Conclusion: 95 FPS enables fast vehicle-side inference, while the model's small size supports mobile-device deployment and real-time recognition.
- Conclusion: The improved network increases mAP by 4.96% over the original YOLOv5 network on TT100K.