Source-linked AI summary
Scaled-YOLOv4: Scaling Cross Stage Partial Network
Chien-Yao Wang, Alexey Bochkovskiy, Hong-Yuan Mark Liao
TL;DR
The paper addresses how to scale YOLOv4 across small and large networks while maintaining speed and accuracy. It modifies depth, width, resolution, and network structure through a CSP-based scaling approach, achieving 56.0% AP for YOLOv4-large and 1774 FPS for YOLOv4-tiny under specified conditions. The resulting models provide strong speed–accuracy performance across deployment scales.
Problem
Object-detector scaling must support diverse devices and computation budgets, while prior methods rarely analyze relationships among scaling parameters for object detection.
Method
The paper redesigns YOLOv4 as YOLOv4-CSP and scales depth, width, resolution, and network structure for small and large models.
Results
56.0% AP is achieved by YOLOv4-large on COCO test-dev, while YOLOv4-tiny reaches 1774 FPS on RTX 2080Ti using TensorRT-FP16 with batch size 4.
Takeaways & Limitations
CSP-based YOLOv4 scaling applies to small and large networks while delivering optimal speed and accuracy across the model family.
Abstract
from arXiv · showhide
We show that the YOLOv4 object detection neural network based on the CSP approach, scales both up and down and is applicable to small and large networks while maintaining optimal speed and accuracy. We propose a network scaling approach that modifies not only the depth, width, resolution, but also structure of the network. YOLOv4-large model achieves state-of-the-art results: 55.5% AP (73.4% AP50) for the MS COCO dataset at a speed of ~16 FPS on Tesla V100, while with the test time augmentation, YOLOv4-large achieves 56.0% AP (73.3 AP50). To the best of our knowledge, this is currently the highest accuracy on the COCO dataset among any published work. The YOLOv4-tiny model achieves 22.0% AP (42.0% AP50) at a speed of 443 FPS on RTX 2080Ti, while by using TensorRT, batch size = 4 and FP16-precision the YOLOv4-tiny achieves 1774 FPS.
1. Introduction
Scaled-YOLOv4 addresses the need to adapt object detectors across diverse devices by scaling network depth, width, resolution, and structure. Built from YOLOv4-CSP, it develops systematically scaled small and large models targeting speed–accuracy trade-offs across deployment settings.
- Model scaling is important for achieving high accuracy and real-time inference across cloud, GPU, IoT, and embedded devices.
- Existing scaling commonly changes backbone depth and width to produce networks suited to different device classes.Examples include deeper ResNet variants for cloud GPUs and smaller variants for embedded systems.
- Prior approaches also use training and search decoupling, knowledge distillation, and compound scaling over width, depth, and resolution.
- CSPDarknet53 matches almost all optimal architecture features identified through network architecture search, motivating scaling methods based on YOLOv4.The proposed process first redesigns YOLOv4 as YOLOv4-CSP, then develops scaled-YOLOv4 with separate considerations for small and large models.
- Scaled-YOLOv4 targets the best speed–accuracy trade-off and real-time detection for 16 FPS, 30 FPS, and 60 FPS video.
2. Related work
Related work spans one-stage and two-stage detection, traditional depth and width scaling, and NAS-based compound scaling. The paper identifies a gap in analyzing relationships among scaling parameters for object detection.
- 2.1. Real-time object detection: One-stage detectors use one CNN operation, whereas two-stage detectors add a second CNN over high-score region proposals.Their inference times are expressed as T_one = T_1st and T_two = T_1st + mT_2nd, so two-stage latency varies with the number of proposals.
- 2.1. Real-time object detection: Real-time detection generally favors one-stage detectors, including anchor-based EfficientDet and YOLOv4 and anchor-free CenterNet.
- 2.2. Model scaling: Traditional model scaling changes depth or width, while later architectures combine both dimensions through compound scaling.Examples include VGG and ResNet depth variants, Wide ResNet width scaling, and compound versions of DenseNet and ResNeXt.
- 2.2. Model scaling: NAS-based methods scale or search over feature-pyramid paths, image size, width, depth, and detector-specific modules or stage parameters.
- 2.2. Model scaling: Few prior methods analyze relationships among scaling parameters, motivating synergistic compound scaling based on object-detection design requirements.
3. Principles of model scaling
The paper analyzes how image size, depth, width, architecture, and detector components affect scaling costs, then uses CSP-based designs to reduce computation and guide tiny- and large-model scaling.
- Scaling image size, depth, and width increases computation quadratically, linearly, and quadratically, respectively.
- 23.5%, 46.7%, and 50.0% FLOPs reductions are obtained by CSP-izing ResNet, ResNeXt, and Darknet, respectively.
- Scaling Tiny Models for Low-End Devices: Tiny-model scaling must account for memory bandwidth, memory access cost, DRAM traffic, and computation complexity, not only computation and model size.
- Scaling Tiny Models for Low-End Devices: OSANet has lower computation complexity than the O(whkb^2) ResNet series, motivating its use in the tiny model.
- Scaling Tiny Models for Low-End Devices: CSPOSANet splits base and generated channels into two paths and can obtain the best CIO when kg > b/2.
- Scaling Large Models for High-End GPUs: For scaling up, the compound of input size and stage count has the best impact, after which depth and width are adjusted according to real-time requirements.
4. Scaled-YOLOv4
Scaled-YOLOv4 adapts YOLOv4-CSP across general, low-end, and high-end GPUs by redesigning CSP-based backbone, neck, tiny, and large-model structures.
- Scaled-YOLOv4 targets general GPUs, low-end GPUs, and high-end GPUs through device-specific model designs.
- YOLOv4-CSP redesigns YOLOv4 to improve its speed/accuracy trade-off, including CSP-based backbone and neck changes.
- 40% of PAN computation is cut after CSP-izing the neck architecture.
- YOLOv4-tiny: YOLOv4-tiny is designed for low-end GPUs using a CSPOSANet backbone with partial computational-block architecture.
- YOLOv4-large: YOLOv4-large uses fully CSP-ized YOLOv4-P5 and scales it to YOLOv4-P6 and YOLOv4-P7 for cloud GPUs.
5. Experiments
Experiments evaluate scaled-YOLOv4 through CSP-ization ablations, tiny-model design, comparisons with real-time detectors, test-time augmentation, embedded-device tests, and once-for-all sub-networks. The results support flexible scaling with strong speed–accuracy trade-offs across model sizes and operating conditions.
- 5.1. Ablation study on CSP-ized model: CSP-ized models reduce parameters and computations by 32% while improving Batch 8 throughput and AP.At equal throughput, selected CSP-ized variants improve AP by 1% or 1.6% with lower computing resources.
- 5.2. Ablation study on YOLOv4-tiny: COSA provides a more flexible and faster tiny-model design than linear scaling, yielding the best speed–accuracy trade-off for YOLOv4-tiny.Under limited operating conditions, tinyCD53s becomes an inference-speed bottleneck, while COSA achieves higher AP at the same computation level.
- 5.4. Scaled-YOLOv4 for object detection: All scaled-YOLOv4 models are Pareto optimal across the reported detection indicators, with faster inference than similarly accurate EfficientDet models.YOLOv4-CSP is 1.9 times faster than EfficientDet-D3 at 47.5% accuracy; YOLOv4-P5 is 2.9 times faster than EfficientDet-D5 at comparable accuracy.
- 5.4. Scaled-YOLOv4 for object detection: TTA increases AP for YOLOv4-P5, YOLOv4-P6, and YOLOv4-P7 by 1.1%, 0.7%, and 0.5%, respectively.The experiments evaluate TTA on YOLOv4-large models.
- 5.4. Scaled-YOLOv4 for object detection: YOLOv4-tiny achieves real-time performance across tested embedded GPUs and reaches 1774 FPS on RTX 2080Ti with TensorRT FP16 at batch size 4.On Xavier AGX and Xavier NX with FP16 and batch size 4, frame rates reach 380 FPS and 199 FPS, respectively.
- 5.5. Scaled-YOLOv4 as na¨ıve once-for-all model: Pruned YOLOv4-P7 sub-networks perform best at progressively lower resolutions, enabling compound scaling of model architecture and input size.YOLOv4-P7 is best at high resolution, YOLOv4-P7\P7 at middle resolution, and YOLOv4-P7\P7\P6 at low resolution.
6. Conclusions
The conclusion presents CSP-based YOLOv4 as scalable both upward and downward across small and large networks while maintaining speed and accuracy. It highlights 56.0% AP for YOLOv4-large with TTA and 1774 FPS for YOLOv4-tiny on RTX 2080Ti with TensorRT-FP16.
- CSP-based YOLOv4 scales up and down across small and large networks while maintaining optimal speed and accuracy.
- 56.0% AP is achieved by YOLOv4-large on COCO test-dev, while YOLOv4-tiny reaches 1774 FPS on RTX 2080Ti using TensorRT-FP16.