PaperlayerView exact revision on arXiv

Source-linked AI summary

YOLOv4: Optimal Speed and Accuracy of Object Detection

Alexey Bochkovskiy, Chien-Yao Wang, Hong-Yuan Mark Liao

arXiv:2004.10934v1cs.CVeess.IV

TL;DR

Real-time object detectors often trade accuracy for speed, limiting their use in production systems. YOLOv4 combines architectural and training improvements to deliver real-time, high-quality detection on conventional GPUs.

  • Problem

    Modern object detectors often trade accuracy for speed, while accurate models are not real-time and require substantial computational resources.

  • Method

    YOLOv4 combines CSPDarknet53, SPP, PANet, and a YOLOv3 head with Bag-of-Freebies and Bag-of-Specials training improvements.

  • Results

    YOLOv4 delivers real-time, high-quality object detection and is designed for efficient training and parallel computation on conventional GPUs.

  • Takeaways & Limitations

    The work targets production-ready object detection that can be trained and used on widely available GPUs.

  • Takeaways & Limitations

    The authors plan to expand the Bag of Freebies and experimentally evaluate each feature’s influence in future work.

Abstract

from arXiv · show

There are a huge number of features which are said to improve Convolutional Neural Network (CNN) accuracy. Practical testing of combinations of such features on large datasets, and theoretical justification of the result, is required. Some features operate on certain models exclusively and for certain problems exclusively, or only for small-scale datasets; while some features, such as batch-normalization and residual-connections, are applicable to the majority of models, tasks, and datasets. We assume that such universal features include Weighted-Residual-Connections (WRC), Cross-Stage-Partial-connections (CSP), Cross mini-Batch Normalization (CmBN), Self-adversarial-training (SAT) and Mish-activation. We use new features: WRC, CSP, CmBN, SAT, Mish activation, Mosaic data augmentation, CmBN, DropBlock regularization, and CIoU loss, and combine some of them to achieve state-of-the-art results: 43.5% AP (65.7% AP50) for the MS COCO dataset at a realtime speed of ~65 FPS on Tesla V100. Source code is at https://github.com/AlexeyAB/darknet

1. Introduction

YOLOv4 targets fast, accurate object detection for production systems and conventional-GPU use. The work develops an efficient detector, evaluates Bag-of-Freebies and Bag-of-Specials, and adapts methods for single-GPU training.

  • Motivation: Real-time object detector accuracy could extend CNN detectors from recommendation systems to stand-alone process management and reduced human input.The introduction contrasts slow accurate models for parking-space searches with fast inaccurate models for collision warnings.
  • Design goal: The main goal is production-speed object detection optimized for parallel computation rather than low theoretical BFLOP.The authors also seek a detector that is easy to train and use while achieving real-time, high-quality results on conventional GPUs.
  • Contributions: The work verifies the influence of state-of-the-art Bag-of-Freebies and Bag-of-Specials methods during detector training.This evaluation is framed as a contribution of the paper.
  • Contributions: YOLOv4 modifies state-of-the-art methods, including CBN, PAN, and SAM, to make them more efficient and suitable for single GPU training.The cited methods are identified as examples of the adapted components.

2. Related work

Related object detectors combine pretrained backbones with prediction heads and are organized into dense one-stage or sparse two-stage architectures. Prior work also improves detectors through training-only “bag of freebies” methods and low-cost inference “bag of specials” modules and post-processing.

  • Detector architecture: Modern object detectors typically combine an ImageNet-pretrained backbone with a head that predicts object classes and bounding boxes.GPU backbones include VGG, ResNet, ResNeXt, and DenseNet, while CPU backbones include SqueezeNet, MobileNet, and ShuffleNet.
  • Detector architecture: Researchers also develop new detection-specific backbones or complete models, including DetNet, DetNAS, SpineNet, and HitDetector.These approaches focus directly on constructing a new backbone or an entirely new object-detection model.
  • Detector architecture: Dense one-stage detectors include anchor-based RPN, SSD, YOLO, and RetinaNet, while sparse two-stage detectors include Faster R-CNN, R-FCN, and Mask R-CNN.The related architectures also include anchor-free methods such as CornerNet, CenterNet, MatrixNet, FCOS, and RepPoints.
  • Bag of freebies: “Bag of freebies” methods change training strategy or training cost to improve detection accuracy without increasing inference cost.Examples include data augmentation, methods addressing class imbalance and label representation, and bounding-box regression objectives.
  • Bag of specials: “Bag of specials” methods add small inference costs while improving accuracy through modules that enlarge receptive fields, introduce attention, strengthen feature integration, or post-process predictions.Representative modules include SPP, ASPP, RFB, SE, SAM, SFAM, ASFF, and BiFPN; NMS is a common post-processing method.

3. Methodology

YOLOv4 is designed for fast production inference and parallel computation by balancing network resolution, depth, parameters, and outputs. Its selected architecture combines CSPDarknet53, SPP, PANet, and a YOLOv3 anchor-based head, with additional training and detection modifications including Mosaic, SAT, CmBN, and modified aggregation mechanisms.

  • Design objectives: The detector design prioritizes an optimal balance among input resolution, convolutional-layer number, parameter number, and layer outputs rather than theoretical BFLOP volume alone.The stated aim is fast operating speed in production systems and optimization for parallel computations.
  • Backbone selection: 29 convolutional 3 × 3 layers, a 725 × 725 receptive field, and 27.6 M parameters characterize CSPDarknet53, versus 16 layers, 425 × 425, and 20.6 M for CSPResNeXt50.The methodology hypothesizes that larger receptive fields and more parameters are suitable for detector backbones.
  • Architecture: YOLOv4 selects CSPDarknet53 as backbone, SPP as the additional module, PANet as the path-aggregation neck, and a YOLOv3 anchor-based head.SPP increases receptive field with almost no reduction in operating speed, while PANet replaces the FPN used in YOLOv3.
  • Training modifications: For single-GPU training, YOLOv4 introduces Mosaic and SAT, selects hyperparameters with genetic algorithms, and modifies SAM, PAN, and batch normalization as CmBN.CmBN collects statistics only between mini-batches within a single batch; modified SAM uses point-wise attention, and modified PAN replaces shortcut connections with concatenation.
  • Training modifications: Mosaic augmentation mixes four training images, exposing objects to different contexts and reducing the need for a large mini-batch through batch-normalization statistics from four images.CutMix mixes only two input images, whereas Mosaic allows detection of objects outside their normal context.
  • Training modifications: Self-Adversarial Training uses two forward-backward stages: the network first alters the image to hide the target object, then learns to detect the object in the modified image.The method treats SAT as a data augmentation technique that operates on the original image rather than initially changing network weights.

4. Experiments

The experiments evaluate training-improvement techniques for ImageNet classification and MS COCO detection. CutMix, Mosaic, label smoothing, Mish, and detector-specific BoF/BoS methods improve or optimize performance, with SPP, PAN, and SAM yielding the detector’s best performance.

  • Experimental scope: The study tests training techniques on ImageNet classification and MS COCO object detection.The experiments assess classifier accuracy on ILSVRC 2012 validation data and detector accuracy on MS COCO test-dev 2017 data.
  • ImageNet classification: CutMix, Mosaic augmentation, class label smoothing, and Mish activation improve classifier accuracy.The resulting classifier BoF-backbone includes CutMix, Mosaic, and label smoothing, while Mish is used as a complementary option.
  • MS COCO detection: The detector BoF study evaluates grid-sensitivity elimination, Mosaic, IoU assignment, genetic hyperparameter selection, label smoothing, CmBN, cosine annealing, dynamic mini-batches, optimized anchors, and box-regression losses.The listed methods are studied as accuracy-improving features without affecting FPS.
  • MS COCO detection: The detector BoS study includes PAN, RFB, SAM, Gaussian YOLO, and ASFF.The experiments report best detector performance when using SPP, PAN, and SAM.

S M IT GA LS CBN CA DM OA loss AP AP50 AP75

The ablations show that detector accuracy depends on the backbone and transfer of classifier pre-training, while BoF and BoS make performance nearly insensitive to mini-batch size. CSPDarknet53 is more suitable for detection than CSPResNeXt50 under the studied improvements.

  • Backbone models: CSPDarknet53 achieves higher object-detection accuracy than CSPResNeXt50 despite CSPResNeXt50 having higher classification accuracy.The best classification model is not always the best detector.
  • Backbone models: BoF and Mish improve both classification and detector accuracy with CSPDarknet53, but reduce detector accuracy when transferred from CSPResNeXt50.Thus, CSPDarknet53 is more suitable for detector training than CSPResNeXt50.
  • Backbone models: CSPDarknet53 has greater ability to improve detector accuracy through various improvements.The paper identifies this backbone as more suitable for detection in the studied comparisons.
  • Mini-batch size: After adding BoF and BoS, mini-batch size has almost no effect on detector performance.The result removes the need for expensive GPUs to train an excellent detector, according to the passage.

5. Results

YOLOv4 lies on the Pareto-optimal curve and is reported as superior to the fastest and most accurate object detectors in both speed and accuracy. Its inference speed is compared across commonly adopted Maxwell, Pascal, and Volta GPU architectures.

  • YOLOv4 lies on the Pareto-optimality curve and is superior to the fastest and most accurate detectors in both speed and accuracy.
  • Inference-time verification compares YOLOv4 with other state-of-the-art methods across commonly adopted Maxwell, Pascal, and Volta GPU architectures.
  • Frame-rate comparisons use GTX Titan X (Maxwell) or Tesla M40 for Maxwell, and Titan X (Pascal), Titan Xp, GTX 1080 Ti, or Tesla P100 for Pascal.

6. Conclusions

The paper presents a state-of-the-art detector that combines speed and accuracy advantages, remains usable on conventional 8–16 GB-VRAM GPUs, and validates one-stage anchor-based detection.

  • 6. Conclusions: The detector is faster and more accurate than all available alternative detectors on MS COCO AP50...95 and AP50.The comparison is stated in terms of FPS, MS COCO AP50...95, and AP50.
  • 6. Conclusions: The detector can be trained and used on conventional GPUs with 8–16 GB of VRAM, enabling broad use.The paper explicitly links the 8–16 GB-VRAM requirement to broader applicability.
  • 6. Conclusions: The results verify the viability of the original one-stage anchor-based detector concept.The conclusion describes the concept as having proven its viability.
  • 6. Conclusions: The authors tested many features and selected those that improve classifier and detector accuracy.The conclusion states that a large number of features were verified before selecting features for accuracy improvement.
Loading 2004.10934v1…