PaperlayerView exact revision on arXiv

Source-linked AI summary

YOLOv1 to YOLOv10: The fastest and most accurate real-time object detection systems

Chien-Yao Wang, Hong-Yuan Mark Liao

arXiv:2408.09332v1cs.CV

TL;DR

Real-time object detection supports many downstream applications, and the review revisits how the YOLO series developed and influenced the field from a modern technical perspective. It traces that evolution and identifies YOLO’s influence across ease of use, accuracy, speed, and versatility in varied fields.

  • Problem

    Because real-time object detection supports many downstream applications, the review addresses how to characterize YOLO’s evolution and influence from a modern technical perspective.

    Source: §I · p. 1, §V · p. 10

  • Method

    The review traces YOLO’s evolution, examines its technologies and contributions through modern object-detection perspectives, and surveys related models and applications across fields.

    Source: §II · p. 2, §V · p. 10

  • Results

    The review analyzes YOLO’s influence in terms of ease of use, accuracy, speed, and versatility across various fields.

    Source: §V · p. 10

  • Takeaways & Limitations

    The review aims to help readers understand how to develop real-time computer-vision methods and identify tasks for which YOLO-related models may be used.

    Source: §V · p. 10

Abstract

from arXiv · show

This is a comprehensive review of the YOLO series of systems. Different from previous literature surveys, this review article re-examines the characteristics of the YOLO series from the latest technical point of view. At the same time, we also analyzed how the YOLO series continued to influence and promote real-time computer vision-related research and led to the subsequent development of computer vision and language models.We take a closer look at how the methods proposed by the YOLO series in the past ten years have affected the development of subsequent technologies and show the applications of YOLO in various fields. We hope this article can play a good guiding role in subsequent real-time computer vision development.

I. INTRODUCTION · II. YOLO SERIES

The review presents YOLO as an influential real-time detection family whose unified one-stage approach supports edge deployment, and examines its methods and impact across computer vision.

I. INTRODUCTION

  • Real-time edge detection has become important for applications such as autonomous driving, industrial robotics, authentication, health care, and surveillance.Object detection can also support instance segmentation, tracking, behavior analysis, and face recognition.Source: §I · p. 1
  • R-CNN introduced a two-stage detection process, while YOLO used per-grid prediction to detect objects in one step.Fast R-CNN and Faster R-CNN moved toward end-to-end detection; YOLO was presented as a groundbreaking real-time approach.Source: §I · p. 1
  • Scaled-YOLOv4 made one-stage object detection more accurate than all contemporary two-stage methods in general object detection.The review describes this as a first for one-stage detection and notes that it prompted subsequent research based on YOLO.Source: §I · p. 1
  • RetinaNet and YOLOv3 [3] advanced one-stage accuracy, and YOLO became a preferred choice for real-time object analysis in industry and research.Source: §I · p. 1
  • YOLO also underpins real-time systems for instance and image segmentation, pose estimation, 3D detection, and open-vocabulary detection.Source: §I · p. 1

II. YOLO SERIES

  • The review covers representative YOLO methods and related literature, their influence on computer vision, and applications across computer vision fields.It emphasizes state-of-the-art detection methods and reviews their advantages and disadvantages.Source: §I · p. 1, §II · p. 2
  • YOLO’s unified one-stage method avoids first proposing object locations and then analyzing each location, enabling streamlined detection on edge devices and in real-time applications.Source: §II · p. 2

A. YOLO (YOLOv1) · B. YOLO9000 (YOLOv2)

YOLOv1 introduced one-stage detection with grid-based predictions and global context, while YOLOv2 combined convolutional detection with clustered anchors and hierarchical joint training to detect 9,000 categories.

A. YOLO (YOLOv1)

  • Redmon et al. introduced YOLOv1’s one-stage detector, which predicts classes and bounding boxes per grid without a separate proposal-generation stage.The design avoids selective search and the additional parameters, computation, and irregular RoI operations of two-stage detection.Source: A. YOLO (YOLOv1) · p. 2
  • YOLOv1 directly regresses box dimensions without anchors, enabling some special examples to be predicted more accurately but making convergence more difficult.Its confidence score multiplies objectness, trained with predicted IoU as a soft label, by classification probability.Source: A. YOLO (YOLOv1) · p. 2
  • YOLOv1 uses fully connected layers to give each grid global context, reducing background error by more than half compared with Fast R-CNN.This lets each grid access a sufficient feature range regardless of the underlying CNN architecture.Source: A. YOLO (YOLOv1) · p. 2

B. YOLO9000 (YOLOv2)

  • Redmon and Farhadi made YOLOv2 fully convolutional, combining high- and low-resolution features with anchor-based prediction; its simple inputs and outputs suit resource-limited devices.The review describes it as commonly used in maintaining and developing industrial applications, especially on low-end devices.Source: B. YOLO9000 (YOLOv2) · p. 2
  • YOLOv2 uses IoU-based k-means dimension clustering for anchors and directly predicts object centers from the responsible grid’s upper-left corner.Clustered dimensions avoid manually chosen aspect ratios and converge more easily than anchor-free regression; direct location prediction avoids unstable early-training offsets from anchor centers.Source: B. YOLO9000 (YOLOv2) · p. 2
  • YOLOv2’s passthrough layer combines losslessly reorganized high-resolution features with low-resolution features to improve small-object detection while maintaining speed.It calibrates resolution by pretraining on classification images at the same size used for detection training, avoiding the need to learn new object-size information during detection training.Source: B. YOLO9000 (YOLOv2) · p. 3
  • YOLOv2 jointly trains ImageNet classification and COCO detection using WordTree to integrate their categories, enabling detection of 9,000 object categories.ImageNet classification uses group softmax with a hierarchy similar to WordTree.Source: B. YOLO9000 (YOLOv2) · p. 3

C. YOLOv3 · D. Gaussian YOLOv3 · E. YOLOv4

YOLOv3 combined multiscale prediction and other architectural and label-assignment changes, while Gaussian YOLOv3 used distribution-based box regression to reduce false positives. YOLOv4 integrated training and inference techniques and introduced architectural changes, reporting high accuracy and inference speed with one GPU.

C. YOLOv3

  • YOLOv3 [3] combined FPN for simultaneous multiscale prediction, designed DarkNet53 with high GPU utilization, and changed label assignment to one anchor per ground truth and hard-label IoU-aware objectness.The paper notes DarkNet53 inferred faster than other architectures at the same calculation amount.Source: C. YOLOv3 · p. 3
  • YOLOv3 [3] used stride-1 max-pooling layers with kernel sizes from local to global so each grid could obtain multiresolution features, and SPP was described as improving accuracy.Source: C. YOLOv3 · p. 3
  • YOLOv3 [3] used FPN to predict across scales, which the paper says can greatly improve detection of small objects.Source: C. YOLOv3 · p. 3

D. Gaussian YOLOv3

  • Gaussian YOLOv3 changed prediction-head decoding to regress bounding-box distributions, representing uncertainty in (x, y, w, h) and significantly reducing false positives.Source: D. Gaussian YOLOv3 · p. 3

E. YOLOv4

  • YOLOv4 [4] replaced FPN with PAN and adopted CSPNet as its backbone; it demonstrated one-GPU training of a detector as accurate as those trained with more than 128 GPUs and with over three times the inference speed.The paper says most subsequent similar YOLO architectures followed this architecture.Source: E. YOLOv4 · p. 3
  • YOLOv4 [4] combined training-only Bag of Freebies with accuracy-improving Bag of Specials, and added self-adversarial training for robustness and GPU–CPU memory sharing to remove GPU-memory limits on batch size.Bag of Freebies includes loss, regularization, augmentation, and label-assignment methods; Bag of Specials includes receptive-field modules, attention, activation, and normalization methods.Source: E. YOLOv4 · pp. 3–4
  • YOLOv4 [4] addressed inaccurate predictions near grid boundaries with a decoder that keeps target values in the effective gradient range, after identifying near-zero Sigmoid gradients at extremes.Source: E. YOLOv4 · p. 4

F. Scaled-YOLOv4 · G. YOLOv5

Scaled-YOLOv4 combined architecture and scaling changes for edge and cloud detection, while YOLOv5 evolved through successive design revisions and introduced decoder and label-assignment features. The sections describe contributions to model performance, adaptability, and training stability.

F. Scaled-YOLOv4

  • Scaled-YOLOv4 achieved the highest accuracy and fastest inference among object detectors, with P5, P6, and P7 models for edge and cloud use.It trains from scratch without ImageNet pre-training and introduces CSPNet into PAN to improve speed, accuracy, parameter count, and computation.Source: F. Scaled-YOLOv4 · p. 4
  • Scaled-YOLOv4 scales image resolution, receptive-field matching, and model stages together to produce efficient architectures for high-resolution images.Its model scaling lets users adapt accuracy to input-image resolution at inference without retraining, by removing the output of the corresponding stage.Source: F. Scaled-YOLOv4 · p. 4
  • Scaled-YOLOv4 designed the hardware-efficient CSPDark and CSPOSA modules, informed by ShuffleNetv2 and HardNet.The design considers hardware performance.Source: F. Scaled-YOLOv4 · p. 4

G. YOLOv5

  • YOLOv5 combined YOLOv4’s CSPNet with scaled-YOLOv4’s CSPPAN in r1.0, then optimized speed-accuracy trade-offs and training hyperparameters through r5.0 and r7.0.Its initial open-source performance was slightly worse than YOLOv3-SPP, and ongoing maintenance made it the most popular YOLO development platform at the time described.Source: G. YOLOv5 · p. 4
  • YOLOv5’s power-based decoder replaces YOLOv3’s exponential length-width regression to improve training stability, but bounds recall through its limited anchor-relative output range.YOLOv5 also uses evolutionary algorithms for automatic anchor and hyperparameter search.Source: G. YOLOv5 · p. 4
  • YOLOv5 adds neighboring grids as positive samples to compensate for the power decoder’s recall limitation and enlarges the YOLOv4 center decoder’s sigmoid scaling coefficient.Source: G. YOLOv5 · p. 4

H. PP-YOLO · I. YOLOR

The sections describe PP-YOLO’s successive design changes and YOLOR’s contributions in implicit-knowledge modeling, multitask learning, and training techniques.

H. PP-YOLO

  • PP-YOLO [35] builds on YOLOv3 with YOLOv4 training techniques, CoordConv, Matrix NMS, and an improved ImageNet-pretrained model.Source: H. PP-YOLO · p. 5
  • PP-YOLOv2 [36] adds scaled-YOLOv4’s CSPPAN, while mobile-oriented PP-PicoDet [37] uses ShuffleNetv2 and GhostNet for one-shot neural architecture search and a YOLOX anchor-free decoder [41].Source: H. PP-YOLO · p. 5
  • PP-YOLOE [38] modifies RepVGG into CSPRepResStage and uses TOOD’s distribution-based bounding-box regression with DFL, which does not require a Gaussian data assumption.Source: H. PP-YOLO · p. 5
  • The review notes that nearly all YOLO versions after YOLOv6 follow the PP-YOLO series’ described design-feature format.Source: H. PP-YOLO · p. 5

I. YOLOR

  • YOLOR proposes vector-, neural-network-, and matrix-factorization-based latent-variable encoders that enhance feature alignment, prediction refinement, and multitask learning.Source: §I · p. 5
  • YOLOR provides models for object detection, image classification, and multiple-object tracking, as well as pose estimation models based on YOLO-Pose.Source: §I · p. 5
  • YOLOR’s advanced autoML technology, hyperparameter-training techniques, large-dataset pretraining, distillation, and self-supervised learning are described as continuing influences; the review calls it the most accurate YOLO model.Source: §I · p. 5

J. YOLOX · K. YOLOv6 · L. YOLOv7

YOLOX [41], YOLOv6, and YOLOv7 are described through changes to detection heads, label assignment, architectures, and training methods. The review also notes YOLOv7’s real-time state-of-the-art models across several vision tasks.

J. YOLOX

  • YOLOX [41] combines CSPNet with FCOS’ anchor-free head and introduces SimOTA to replace confusing manual label assignment.Its decoupled head makes classification and bounding-box regression easier to learn; IoU-based loss enables anchor-free heads to train without loss imbalance from object dimensions.Source: J. YOLOX · p. 5

K. YOLOv6

  • YOLOv6 moves from RepVGG in its initial version to CSPNet in later versions, including Li et al. (2022) [6] and Li et al. (2023) [51].Designed for industry, YOLOv6 emphasizes quantization and proposes YOLOv6-lite, a depth-wise-convolution architecture for lower-end devices.Source: K. YOLOv6 · pp. 5–6
  • YOLOv6 version 2.0 uses RepOPT, QAT, and self-distillation and channel-wise distillation to improve quantized-model stability and accuracy.The review says RepOPT slows accuracy loss after quantization; QAT improves quantized-model accuracy and reduces quantization-related accuracy loss.Source: K. YOLOv6 · pp. 5–6
  • YOLOv6 version 3.0 [51] introduces anchor-aided training, using an anchor-based head to assist anchor-free learning and improve accuracy.The later version 4.0 proposes YOLOv6-lite for lower-end computing devices.Source: K. YOLOv6 · p. 6

L. YOLOv7

  • YOLOv7 replaces YOLOv4’s CSPNet with ELAN, uses E-ELAN for large models, and applies RepConv to extend reparameterization to residual- and concatenation-based architectures.Its trainable auxiliary architectures can be removed or integrated during inference, and it supports anchor-based and anchor-free architectures.Source: L. YOLOv7 · p. 6
  • YOLOv7 uses consistent label assignment to align targets and feature-learning directions across branches, and coarse-to-fine assignment to refine predictions through auxiliary loss.The coarse-to-fine mechanism provides prediction refinement without changing the architecture.Source: L. YOLOv7 · p. 6
  • YOLOv7 [7] uses partial auxiliary loss so some features receive auxiliary updates while others focus on the target task.The developers found this design improved the main tasks.Source: L. YOLOv7 · p. 6
  • YOLOv7 [7] provides object-detection, instance-segmentation, and pose-estimation models and achieves real-time state-of-the-art in these tasks.Source: L. YOLOv7 · p. 6

M. DAMO-YOLO … P. Gold-YOLO

The sections describe distinct YOLO designs built around architecture search, feature integration, code and API optimization, multi-stage training, and global feature aggregation. They also summarize the mechanisms each system uses to support its design goals.

M. DAMO-YOLO

  • DAMO-YOLO [57] uses MAE-NAS to search CSPNet and ELAN, then combines ELAN with selected GFPN fusion layers in RepGFPN.The passage describes the retained fusion layers as a speed–accuracy trade-off.Source: M. DAMO-YOLO · p. 6
  • DAMO-YOLO [57] simplifies its decoupled head into a feature projection layer and uses AlignedOTA to address misalignment among classification, regression, and dynamic label assignment.Source: M. DAMO-YOLO · p. 6

N. YOLOv8

  • YOLOv8 reports about 30% improvement in training performance from underlying code optimization.Source: N. YOLOv8 · p. 6
  • YOLOv8 provides an API connecting detection models to downstream tasks including segmentation, pose estimation, and multiple object tracking.The passage also lists Segment Anything and instance segmentation.Source: N. YOLOv8 · p. 6

O. YOLO-NAS

  • YOLO-NAS uses its AutoNAC NAS to design a quantization-friendly architecture.Source: O. YOLO-NAS · p. 7
  • YOLO-NAS trains through pre-training on Object365 and COCO Pseudo-Labeled data, Knowledge Distillation, and Distribution Focal Loss.Source: O. YOLO-NAS · p. 7

P. Gold-YOLO

  • Gold-YOLO replaces PAN with Gather-and-Distribute modules that collect layer features, use transformers to integrate global features, and inject them back into low- and high-level layers.The architecture is shown in Figure 12.Source: P. Gold-YOLO · p. 7, Figure 12 · p. 7
  • Gold-YOLO uses masked image modeling for pre-training during the training process.Source: P. Gold-YOLO · p. 7

Q. YOLOv9 … B. Better

YOLOv9 and YOLOv10 introduce distinct design mechanisms, while the review describes the YOLO series’ broader influence through simpler detection and deployment and techniques that extend across models and tasks.

Q. YOLOv9

  • YOLOv9 introduces Programmable Gradient Information (PGI) and G-ELAN, using reversible architecture and multi-level information to retain task-relevant data and support accuracy, stability, and speed in low-parameter models.PGI shares retained information with the main branch as auxiliary information, aims to preserve information across feature levels, and is described as supporting robustness on small datasets and transfer, multi-task, and downstream learning.Source: Q. YOLOv9 · p. 7

R. YOLOv10

  • YOLOv10 [10] combines one-to-many and one-to-one matching to enable end-to-end detection without NMS, alongside rank-guided convolution choices and a CSPNet–Transformer self-attention module.Its architecture also adds a transformer-based module for global-feature extraction and uses dual label assignment with stop-gradient on the one-to-one branch.Source: R. YOLOv10 · p. 7, Figure 14 · p. 7

III. IMPACT OF YOLO SERIES

  • The review identifies relatively simple frameworks and ease of deployment as characteristic strengths of the YOLO series.It frames the series’ influence as extending to simpler detection approaches and practical computer vision systems.Source: §III · p. 8

A. Simpler

  • YOLO’s one-stage approach helped shift instance-level tasks from cascades toward end-to-end, top-down prediction, while simple modules and Darknet supported deployment across devices.Examples include pose estimation, facial landmarks, and multiobject tracking; YOLO designs also simplified specialized modules, and Darknet enabled training and inference without additional software packages.Source: A. Simpler · p. 8

B. Better

  • The series’ training techniques are designed to complement advanced methods, with compatibility considered since YOLOv2 and incompatible technologies analyzed in YOLOv3, YOLOv4, and PP-YOLO.The review presents this approach as guidance for subsequent developers rather than treating proposed methods as isolated from current state-of-the-art techniques.Source: B. Better · p. 8
  • YOLO scaling methods are integrated into frameworks, with guidelines from Scaled-YOLOv4 and YOLOv7 and EfficientDet-style scaling in YOLOv5 supporting stable performance across hyperparameter adjustments.The passage states that users can obtain stable and great performance as they adjust model-scaling hyperparameters.Source: B. Better · p. 8
  • YOLO methods generalize across tasks and architectures: examples include YOLOv2’s K-means anchors for pose estimation, SimOTA for dynamic label assignment, and CSPNet across CNNs, Transformers, and other architectures.The passage also describes YOLOv1’s prediction–ground-truth metric concept in soft-label generation, WorldTree group softmax for long-tailed learning, YOLOv7’s hybrid label assignment, and ELAN’s use across vision fields.Source: B. Better · p. 8

C. Faster · D. Stronger

The review links YOLO’s speed to architectures designed for hardware inference and describes its adaptability across domains, tasks, and downstream systems. Its examples span training without ImageNet pretraining, multi-task models, and integrated downstream applications.

C. Faster

  • YOLOv3’s designers found that lower computational load from combining 1×1 and 3×3 convolutions did not guarantee faster inference, so they designed DarkNet for real-time detection.Source: C. Faster · p. 8
  • Scaled-YOLOv4 considered high-speed architecture needs from edge to cloud, while YOLOv6 used RepVGG and DAMO-YOLO used NAS to search efficient CSPNet and ELAN architectures.Scaled-YOLOv4 developed Fully CSPOSANet and CSPDark-Net, drawing on work including ShuffleNetv2 and HarDNet.Source: C. Faster · pp. 8–9

D. Stronger

  • YOLO training can avoid ImageNet pretraining and adapt across domains and datasets; evolutionary hyperparameter search and anchor-free designs from YOLOX to PP-YOLOE reduce training dependence on hyperparameters.Source: D. Stronger · p. 9
  • YOLO-based models extend beyond detection to instance segmentation and multi-object tracking, while YOLOR jointly predicts image recognition, detection, and tracking.The review names YOLACT and JDE as examples; it contrasts YOLOR’s joint learning with YOLOv5’s separately trained recognition and detection models.Source: D. Stronger · p. 9
  • YOLOv7 was then state of the art in real-time detection, instance segmentation, and pose estimation; YOLOv8 added rotating-object detection and pose estimation, and YOLOv9 extended multi-task modeling to vision-language.Source: D. Stronger · p. 9
  • PP-YOLO’s design supports integration with downstream task models, and its series offers systems for tasks including face analysis, license plate recognition, tracking, traffic statistics, and behavior analysis.The review says these systems cover dozens of downstream tasks.Source: D. Stronger · p. 9

IV. YOLO FOR VARIOUS COMPUTER VISION TASKS … C. Automated Driving

The review surveys YOLO extensions for multiple object tracking, instance segmentation, and automated driving, describing design changes intended to achieve real-time performance.

IV. YOLO FOR VARIOUS COMPUTER VISION TASKS

  • Representative YOLO works in other vision fields introduce architectural or methodological changes to achieve real-time performance.The section frames its task-specific examples around those design changes.Source: §IV · p. 9

A. Multiple Object Tracking

  • Earlier tracking methods cropped detected objects and used additional feature networks, while ROLO used YOLO detections with LSTM for single-object tracking and proposed MOLO for multiple-object tracking.MOLO uses multiple LSTMs.Source: A. Multiple Object Tracking · p. 9
  • JDE outputs re-ID features alongside detections, but its dense multi-scale predictions are computationally costly and can confuse IDs; CSTrack outputs re-ID features at one scale after integrating multi-scale features.CSTrack combines JDE and FairMOT and is described as achieving more accurate multi-object tracking.Source: A. Multiple Object Tracking · p. 9

B. Instance Segmentation

  • YOLACT and YOLACT++ form instance masks from predicted coefficients and prototypes, reducing the operations required for instance segmentation.YOLACTEdge extends this approach to video and uses FeatFlowNet to reduce the number of backbone features extracted.Source: B. Instance Segmentation · p. 9
  • Insta-YOLO and Poly YOLO predict instance masks as polygons, representing masks in few dimensions while introducing some distortion.The passage presents polygon prediction as another way to reduce computation.Source: B. Instance Segmentation · p. 9

C. Automated Driving

  • YOLOP uses CSPNet and YOLOPv2 uses ELAN for object detection, enabling area detection and lane prediction in self-driving scenarios.The passage also identifies YOLOPv3, HybridNet, and YOLOPX as modified YOLO versions for self-driving tasks.Source: C. Automated Driving · p. 10

D. Human Pose Estimation … G. Face Detection

The review describes YOLO-based approaches across pose estimation, 3D detection, video perception, and face detection. These applications adapt YOLO to different prediction targets and input modalities.

D. Human Pose Estimation

  • KAPAO predicts human poses by separating them into human-pose objects and keypoint objects, then combining the predictions.Source: D. Human Pose Estimation · p. 10
  • YOLO-Pose predicts keypoint regressions relative to grid centers to perform human pose estimation.The passage notes that keypoints may fall outside grids, requiring additional decoder design.Source: D. Human Pose Estimation · p. 10

E. 3D Object Detection

F. Video Perception

G. Face Detection

  • YOLO-Face, YOLO-Face v2, and YOLO5Face are YOLO-based face detection models, which the review says perform quite well.Source: G. Face Detection · p. 10

H. Image Segmentation … V. CONCLUSIONS

The review describes YOLO’s use in segmentation, open-vocabulary detection, and combinations with other architectures, then summarizes its evolution, influence, and potential to guide future real-time computer vision work.

H. Image Segmentation

I. Open Vocabulary Object Detection

  • YOLO-World and Open-YOLO 3D [102] combine YOLO with CLIP for 2D and 3D open-vocabulary object detection, respectively.Source: §I · p. 10

J. Combine With Other Architecture

V. CONCLUSIONS

  • The review traces YOLO’s evolution and identifies the key contributions made at each stage from a modern object-detection perspective.Source: §V · p. 10
  • It analyzes YOLO’s influence on modern computer vision through ease of use, accuracy and speed improvements, and versatility across fields.Source: §V · p. 10
  • The review introduces YOLO-related models across fields to help readers understand possible tasks and future directions for real-time computer vision methods.Source: §V · p. 10

Related papers

Loading 2408.09332v1…