PaperlayerView exact revision on arXiv

Source-linked AI summary

YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information

Chien-Yao Wang, I-Hau Yeh, Hong-Yuan Mark Liao

arXiv:2402.13616v2cs.CV

TL;DR

Deep networks can lose information during feedforward processing, biasing gradient flow and limiting training. YOLOv9 addresses this with Programmable Gradient Information and GELAN, achieving top performance across MS COCO comparisons while using a lightweight convolutional architecture.

  • Problem

    Deep networks can lose substantial information during feedforward processing, producing biased gradients and weakening target-task learning.

  • Method

    The paper introduces PGI with an auxiliary reversible branch and designs GELAN to improve parameter use across computational and inference settings.

  • Results

    YOLOv9 achieved top performance in all MS COCO comparisons, while GELAN used conventional convolutions with higher parameter usage than depth-wise designs.

  • Takeaways & Limitations

    PGI and GELAN support lightweight and deep object detectors across different computational blocks and inference-device requirements.

  • Takeaways & Limitations

    A reversible architecture with added deep-to-shallow connections increased inference time by 20%, while repeated input additions more than doubled it.

Abstract

from arXiv · show

Today's deep learning methods focus on how to design the most appropriate objective functions so that the prediction results of the model can be closest to the ground truth. Meanwhile, an appropriate architecture that can facilitate acquisition of enough information for prediction has to be designed. Existing methods ignore a fact that when input data undergoes layer-by-layer feature extraction and spatial transformation, large amount of information will be lost. This paper will delve into the important issues of data loss when data is transmitted through deep networks, namely information bottleneck and reversible functions. We proposed the concept of programmable gradient information (PGI) to cope with the various changes required by deep networks to achieve multiple objectives. PGI can provide complete input information for the target task to calculate objective function, so that reliable gradient information can be obtained to update network weights. In addition, a new lightweight network architecture -- Generalized Efficient Layer Aggregation Network (GELAN), based on gradient path planning is designed. GELAN's architecture confirms that PGI has gained superior results on lightweight models. We verified the proposed GELAN and PGI on MS COCO dataset based object detection. The results show that GELAN only uses conventional convolution operators to achieve better parameter utilization than the state-of-the-art methods developed based on depth-wise convolution. PGI can be used for variety of models from lightweight to large. It can be used to obtain complete information, so that train-from-scratch models can achieve better results than state-of-the-art models pre-trained using large datasets, the comparison results are shown in Figure 1. The source codes are at: https://github.com/WongKinYiu/yolov9.

1. Introduction

The paper identifies information loss in deep networks as a central limitation and introduces PGI and GELAN to preserve reliable gradient information while enabling lightweight, efficient object detection.

  • Motivation: Deep networks lose input information during feedforward, a phenomenon termed the information bottleneck; reversible architectures and masked modeling are existing mitigation strategies.The paper frames this loss as affecting the information available for objective-function and gradient computation.
  • PGI: PGI generates reliable gradients through an auxiliary reversible branch that preserves key deep-feature characteristics while avoiding semantic loss from traditional deep supervision.The branch adds no additional cost and allows loss functions to be selected for the target task.
  • PGI: PGI applies across neural networks of various sizes, overcoming deep supervision’s restriction to extremely deep architectures and enabling lightweight models.The paper explicitly positions PGI as more general than deep supervision.
  • GELAN: GELAN is designed around parameter count, computational complexity, accuracy, and inference speed, while allowing computational blocks to be selected for different inference devices.Its contribution claims higher parameter usage than depth-wise convolution designs using only conventional convolution.
  • YOLOv9: Combining PGI and GELAN yields YOLOv9, whose MS COCO object-detection performance greatly surpasses existing real-time object detectors in all aspects.The introduction presents YOLOv9 as a new YOLO-series detection system built from the two proposed components.

2. Related work

Related real-time object detectors predominantly build on YOLO-family designs with CSPNet or ELAN units, PAN/FPN feature integration, and YOLOv3- or FCOS-based heads. Related information-preservation and supervision strategies include reversible architectures, deep supervision, and intermediate-feature guidance.

  • Real-time object detectors: Mainstream real-time object detectors are dominated by YOLO variants using CSPNet or ELAN computing units, often with improved PAN/FPN integration and YOLOv3 or FCOS prediction heads.RT DETR is also identified as a recent real-time detector founded on DETR.
  • Reversible architectures: Reversible architectures preserve complete original information in each layer’s output feature map by maintaining reversible conversion characteristics.RevCol extends traditional reversible units to multiple levels, expanding the semantic levels expressed by different layer units.
  • Auxiliary supervision: Deep supervision trains networks by inserting additional prediction layers into intermediate layers, while other auxiliary methods guide intermediate feature maps using relevant meta information.Multi-layer decoders in transformer-based methods are described as a particularly common form of deep supervision.

3. Problem Statement

Modern deep networks have addressed vanishing and saturation with normalization and activation functions, yet convergence can remain slow or poor because deep transformations lose information needed by the initial gradient. This information bottleneck is especially damaging in lightweight, under-parameterized models, motivating architectures that preserve task-relevant information and support reliable gradients.

  • Despite advances against gradient vanishing and saturation, modern deep networks can still converge slowly or achieve poor convergence results.
  • Information bottlenecks cause deep transformations to lose information needed by the initial gradient, which undermines subsequent loss-based weight updates.As network depth increases, original data is more likely to be lost during consecutive layer transformations.
  • Increasing model size can preserve enough information for target mapping, helping explain why width is often more important than depth in modern models.
  • Reversible transformations preserve information and can provide more reliable gradients, but directly passing inputs through layers can limit the network’s ability to learn simple mappings for difficult problems.PreAct ResNet demonstrates that explicit input reuse can support convergence beyond a thousand layers while compromising this mapping objective.
  • Lightweight models are under-parameterized for raw data and therefore risk losing task-critical information I(Y, X) during feedforward processing, even when overall information loss is small.The paper identifies preserving information that maps data X to target Y as the lightweight-model goal.

4. Methodology

The methodology introduces Programmable Gradient Information (PGI), an auxiliary supervision framework that provides reliable gradients without adding inference cost, and GELAN, a lightweight architecture designed through gradient path planning. PGI combines a main branch, auxiliary reversible branch, and multi-level auxiliary information, while GELAN generalizes ELAN to support varied computational blocks.

  • Programmable Gradient Information: PGI comprises a main branch, auxiliary reversible branch, and multi-level auxiliary information, with inference using only the main branch and requiring no additional inference cost.The auxiliary components support training and gradient computation rather than inference.
  • Programmable Gradient Information: The auxiliary reversible branch supplies target-relevant information for reliable gradients, reducing false correlations caused by incomplete feedforward features.It maintains complete information through reversible architecture while guiding parameters in the main branch.
  • Programmable Gradient Information: 20% is the inference-time increase caused by adding deep-to-shallow connections, while repeatedly injecting input into high-resolution layers exceeds twice the original time.PGI avoids these costs by using the reversible architecture only as an auxiliary branch during training.
  • Multi-Level Auxiliary Information: Multi-level auxiliary information aggregates gradients from prediction heads across feature-pyramid levels and passes them to the main branch to capture information for different object sizes.An integration network combines returned gradients from the auxiliary supervision hierarchy.
  • GELAN: GELAN combines CSPNet and ELAN using gradient path planning to jointly target lightweight computation, inference speed, and accuracy.It generalizes ELAN beyond stacked convolutional layers to support arbitrary computational blocks.

5. Experiments

Experiments on MS COCO show that YOLOv9 improves real-time detection across model scales while using train-from-scratch training. Ablations and visualizations attribute these gains to GELAN’s architecture and PGI’s preservation of reliable information and gradients.

  • Overall comparison: YOLOv9 improves across lightweight, medium, general, and large real-time detectors while using fewer parameters and less computation than comparable YOLO MS models.The comparisons use train-from-scratch real-time object detectors on MS COCO 2017.
  • Overall comparison: YOLOv9’s conventional convolutions achieve better parameter utilization than YOLO MS’s depth-wise convolutions, including against ImageNet-pretrained detectors.The comparison includes RT DETR, RTMDet, and PP-YOLOE, among other ImageNet-pretrained models.
  • Overall comparison: YOLOv9 is far superior to train-from-scratch methods in computational complexity and remains competitive with depth-wise-convolution and ImageNet-pretrained models.The strongest existing computation-efficient models identified are YOLO MS, PP YOLOE, and RT DETR, from smallest to largest.
  • GELAN ablations: GELAN ablations show that Res, Dark, and CSP computational blocks preserve good performance, with CSP blocks performing particularly well; beyond depth 2, depth changes scale parameters, computation, and accuracy linearly.The experiments also find a significant accuracy improvement when ELAN depth increases from 1 to 2.
  • PGI ablations: Across model sizes, deep supervision can hurt shallow models or behave unstably in general models, whereas PGI effectively addresses these problems.Deep supervision provides gains mainly for extremely deep models, while PGI is evaluated across multiple model sizes.
  • Information and gradient visualization: Feature-map visualizations show that GELAN retains more complete and discriminative information at depth, while PGI reduces divergence and background responses by providing better gradients.Adding PGI’s reversible branch helps the main branch focus on target objects and retain important features during training.

6. Conclusions

The paper proposes PGI to address information bottlenecks and the limitations of deep supervision for lightweight networks, alongside GELAN, an efficient architecture with stable, adaptable detection performance.

  • PGI is proposed to address the information bottleneck problem and the unsuitability of deep supervision for lightweight neural networks.
  • GELAN is designed as a highly efficient and lightweight neural network.
  • GELAN delivers strong and stable object-detection performance across different computational blocks and depth settings.
  • GELAN can be widely expanded into models suitable for various inference devices.

Appendix · A. Implementation Details

YOLOv9’s implementation follows specified training settings and a YOLOv7 AF-derived network configuration. The appendix also provides hyperparameter, topology, and training-setting comparison tables.

  • A. Implementation Details: YOLOv9 follows YOLOv7 AF training settings, using SGD for 500 epochs with a 3-epoch warm-up that updates only biases.The initial learning rate is 0.01, then linearly decays to 0.0001.
  • A. Implementation Details: Mosaic augmentation is disabled during the final 15 training epochs.This is part of the data augmentation schedule specified for YOLOv9.
  • A. Implementation Details: YOLOv9’s network topology completely follows YOLOv7 AF while replacing ELAN with the proposed CSP-ELAN block.The configuration is documented in Table 2.
  • A. Implementation Details: CSP-ELAN depth parameters are reported separately as ELAN depth and CSP depth.These parameter names are defined for the network configurations in Table 2.
  • A. Implementation Details: CSP-ELAN filter parameters are represented as ELAN output filter, CSP output filter, and CSP inside filter.The terminology corresponds to the network configuration table.
  • A. Implementation Details: The downsampling design simplifies CSP-DOWN to DOWN, which contains a pooling layer with size 2.The supplied passage truncates the remaining description of the DOWN module.

B. More Comparison

YOLOv9 is compared with real-time object detectors under train-from-scratch, ImageNet-pretrained, knowledge-distillation, and complex training settings. It outperforms the compared methods while achieving Pareto-optimal performance across model sizes and computational scales.

  • B. More Comparison: The comparison covers train-from-scratch, ImageNet-pretrained, knowledge-distillation, and complex training settings.Complex settings combine procedures such as ImageNet pretraining, knowledge distillation, DAMO-YOLO, and additional large-object-detection pretraining.
  • B. More Comparison: YOLOv9 performs better than all other compared methods, including PPYOLOE+-X trained with ImageNet and Objects365.Compared with PPYOLOE+-X, YOLOv9 reduces parameters by 55% and computation by 11%.
  • B. More Comparison: YOLOv9 is Pareto optimal across models of different parameter sizes, with no other method Pareto optimal above 20M parameters.The results indicate strong parameter usage efficiency.
  • B. More Comparison: YOLOv9 is Pareto optimal across models with different computational scales.Above 60 GFLOPs, only ELAN-based DAMO-YOLO and DETR-based RT DETR can rival YOLOv9, indicating the strongest computation–accuracy trade-off.
Loading 2402.13616v2…