PaperlayerView exact revision on arXiv

Source-linked AI summary

Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanism

Chengcheng Wang, Wei He, Ying Nie, Jianyuan Guo, Chuanjian Liu, Kai Han, Yunhe Wang

arXiv:2309.11331v5cs.CVcs.AI

TL;DR

YOLO detectors still face information-fusion limitations despite FPN and PANet improvements. Gold-YOLO introduces a convolution- and attention-based Gather-and-Distribute mechanism with MAE pretraining, achieving strong accuracy and speed while retaining stated hardware and computational boundaries.

  • Problem

    Previous YOLO models still suffer from information-fusion problems despite FPN and PANet alleviating them.

  • Method

    Gold-YOLO globally gathers and distributes multi-level features through convolution- and attention-based branches, adds adjacent-layer fusion, and pretrains its backbone with MAE on ImageNet 1K.

  • Results

    Gold-YOLO-S with pretraining achieves 46.4% AP versus 45.0% AP for YOLOv6-3.0-S at similar speed.

  • Takeaways & Limitations

    The proposed mechanism improves information fusion and supports Gold-YOLO performance across varying object sizes and model scales.

  • Takeaways & Limitations

    Finer structural adjustments require substantial computation, and heavy attention usage may be less compatible with some earlier hardware.

Abstract

from arXiv · show

In the past years, YOLO-series models have emerged as the leading approaches in the area of real-time object detection. Many studies pushed up the baseline to a higher level by modifying the architecture, augmenting data and designing new losses. However, we find previous models still suffer from information fusion problem, although Feature Pyramid Network (FPN) and Path Aggregation Network (PANet) have alleviated this. Therefore, this study provides an advanced Gatherand-Distribute mechanism (GD) mechanism, which is realized with convolution and self-attention operations. This new designed model named as Gold-YOLO, which boosts the multi-scale feature fusion capabilities and achieves an ideal balance between latency and accuracy across all model scales. Additionally, we implement MAE-style pretraining in the YOLO-series for the first time, allowing YOLOseries models could be to benefit from unsupervised pretraining. Gold-YOLO-N attains an outstanding 39.9% AP on the COCO val2017 datasets and 1030 FPS on a T4 GPU, which outperforms the previous SOTA model YOLOv6-3.0-N with similar FPS by +2.4%. The PyTorch code is available at https://github.com/huawei-noah/Efficient-Computing/tree/master/Detection/Gold-YOLO, and the MindSpore code is available at https://gitee.com/mindspore/models/tree/master/research/cv/Gold_YOLO.

1 Introduction

Real-time object detectors must balance accuracy with low latency, but YOLO necks still face multi-scale information-fusion limitations. Gold-YOLO addresses this with Gather-and-Distribute fusion and MAE pretraining, improving accuracy at comparable speed.

  • Real-time detection prioritizes high performance and low latency for edge-device deployment.Object detection supports applications including security, autonomous driving, robot navigation, and medical diagnosis.
  • Transformer detectors model long-range dependencies but remain slower than CNN-based YOLO models in the speed-accuracy trade-off.
  • Gold-YOLO introduces convolution- and attention-based Gather-and-Distribute branches plus adjacent-layer fusion to improve multi-level information exchange without significantly increasing latency.The shallow and deep branches use convolution-based and attention-based blocks, respectively.
  • 46.4% AP: Gold-YOLO-S with ImageNet 1K MAE pretraining versus 45.0% AP for YOLOv6-3.0-S at similar speed.The authors report that pretraining improves convergence speed and accuracy.

2 Related works

YOLO evolved toward efficient single-stage, anchor-free detection, while transformer-based detectors improved global modeling at a speed cost. Multi-scale fusion methods expanded FPN, but excessive paths and indirect interactions still limited speed, cross-level exchange, and information retention.

  • 2.1 Real-time object detectors: YOLO models developed as single-stage detectors that predict objects of different sizes through multi-scale branches.
  • 2.1 Real-time object detectors: YOLOv6 introduced reparameterization, YOLOv7 proposed E-ELAN, and YOLOv8 integrated strengths of earlier models to reach the current YOLO-family SOTA.
  • 2.1 Real-time object detectors: DETR-like models capture long-range object dependencies, but CNN-based YOLO models remain faster for small-scale real-time detection.
  • 2.3 Multi-scale features for object detection: FPN, PANet, BiFPN, MLFPN, and bidirectional fusion methods progressively improve information exchange among feature levels.
  • 2.3 Multi-scale features for object detection: Excessive paths and indirect interactions in prior FPN-based structures limit speed, cross-level information exchange, and information retention.

3 Method

Gold-YOLO replaces recursive neighboring-level fusion with a Gather-and-Distribute mechanism that globally gathers multi-level features and redistributes them to enhance detection across object sizes. Its low- and high-stage branches combine feature alignment, information fusion, attention-based injection, and lightweight adjacent-layer fusion to balance accuracy and latency.

  • Motivation: Traditional FPN fusion exchanges information mainly between neighboring levels, so distant-level features are obtained recursively and may lose information during transmission.Intermediate layers select which information continues onward, weakening assistance between nonadjacent levels.
  • Gather-and-Distribute mechanism: GD abandons recursive transmission by gathering and fusing features from all levels, then distributing global information back to each level.The mechanism uses Feature Alignment, Information Fusion, and Information Injection modules and can integrate with existing backbone-neck-head structures.
  • Branch design: Low-GD and High-GD branches fuse features for large and small object sizes, respectively, using separate low-stage and high-stage feature groups.The neck receives backbone features B2, B3, B4, and B5; High-GD subsequently fuses P3, P4, and P5 generated by Low-GD.
  • Feature alignment: Feature Alignment uses average pooling to resize inputs to the smallest group resolution, reducing subsequent computational cost while retaining a controlled amount of low-level information.The low-stage target is RB4 to balance speed and accuracy, while the high-stage target is RP5 for transformer processing.
  • Information fusion: Information Fusion combines aligned features with RepBlocks or transformer blocks, then splits the fused representation into injection features for different levels.The high-stage fusion additionally reduces channels with a 1 × 1 convolution before splitting.
  • Information injection and LAF: Information Injection combines each level’s local feature with globally fused information through convolutional embeddings, attention operations, resizing, and a final RepBlock.Lightweight adjacent layer fusion merges neighboring features, increasing information-flow paths without significantly increasing latency.

4 Experiment

Experiments evaluate Gold-YOLO on COCO using deployment throughput and latency, compare model scales against YOLO-series detectors, and ablate GD, LAF, and cross-task extensions. Results show accuracy and speed gains, while GD is also tested beyond YOLOs.

  • Comparisons: Gold-YOLO-M achieves 1.1%, 4.2%, and 2.1% higher AP than YOLOv6-3.0-M, YOLOX-M, and PPYOLOE-M at comparable speed.It also exceeds YOLOv5-M and YOLOv8-M in AP and speed, and improves YOLOv7 speed by 98 FPS at the same AP.
  • Ablation study on GD structure: The GD ablation evaluates Low-GD, High-GD, and LAF using AP, parameter count, and T4 latency, with Low-GD targeting small and medium objects and High-GD targeting large objects.LAF supports both GD branches.
  • Ablation study on LAF: LAF experiments compare additive and concatenative designs, model scaling, and removal of LAF; the reported results support the existing LAF structure as optimal.The study reports results in Table 3 using TensorRT 7 on a T4 GPU.
  • Ablation study on other model and task: The GD mechanism is extended to instance segmentation, semantic segmentation, and object detection models beyond YOLO, with experiments reported for Mask R-CNN, PointRend, and EfficientDet.The paper states that these extensions obtain significant improvement.

5 Conclusion

The paper revisits information transmission constraints in FPN and presents Gold-YOLO as a detector using GD to improve information fusion and transmission efficiency. The authors report state-of-the-art object-detection results and broader applicability as a goal.

  • Conclusion: Gold-YOLO addresses analyzed FPN information-transmission constraints with a gather-and-distribute mechanism for more effective and efficient feature fusion.The conclusion describes GD as avoiding unnecessary information losses while improving detection capabilities.
  • Conclusion: The Gold-YOLO series achieves state-of-the-art results for object detection tasks.This is the paper’s reported overall conclusion.

A.1 More detailed accuracy and speed data for Gold-YOLO

This appendix reports Gold-YOLO deployment results on COCO val2017, using FP16 TensorRT measurements on a Tesla T4 and marking self-distillation, LAF, and MIM pretraining configurations.

  • Detailed accuracy and speed data: Gold-YOLO test performance is evaluated at 640x640 using FP16 TensorRT measurements of FPS and latency on a Tesla T4.The comparison uses the same environment with TensorRT 7.
  • Detailed accuracy and speed data: Table 7 reports COCO 2017 validation results for Gold-YOLO models with or without LAF and pretraining, using symbols to identify training configurations.The symbols distinguish self-distillation, absent LAF, and MIM pretraining.

A.2 MIM pre-training ablation experiment

This appendix examines Gold-YOLO-S on COCO 2017 validation across different MIM pretraining epochs, without self-distillation, with results summarized in Table 8.

  • MIM pre-training ablation experiment: Gold-YOLO-S is compared on COCO 2017 validation using different MIM pretraining epoch settings without self-distillation.The results are reported in Table 8.

B Comprehensive Latency and Throughput Benchmark

Tables 9 and 10 compare YOLO-series latency and throughput under separate TensorRT configurations on T4 and V100 GPUs.

  • T4 benchmark: Table 9 reports YOLO-series FPS and latency on a Tesla T4 using FP16 precision and TensorRT 8.2.The comparison uses a shared environment.
  • V100 benchmark: Table 10 reports YOLO-series FPS and latency on a Tesla V100 using FP16 precision and TensorRT 7.2.The comparison uses a shared environment.

C Broader impacts and limitations

The paper reports broader benefits and risks for YOLO, while noting deployment limitations from computational demands and attention-heavy operations.

  • Broader impacts: YOLO models can support healthcare and intelligent transportation applications, but military uses create risks the authors seek to prevent.The stated healthcare benefits include earlier diagnosis and lower initial-diagnosis costs; transportation uses include autonomous driving support.
  • Limitations: Finer structural adjustments may improve performance but require substantial computation, while heavy attention use may limit compatibility with earlier hardware.These are the paper’s stated limitations.
  • Visualization: Gold-YOLO’s neck CAM visualizations assign higher weights to detected target regions than the compared YOLO models.The visualizations cover YOLOv5, YOLOv6, YOLOv7, YOLOv8, and Gold-YOLO.
  • Visualization: The GD mechanism globally fuses high- and low-level information separately, strengthening information fusion without significantly increasing latency.The mechanism is described as improving performance across varying object sizes.
  • Architecture: Gold-YOLO separates feature alignment and feature fusion into distinct modules, with FAM dedicated to alignment and LAF merging multi-level features.This contrasts with M2Det and RHF-Net, whose alignment modules incorporate additional fusion information.

E.2 Simple calculation operation

The paper emphasizes a conceptual shift toward simple operators and reports that GD generalizes beyond YOLO to other models and tasks.

  • Design rationale: GD achieves state-of-the-art YOLO performance using simple, easily applicable operators rather than relying solely on specialized local structures.The authors frame this as evidence for the mechanism’s effectiveness.
  • Design rationale: The network uses simple, validated structures to reduce development and performance risks from unsupported deployment operators.The authors associate this choice with usability and portability.
  • Generalization: GD is presented as a general concept applicable beyond YOLO, with experiments extending it to other models.The extension is reported to produce significant improvement.
  • Generalization: Across different tasks and models, GD consistently produces performance improvements, indicating robust adaptability and generalization.This conclusion follows the paper’s experiments with extensions beyond YOLO.
Loading 2309.11331v5…