Source-linked AI summary
DAMO-YOLO : A Report on Real-Time Object Detection Design
Xianzhe Xu, Yiqi Jiang, Weihua Chen, Yilun Huang, Yuan Zhang, Xiuyu Sun
TL;DR
Real-time object detection still requires better accuracy–latency trade-offs and more effective network and training designs. DAMO-YOLO combines NAS-based backbones, RepGFPN, a lightweight head, AlignedOTA, and distillation, and reports higher performance than other YOLO-series methods across general and lightweight deployment scenarios. The paper provides model families at multiple scales for these scenarios.
Problem
Real-time object detection seeks high performance under latency constraints, while existing detector backbones and label-assignment methods leave room for improvement.
Method
DAMO-YOLO combines MAE-NAS backbones, RepGFPN, ZeroHead, AlignedOTA label assignment, and distillation enhancement.
Results
DAMO-YOLO outperforms other YOLO-series detectors on COCO in both general and lightweight categories.
Takeaways & Limitations
The paper presents general and lightweight DAMO-YOLO model families at multiple scales for different deployment scenarios.
Abstract
from arXiv · showhide
In this report, we present a fast and accurate object detection method dubbed DAMO-YOLO, which achieves higher performance than the state-of-the-art YOLO series. DAMO-YOLO is extended from YOLO with some new technologies, including Neural Architecture Search (NAS), efficient Reparameterized Generalized-FPN (RepGFPN), a lightweight head with AlignedOTA label assignment, and distillation enhancement. In particular, we use MAE-NAS, a method guided by the principle of maximum entropy, to search our detection backbone under the constraints of low latency and high performance, producing ResNet/CSP-like structures with spatial pyramid pooling and focus modules. In the design of necks and heads, we follow the rule of ``large neck, small head''.We import Generalized-FPN with accelerated queen-fusion to build the detector neck and upgrade its CSPNet with efficient layer aggregation networks (ELAN) and reparameterization. Then we investigate how detector head size affects detection performance and find that a heavy neck with only one task projection layer would yield better results.In addition, AlignedOTA is proposed to solve the misalignment problem in label assignment. And a distillation schema is introduced to improve performance to a higher level. Based on these new techs, we build a suite of models at various scales to meet the needs of different scenarios. For general industry requirements, we propose DAMO-YOLO-T/S/M/L. They can achieve 43.6/47.7/50.2/51.9 mAPs on COCO with the latency of 2.78/3.83/5.62/7.95 ms on T4 GPUs respectively. Additionally, for edge devices with limited computing power, we have also proposed DAMO-YOLO-Ns/Nm/Nl lightweight models. They can achieve 32.3/38.2/40.5 mAPs on COCO with the latency of 4.08/5.05/6.69 ms on X86-CPU. Our proposed general and lightweight models have outperformed other YOLO series models in their respective application scenarios.
1. Introduction
DAMO-YOLO extends YOLO with NAS-based backbones, an efficient RepGFPN neck, a lightweight head, AlignedOTA, and distillation to target real-time detection. The resulting general and lightweight model families are reported to outperform other YOLO-series detectors on COCO across their respective deployment scenarios.
- Motivation and design: MAE-NAS searches detection-friendly backbones under real-time constraints, producing ResNet-like or CSP-like structures with spatial pyramid pooling and focus modules.The method is heuristic and training-free, without supernet dependence.
- Motivation and design: RepGFPN builds on GFPN with accelerated queen-fusion, efficient layer aggregation networks, and re-parameterization.The neck is designed to improve fusion of high-level semantic and low-level spatial features.
- Motivation and design: Experiments support a “large neck, small head” design, retaining only a task projection layer in ZeroHead and moving saved computation into the neck.The paper reports that this configuration yields better performance while balancing latency and accuracy.
- Motivation and design: AlignedOTA addresses classification–regression misalignment in dynamic label assignment by balancing their importance.The paper presents it as a partial solution to a problem that remains in existing dynamic assignment methods.
- Contributions: DAMO-YOLO extends YOLO with MAE-NAS backbones, RepGFPN, ZeroHead, AlignedOTA, and distillation enhancement.These components address backbone design, feature fusion, head size, label assignment, and model improvement.
- Results and deployment: DAMO-YOLO reports state-of-the-art performance on COCO for both general and lightweight categories and provides models at multiple scales for different deployments.The general and lightweight families are intended for industry and limited-computing scenarios, respectively.
2. DAMO-YOLO
DAMO-YOLO combines latency-aware NAS, an efficient RepGFPN neck, a lightweight ZeroHead, aligned label assignment, and feature distillation into a real-time detection framework. Its modules are selected and evaluated across model scales and deployment constraints.
- Design principle: DAMO-YOLO prioritizes the latency-mAP curve rather than FLOPs-mAP because FLOPs do not necessarily predict real-world latency.The design process targets industrial deployment performance.
- 2.1. MAE-NAS Backbone: MAE-NAS searches networks under latency budgets using an information-theoretic proxy, with Mob-, Res-, and CSP-blocks supporting different model scales.The proxy ranks initialized networks without training, reducing the search process to a few hours.
- 2.1. MAE-NAS Backbone: MAE-CSP outperforms manually designed CSP-Darknet in both speed and accuracy, while Res-blocks suit smaller models and CSP-blocks suit larger, deeper networks.The final setting uses MAE-Res for Tiny and Small models and MAE-CSP for Medium and Large models.
- 2.2. Efficient RepGFPN: Efficient RepGFPN extends GFPN with accelerated queen-fusion, ELAN, reparameterization, and scale-specific channel dimensions to improve real-time feature fusion.The neck refines and fuses high-level semantic and low-level spatial features, while addressing GFPN efficiency limitations.
- 2.3. ZeroHead and AlignOTA: AlignOTA addresses classification-regression misalignment by incorporating focal loss into classification cost and IoU as a soft label, then selects aligned samples globally.The authors report that AlignOTA outperforms other label assignment methods.
- 2.4. Distillation Enhancement: Feature-based distillation transfers recognition and localization information through intermediate feature maps, with CWD selected as the best fit among tested methods.The authors also find that a lower distillation weight and a shallower detector head improve distillation behavior.
- 2.5. General Class DAMO-YOLO Model: Training DAMO-YOLO-S on COCO, Objects365, and OpenImage improves COCO mAP by 1.1% over its baseline and reaches 26.6% mAP on VisDrone after fine-tuning.The multi-dataset model reduces overlapping labels to a unified label space and uses polynomial smoothing with weighted sampling.
3. Implementation Details
DAMO-YOLO uses image-level Mosaic and Mixup augmentation together with box-level SADA augmentation to enhance data diversity and robustness.
- 300 epochs of training use SGD with grouped weight decay, EMA, and cosine learning-rate decay.The initial learning rate is 0.4 with batch size 256; weight decay and momentum are 5e-4 and 0.9.
- Mosaic and Mixup provide image-level augmentation, while SADA supplies box-level augmentation after them.The authors apply box-level augmentation to pursue more robust augmentation.
4. Comparison with the SOTA
DAMO-YOLO releases general and lightweight models for both standard and resource-limited deployments, with reported advantages in accuracy and speed over competing detectors.
- DAMO-YOLO provides general and lightweight model series for general scenarios and resource-limited edge scenarios.The release includes models at multiple scales and supports varied deployment needs.
- DAMO-YOLO general models outperform rival state-of-the-art models in both accuracy and speed on T4-GPU with TensorRT-FP16.The reported evaluation also finds significant accuracy improvements from the proposed distillation technique.
- DAMO-YOLO lightweight models surpass competitors by a considerable margin in both speed and accuracy on Intel-8163 CPU using Openvino.
5. Conclusion
DAMO-YOLO is an object detection method whose reported advantages over YOLO-series methods come from several architectural, label-assignment, and distillation technologies.
- DAMO-YOLO combines an MAE-NAS backbone, efficient RepGFPN neck, ZeroHead, AlignedOTA label assignment, and distillation enhancement.
- The method is reported to achieve superior performance to other YOLO-series methods.