Source-linked AI summary
YOLOv12: Attention-Centric Real-Time Object Detectors
Yunjie Tian, Qixiang Ye, David Doermann
TL;DR
Attention mechanisms provide stronger modeling capabilities but have been difficult to use in YOLO because of slower computation and memory access. YOLOv12 builds an attention-centric YOLO framework with area attention, R-ELAN, and YOLO-specific architectural optimizations, achieving state-of-the-art latency-accuracy trade-offs and accuracy improvements over popular real-time detectors.
Problem
Attention mechanisms have stronger modeling capabilities than CNNs, but quadratic computation and inefficient memory access make them too slow for speed-critical YOLO systems.
Method
YOLOv12 combines area attention, R-ELAN, FlashAttention, and other architectural refinements to make attention efficient and compatible with the YOLO framework.
Results
YOLOv12 achieves state-of-the-art latency-accuracy trade-offs and outperforms popular real-time detectors across model scales, including YOLOv10, YOLOv11, and RT-DETR variants.
Takeaways & Limitations
YOLOv12 demonstrates that attention-centric design can be integrated into real-time YOLO detection while maintaining fast inference and improving detection accuracy.
Takeaways & Limitations
YOLOv12 requires FlashAttention, which currently supports only Turing, Ampere, Ada Lovelace, or Hopper GPUs.
Abstract
from arXiv · showhide
Enhancing the network architecture of the YOLO framework has been crucial for a long time, but has focused on CNN-based improvements despite the proven superiority of attention mechanisms in modeling capabilities. This is because attention-based models cannot match the speed of CNN-based models. This paper proposes an attention-centric YOLO framework, namely YOLOv12, that matches the speed of previous CNN-based ones while harnessing the performance benefits of attention mechanisms. YOLOv12 surpasses all popular real-time object detectors in accuracy with competitive speed. For example, YOLOv12-N achieves 40.6% mAP with an inference latency of 1.64 ms on a T4 GPU, outperforming advanced YOLOv10-N / YOLOv11-N by 2.1%/1.2% mAP with a comparable speed. This advantage extends to other model scales. YOLOv12 also surpasses end-to-end real-time detectors that improve DETR, such as RT-DETR / RT-DETRv2: YOLOv12-S beats RT-DETR-R18 / RT-DETRv2-R18 while running 42% faster, using only 36% of the computation and 45% of the parameters. More comparisons are shown in Figure 1.
1. Introduction
YOLOv12 addresses the speed gap that has limited attention mechanisms in YOLO by combining attention-centric architecture with efficiency-oriented design. Across model scales, it reports improved accuracy and competitive latency against popular real-time detectors.
- Motivation: Attention-centric vision transformers offer stronger modeling capabilities, but YOLO architecture research has largely remained focused on CNN-based designs.The paper identifies attention inefficiency as the reason attention mechanisms have not been widely adopted in speed-critical YOLO systems.
- Contributions: YOLOv12 introduces area attention, R-ELAN, and additional architectural changes to reduce attention cost and address optimization challenges.The framework also adopts FlashAttention, removes positional encoding, adjusts the MLP ratio, and modifies the stacked-block design for YOLO compatibility.
- Results: 40.6% mAP: YOLOv12-N outperforms YOLOv10-N by 2.1% mAP with faster inference and YOLOv11-N by 1.2% mAP with comparable speed.The reported advantage remains consistent across other model scales.
- Results: YOLOv12-S is 1.5%/0.1% mAP better than RT-DETR-R18 / RT-DETRv2-R18 while reporting 42%/42% faster latency, 36%/36% of their computations, and 45%/45% of their parameters.These comparisons position YOLOv12 as competitive with end-to-end real-time detectors that improve DETR.
- Conclusion: YOLOv12 claims state-of-the-art accuracy with fast inference without relying on additional techniques such as pretraining.The paper presents this as evidence that its attention-centric framework can challenge CNN-dominated YOLO designs.
2. Related Work
Related work situates YOLO as a leading real-time detection framework and reviews efforts to make attention-based vision models more efficient. Existing approaches reduce attention complexity or localize dependencies, but speed and modeling limitations remain relevant.
- Real-time Object Detectors: The YOLO series established a leading balance between latency and accuracy, with successive versions advancing model components and architecture.Examples include CSPNet, augmentation, multiple feature scales, and later backbone and neck modules.
- Area Attention: Area attention is presented as a simple local-attention alternative that divides feature maps into equal vertical or horizontal areas while retaining a large receptive field.Its default partition count is 4, avoiding complex partitioning operations.
- Efficient Attention: Local-global methods, token migration, token gathering, linear attention, and Mamba-based models have been explored to improve attention efficiency.These approaches target reduced reliance on global self-attention, lower complexity, or faster downstream inference.
- Efficient Attention: Mamba-based vision models still fall short of real-time speeds, while FlashAttention identifies high-bandwidth-memory bottlenecks in attention computation.The paper uses these lines of work to motivate its own efficiency-oriented attention design.
3. Approach
YOLOv12’s approach combines attention-specific efficiency mechanisms with residual feature aggregation and YOLO-compatible architectural refinements. The design targets attention’s computational and memory costs while improving optimization and retaining real-time suitability.
- Efficiency Analysis: Attention is slower than CNNs because self-attention has quadratic complexity and inefficient memory access patterns.These constraints are especially problematic for real-time or resource-constrained object detection.
- Efficiency Analysis: FlashAttention addresses attention’s memory-access inefficiency, and YOLOv12 adopts it during model design.The paper describes intermediate attention maps moving between GPU SRAM and high-bandwidth memory as a major source of overhead.
- Area Attention: Linear attention reduces vanilla attention complexity from 2n^2hd to 2nhd^2, but can degrade global dependencies and offers limited speed advantages at 640 × 640 YOLO resolution.The paper also cites instability, distribution sensitivity, and a low-rank bottleneck as limitations.
- Area Attention: Area attention divides feature maps into segments using a simple reshape, reducing attention cost while preserving a large receptive field.With default l = 4, the receptive field becomes one quarter of the original, while the paper reports only slight performance impact and improved speed.
- Residual Efficient Layer Aggregation Networks: R-ELAN adds a residual shortcut with scaling and redesigns feature aggregation to address instability and convergence problems in attention-based large models.The redesigned bottleneck preserves feature integration while reducing computational cost and parameter or memory usage.
- Architectural Optimizations: YOLOv12 retains YOLO’s hierarchical design, reduces the final backbone stage to one R-ELAN block, and modifies attention configurations for efficiency.Changes include reducing the MLP ratio, using convolution with batch normalization, removing positional encoding, and adding a 7 × 7 separable convolution.
4. Experiment
Experiments on standard benchmarks evaluate YOLOv12 across five scales, showing improved accuracy-efficiency trade-offs, faster attention-based inference, and diagnostic support for its architectural choices.
- Experimental setup: YOLOv12 is evaluated through benchmark comparisons, ablations, speed tests, and diagnostic studies across five model scales.The experiments use YOLOv12-N, S, M, L, and X, with comparisons spanning accuracy, FLOPs, parameters, and latency.
- Comparison with state-of-the-arts: 40.6% mAP and 1.64 ms/image characterize YOLOv12-N, which exceeds YOLOv6-3.0-N, YOLOv8-N, YOLOv10-N, and YOLOv11 in mAP by 3.6%, 3.3%, 2.1%, and 1.2%.The gains are achieved with similar or fewer computations and parameters.
- Comparison with state-of-the-arts: 48.0 mAP with 2.61 ms/image latency, 21.4G FLOPs, and 9.3M parameters describes YOLOv12-S, which surpasses several YOLO baselines and improves efficiency against RT-DETR variants.The cited comparison reports better inference speed, computational cost, and parameter count than RT-DETR-R18 and RT-DETRv2-R18.
- Comparison with state-of-the-arts: YOLOv12-L and YOLOv12-X outperform corresponding YOLO and RT-DETR baselines with comparable or reduced resources.YOLOv12-L uses 31.4G fewer FLOPs than YOLOv10-L and beats RT-DETR-R50 variants with 34.6% fewer FLOPs and 37.1% fewer parameters; YOLOv12-X beats RT-DETR-R101 variants with 23.4% fewer FLOPs and 22.2% fewer parameters.
- Ablation studies: R-ELAN residual connections are unnecessary for YOLOv12-N but essential for stable training in YOLOv12-L/X, with YOLOv12-X requiring a 0.01 scaling factor.The feature integration method also reduces FLOPs and parameters while maintaining model performance.
- Ablation studies: Area attention reduces inference time across models and hardware; with FP32 on RTX 3080, YOLOv12-N gains a 0.7 ms reduction.The speedup is reported consistently across GPU and CPU configurations without FlashAttention in the ablation experiment.
- Speed comparison: Across RTX 3080, RTX A5000, and RTX A6000, YOLOv12 is faster than YOLOv9 and remains on par with YOLOv10 and YOLOv11.On RTX 3080, YOLOv12-N reaches 1.7 ms FP32 and 1.1 ms FP16, compared with YOLOv9’s 2.4 ms and 1.5 ms.
- Diagnosis and visualization: Diagnostic studies find batch normalization preferable to layer normalization with convolution, hierarchical design necessary, and no positional embedding best-performing.A 7 × 7 Position Perceiver kernel is selected because larger kernels significantly slow inference, while plain vision-transformer design reaches only 38.3% mAP.
5. Conclusion
YOLOv12 integrates attention into YOLO through area attention, R-ELAN, and architectural refinements, achieving a state-of-the-art latency-accuracy trade-off with improved accuracy and efficiency.
- Conclusion: YOLOv12 adopts an attention-centric design for real-time detection while achieving a state-of-the-art latency-accuracy trade-off.The framework combines attention with YOLO’s real-time requirements rather than relying exclusively on CNN-based designs.
- Conclusion: Area attention reduces attention computational complexity, while R-ELAN improves feature aggregation and supports efficient inference.Architectural optimizations refine vanilla attention to align with real-time constraints.
- Conclusion: YOLOv12 achieves significant improvements in both accuracy and efficiency, supported by comprehensive ablation studies.The conclusion presents these results as advancing attention integration in real-time YOLO systems.
6. Limitations
YOLOv12’s use of FlashAttention limits supported hardware to newer NVIDIA GPU architectures.
- FlashAttention currently supports Turing, Ampere, Ada Lovelace, and Hopper GPUs.Examples include T4, RTX20/30/40 series, A100, and H100 GPUs.
- YOLOv12 therefore depends on GPUs from these supported architecture generations for FlashAttention.
- Older or unsupported GPU architectures fall outside the hardware support listed for YOLOv12’s FlashAttention requirement.
7. More Details
The reported training setup uses SGD for 600 epochs with specified optimization schedules and several data augmentations; the supplied metric passage is fragmentary.
- All YOLOv12 models are trained with SGD for 600 epochs by default.
- SGD momentum and weight decay are set to 0.937 and 5 × 10−4, respectively.
- The learning rate starts at 1×10−2 and decays linearly to 1 × 10−4 during training.
- Training applies Mosaic, Mixup, and copy-paste data augmentations.