PaperlayerView exact revision on arXiv

Source-linked AI summary

End-to-End Object Detection with Transformers

Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, Sergey Zagoruyko

arXiv:2005.12872v3cs.CV

TL;DR

Object detection typically relies on proposals, anchors, and postprocessing rather than direct set prediction. DETR replaces these components with bipartite matching and transformers, achieving comparable COCO performance to optimized Faster R-CNN and competitive panoptic segmentation results.

  • Problem

    Existing detectors solve set prediction indirectly through proposals, anchors, and postprocessing, motivating a direct approach that simplifies these pipelines.

  • Method

    DETR predicts all objects in parallel using transformer encoder-decoder architecture, learned object queries, and a bipartite-matching set loss.

  • Results

    DETR achieves comparable COCO detection performance to optimized Faster R-CNN and outperforms competitive baselines on panoptic segmentation.

  • Takeaways & Limitations

    DETR provides a straightforward, flexible detection design that extends to panoptic segmentation with competitive results.

  • Takeaways & Limitations

    DETR introduces training and optimization challenges and performs worse on small objects, leaving these issues for future work.

Abstract

from arXiv · show

We present a new method that views object detection as a direct set prediction problem. Our approach streamlines the detection pipeline, effectively removing the need for many hand-designed components like a non-maximum suppression procedure or anchor generation that explicitly encode our prior knowledge about the task. The main ingredients of the new framework, called DEtection TRansformer or DETR, are a set-based global loss that forces unique predictions via bipartite matching, and a transformer encoder-decoder architecture. Given a fixed small set of learned object queries, DETR reasons about the relations of the objects and the global image context to directly output the final set of predictions in parallel. The new model is conceptually simple and does not require a specialized library, unlike many other modern detectors. DETR demonstrates accuracy and run-time performance on par with the well-established and highly-optimized Faster RCNN baseline on the challenging COCO object detection dataset. Moreover, DETR can be easily generalized to produce panoptic segmentation in a unified manner. We show that it significantly outperforms competitive baselines. Training code and pretrained models are available at https://github.com/facebookresearch/detr.

1 Introduction

DETR reframes object detection as direct set prediction, combining bipartite matching with a transformer encoder-decoder to emit unique detections in parallel. It removes several hand-designed components, matches Faster R-CNN overall on COCO, improves large-object performance, and extends to panoptic segmentation.

  • Method: The transformer architecture models pairwise interactions among elements, helping DETR remove duplicate predictions without indirect proposal, anchor, or window-center formulations.This design drops spatial anchors and non-maximal suppression while avoiding customized layers.
  • Method: DETR predicts the final set of objects directly and trains end-to-end with a bipartite matching loss that uniquely assigns predictions to ground-truth objects.The matching loss is permutation-invariant, enabling parallel prediction rather than autoregressive decoding.
  • Results: DETR achieves comparable performance to the highly optimized Faster R-CNN baseline on COCO, with significantly better performance on large objects.The paper attributes the large-object result likely to the transformer's non-local computations.
  • Extensions: A simple segmentation head trained on a pretrained DETR outperforms competitive baselines on the challenging Panoptic Segmentation task.This demonstrates that the DETR design extends beyond object detection to pixel-level recognition.

2 Related work

Prior work addressed set prediction, transformers, parallel decoding, and object detection, but existing detectors relied on duplicate-suppression or hand-crafted initial guesses. DETR builds on end-to-end set prediction while combining bipartite matching, transformer encoder-decoder architectures, and parallel decoding.

  • Set prediction: Set prediction in detection must avoid near-duplicate outputs, a problem commonly handled by non-maximal-suppression postprocessing.One-vs-rest multilabel classification does not directly address the structure among near-identical detection boxes.
  • Object detection: Modern two-stage detectors predict relative to proposals, while single-stage detectors use anchors or grids of possible object centers.Prior work found final performance depends heavily on how these initial guesses are set.
  • Bipartite matching: Earlier detectors used bipartite matching losses but still modeled prediction relations with convolutional or fully connected layers and could benefit from hand-designed NMS.More recent detectors combined non-unique assignment rules with NMS.
  • Relation modeling: Learnable NMS and relation networks modeled relations between predictions with attention and direct set losses, but relied on hand-crafted context features such as proposal coordinates.These methods therefore retained prior knowledge that DETR seeks to reduce.
  • End-to-end set prediction: Closest prior approaches used bipartite-matching losses and CNN-based encoder-decoder architectures for direct bounding-box set prediction, but were evaluated only on small datasets and used autoregressive RNNs.The passage distinguishes these methods from the present combination with transformers and parallel decoding.

3 The DETR model

DETR formulates detection as fixed-size direct set prediction, using Hungarian bipartite matching to enforce one-to-one assignments and object-specific losses. Its simple architecture combines a CNN backbone, transformer encoder-decoder, and prediction FFN, with decoder outputs computed in parallel.

  • Set prediction and loss: DETR produces a fixed-size set of N predictions in one decoder pass, with N larger than the typical number of image objects.Ground-truth sets are padded with ∅ when necessary.
  • Set prediction and loss: The loss first finds an optimal bipartite matching between predictions and ground truth, then applies object-specific bounding-box losses.The optimal assignment is computed efficiently with the Hungarian algorithm.
  • Set prediction and loss: Matching costs combine class prediction with predicted–ground-truth box similarity, enabling one-to-one matching without duplicate predictions.This replaces heuristic proposal- or anchor-to-ground-truth assignment rules used in modern detectors.
  • Architecture: The architecture contains a CNN backbone, transformer encoder-decoder, and feed-forward prediction network for final detections.The backbone produces a compact feature representation that is flattened and supplied to the transformer with positional information.
  • Architecture: The decoder transforms N embeddings using self- and encoder-decoder attention while decoding all N objects in parallel at each layer.This differs from the original transformer’s autoregressive, one-element-at-a-time output sequence.
  • Prediction and training: A 3-layer ReLU perceptron predicts normalized box coordinates and dimensions, while a softmax projection predicts class labels including the ∅ class.Auxiliary prediction FFNs and Hungarian losses after decoder layers help produce the correct number of objects per class during training.

4 Experiments

Experiments on COCO show that DETR achieves competitive object-detection results against Faster R-CNN, while ablations identify key transformer components and extensions demonstrate panoptic-segmentation performance. DETR also exhibits query specialization without strong class specialization.

  • Object detection: DETR achieves comparable results to heavily tuned Faster R-CNN baselines on COCO, with lower APS but greatly improved APL.The comparison uses ResNet-50 and ResNet-101 backbones on the COCO validation set.
  • Ablation study: Removing encoder layers reduces overall AP by 3.9 points and large-object AP by 6.0 points, supporting the importance of global image-level self-attention.The authors hypothesize that global scene reasoning helps disentangle objects.
  • Ablation study: Adding decoder layers yields a total +8.2/9.5 AP improvement from the first to the last layer, while DETR’s set-based loss eliminates the need for NMS.Both AP and AP50 improve after every decoder layer.
  • Ablation study: Removing transformer FFNs reduces parameters from 41.3M to 28.7M and performance by 2.3 AP, showing that FFNs contribute substantially to detection quality.Only 10.8M parameters remain in the transformer after removing the FFNs.
  • Query analysis: DETR’s object-query slots learn distinct spatial and box-size specializations, yet the model finds all 24 giraffes in a synthetic out-of-distribution image.This experiment argues against strong class specialization in individual object queries.
  • Panoptic segmentation: Adding a mask head naturally extends DETR to unified panoptic segmentation, where it outperforms published COCO-val 2017 results.The evaluation reports PQ, PQth, PQst, and mask AP against established methods.

5 Conclusion

DETR uses transformers and bipartite matching for direct set prediction, achieving results comparable to optimized Faster R-CNN on COCO while remaining straightforward and extensible. Its main remaining challenges concern training, optimization, and performance on small objects.

  • Contributions: DETR combines transformers with bipartite matching loss to formulate object detection as direct set prediction.This design provides the framework’s core mechanism for producing detection sets.
  • Contributions: DETR achieves comparable results to an optimized Faster R-CNN baseline on the challenging COCO dataset.The comparison is reported for the COCO object detection benchmark.
  • Contributions: DETR is straightforward to implement and has a flexible architecture that extends easily to panoptic segmentation with competitive results.The same design supports extension beyond object detection.
  • Limitations: DETR introduces challenges in training, optimization, and performance on small objects, which future work is expected to address.The paper notes that comparable issues required several years of improvements in current detectors.

A Appendix · A.1 Preliminaries: Multi-head attention layers

The appendix introduces the attention mechanisms used by the Transformer-based model and specifies the general multi-head attention formulation. It follows prior work while distinguishing the positional-encoding details.

  • A Appendix: The appendix provides a general review of the attention mechanisms used in the model.This review is included because the model is based on the Transformer architecture.
  • A.1 Preliminaries: Multi-head attention layers: The attention mechanism follows reference.
  • A.1 Preliminaries: Multi-head attention layers: The positional-encoding details instead follow reference.These details are associated with Equation 8.
  • A.1 Preliminaries: Multi-head attention layers: The appendix states the general form of multi-head attention with M heads of dimension d.
  • A.1 Preliminaries: Multi-head attention layers: The multi-head attention mechanism is presented as a function with a specified signature.The formulation uses d′ = d M.
  • A.1 Preliminaries: Multi-head attention layers: The formulation indicates matrix and tensor sizes using underbraces.

Multi-head

Multi-head attention concatenates M single attention heads and projects their combined output with L. Each head computes query, key, and value embeddings using positional encodings, while the standard formulation commonly adds residual connections, dropout, and layer normalization.

  • Multi-head: Multi-head attention concatenates M single attention heads and applies a projection with L.The concatenation is performed along the channel axis, and the output has the same size as the query sequence.
  • Multi-head: Residual connections, dropout, and layer normalization are common additions to the multi-head attention formulation.These components are described as common practice in the standard implementation.
  • Multi-head: Each attention head computes query, key, and value embeddings after adding query and key positional encodings.The head uses a weight tensor T′ ∈ R3×d′×d and depends on positional encodings Pq and Pkv.

Single head

The single-head attention mechanism computes softmax attention weights from query–key dot products, allowing each query element to attend to the full key–value sequence. Its output aggregates value elements using these weights, with positional encodings shared across attention layers for each sequence.

  • Single head: Attention weights α are computed by applying softmax to dot products between queries and keys.Each query index i is paired with key-value index j when determining the attention weights.
  • Single head: Each query element attends to all elements of the key-value sequence.The attention formulation uses the query sequence together with the concatenated key-value sequence.
  • Single head: The final output aggregates values weighted by the attention weights.The i-th output row is defined by the attention operation over the query, key-value, and positional-encoding inputs.
  • Single head: Positional encodings may be learned or fixed and are shared across all attention layers for a given query/key-value sequence.Their exact values are specified later for the encoder and decoder.

Feed-forward network (FFN) layers

DETR’s transformer alternates multi-head attention with FFN layers implemented as two-layer 1x1 convolutions with ReLU activations. These layers use M_d input and output channels and include residual connection, dropout, and layer normalization afterward.

  • Feed-forward network (FFN) layers: FFN layers alternate with multi-head attention and function as two-layer 1x1 convolutions with ReLU activations.The convolutions have M_d input and output channels in this setting.
  • Feed-forward network (FFN) layers: A residual connection, dropout, and layer normalization follow the two convolutional layers.This arrangement is analogous to equation 6.

A.2 Losses

The approach normalizes all losses by the total number of objects in the batch, including across all GPU sub-batches during distributed training.

  • Loss normalization: All losses are normalized by the number of objects in the batch.This normalization is part of the loss formulation used by the approach.
  • Distributed training: In distributed training, normalization uses the total object count across GPUs rather than each local sub-batch.Local sub-batches may contain unequal numbers of objects, making per-GPU normalization insufficient.

Box loss … A.6 PyTorch inference code

The supplementary sections specify DETR’s box loss, transformer architecture, training configuration, evaluation methodology, additional results, instance-count limitation, and concise PyTorch inference implementation. Together, they detail both the model’s mechanics and practical reproduction considerations.

  • Box loss: DETR combines generalized IoU and ℓ1 terms for bounding-box regression, weighted by λiou and λL1.The loss is Lbox(bσ(i),ˆbi) = λiouLiou(bσ(i),ˆbi) + λL1||bσ(i) −ˆbi||1; λiou and λL1 are hyperparameters.
  • Box loss: The generalized IoU loss computes union, intersection, and enclosing-box areas using min/max operations on linear box-coordinate functions, enabling stochastic gradients.The enclosing box is the largest box containing the predicted and target boxes.
  • A.3 Detailed architecture: The transformer encodes CNN image features with spatial positional encodings and decodes them using object queries and encoder memory to produce the final predictions.Positional encodings are added to queries and keys at every multi-head self-attention layer, while decoder queries are initially zero.
  • FLOPS computation: FLOPS are averaged over the first 100 COCO 2017 validation images because Faster R-CNN computation depends on proposal count, using Detectron2’s operator-counting tool extended with bmm for DETR.The tool is used without modifications for Detectron2 models.
  • A.4 Training hyperparameters: Training uses AdamW with weight decay 10−4, gradient clipping at maximal norm 0.1, a backbone learning rate of 10−5, and a transformer learning rate of 10−4.ResNet-50 is ImageNet-pretrained, batch-normalization weights and statistics are frozen, and transformer dropout is 0.1 with Xavier initialization.
  • Transformer and Baseline: All models use λL1 = 5, λiou = 2, and N = 100 decoder query slots, while enhanced Faster-RCNN+ baselines use GIoU loss weights 20 and 1 for box and proposal regression.The baselines use DETR’s data augmentation and a 9× schedule of approximately 109 epochs.
  • Spatial positional encoding: DETR uses fixed absolute 2D positional encodings formed by concatenating independently generated sine and cosine functions for the two spatial coordinates.The frequencies differ across the functions, producing the final d-channel positional encoding.
  • A.5 Additional results and Increasing the number of instances: DETR’s panoptic predictions are additionally illustrated qualitatively, while instance-count experiments show saturation and increasing misses as the number of objects approaches its 100-query limit.The instance-count test is deliberately out-of-distribution because COCO contains few images with many objects of one class.
Loading 2005.12872v3…