PaperlayerView exact revision on arXiv

Source-linked AI summary

GroupEnsemble: Efficient Uncertainty Estimation for DETR-based Object Detection

Yutong Yang, Katarina Popović, Julian Wiederer, Markus Braun, Vasileios Belagiannis, Bin Yang

arXiv:2603.01847v1cs.CV

TL;DR

Object detectors need uncertainty estimates for both semantic predictions and bounding-box geometry, but existing ensemble approaches impose latency or memory costs. GroupEnsemble uses diverse query groups and masked, parallel decoder processing to generate multiple detection sets in one pass. Across three datasets, GroupEnsemble performs comparably to MC-Dropout, while MC-GroupEnsemble outperforms Deep Ensembles on certain metrics at significantly lower cost.

  • Problem

    DETR confidence scores capture semantic uncertainty but not spatial uncertainty, while MC-Dropout and Deep Ensembles impose inference or memory costs that hinder real-time use.

  • Method

    GroupEnsemble feeds diverse additional query groups through a shared decoder with blocked inter-group attention to produce independent detection sets simultaneously in one pass.

  • Results

    Across three datasets, GroupEnsemble performs comparably to MC-Dropout, while MC-GroupEnsemble outperforms Deep Ensembles on certain metrics at significantly lower computational cost.

  • Takeaways & Limitations

    GroupEnsemble provides a performance-runtime trade-off for uncertainty estimation and is promising for online autonomous driving.

Abstract

from arXiv · show

Detection Transformer (DETR) and its variants show strong performance on object detection, a key task for autonomous systems. However, a critical limitation of these models is that their confidence scores only reflect semantic uncertainty, failing to capture the equally important spatial uncertainty. This results in an incomplete assessment of the detection reliability. On the other hand, Deep Ensembles can tackle this by providing high-quality spatial uncertainty estimates. However, their immense memory consumption makes them impractical for real-world applications. A cheaper alternative, Monte Carlo (MC) Dropout, suffers from high latency due to the need of multiple forward passes during inference to estimate uncertainty. To address these limitations, we introduce GroupEnsemble, an efficient and effective uncertainty estimation method for DETR-like models. GroupEnsemble simultaneously predicts multiple individual detection sets by feeding additional diverse groups of object queries to the transformer decoder during inference. Each query group is transformed by the shared decoder in isolation and predicts a complete detection set for the same input. An attention mask is applied to the decoder to prevent inter-group query interactions, ensuring each group detects independently to achieve reliable ensemble-based uncertainty estimation. By leveraging the decoder's inherent parallelism, GroupEnsemble efficiently estimates uncertainty in a single forward pass without sequential repetition. We validated our method under autonomous driving scenes and common daily scenes using the Cityscapes and COCO datasets, respectively. The results show that a hybrid approach combining MC-Dropout and GroupEnsemble outperforms Deep Ensembles on several metrics at a fraction of the cost. The code is available at https://github.com/yutongy98/GroupEnsemble.

I. INTRODUCTION

DETR confidence scores capture semantic uncertainty but not uncertainty in predicted box location or size, motivating efficient uncertainty estimation for safety-critical detection. GroupEnsemble uses diverse query groups and decoder parallelism to estimate both uncertainties in one pass, while MC-GroupEnsemble improves performance-cost trade-offs.

  • DETR reformulates object detection as end-to-end set prediction without hand-crafted components such as NMS and anchor priors.
  • Semantic uncertainty concerns class and object existence, whereas spatial uncertainty concerns location and size.
  • A high-confidence class prediction does not guarantee certainty in the bounding-box location because classification and localization are distinct tasks.
  • MC-Dropout and Deep Ensembles require repeated inference, while Deep Ensembles additionally store multiple models, creating latency and memory costs for real-time applications.
  • GroupEnsemble feeds diverse additional query groups through a shared decoder with blocked inter-group attention, producing independent complete detection sets simultaneously in one pass.
  • GroupEnsemble performs comparably to MC-Dropout, while MC-GroupEnsemble outperforms Deep Ensembles on certain metrics at significantly lower computational cost across Cityscapes, Foggy Cityscapes, and COCO.

II. RELATED WORK

Prior uncertainty-estimation methods provide ensembles but incur repeated-inference or multi-model costs. GroupEnsemble instead exploits DETR-like query groups and decoder structure to produce diverse predictions efficiently.

  • DETR-like models extend transformer-based end-to-end set prediction through architectural changes and universal training methods.
  • DETR-based Object Detection: GroupEnsemble builds on Group DETR because its learned high-quality query groups support multiple detection sets and retain flexibility across DETR-like architectures.
  • Uncertainty Estimation: Uncertainty estimation quantifies prediction confidence, often using ensembles that generate redundant diverse predictions and aggregate their variance or entropy.
  • Uncertainty Estimation: MC-Dropout repeatedly runs one stochastic network, whereas Deep Ensembles train and store multiple independent models.
  • Uncertainty Estimation: GroupEnsemble addresses ensemble latency by leveraging DETR architecture properties to estimate uncertainty without the forward-pass scaling associated with repeated inference.

III. METHOD

DETR encodes images into features, transforms learnable object queries with a decoder, and maps them to detections. GroupEnsemble adds multiple inference-time query groups and preserves independent group processing through attention masking.

  • Overview: DETR consists of an image encoder, transformer decoder, and classification and regression heads.
  • Overview: The decoder combines image features and N learnable object queries to produce transformed queries for detection.
  • Overview: Self-attention enables query interactions, while cross-attention extracts object-centric information from image features.
  • Overview: Classification and regression heads convert transformed queries into detections containing a bounding box, predicted class, and class confidence.
  • Overview: During inference, DETR queries probe image features through the decoder to produce the detection set.
  • GroupEnsemble: GroupEnsemble adds three inference-time components to standard DETR, including multiple query groups and an attention mask that preserves independent group processing.

1) Additional Groups of Object Queries:

GroupEnsemble augments the standard query set with additional groups and masks decoder attention between groups. This yields independently transformed detection sets while retaining full within-group interactions.

  • 1) Additional Groups of Object Queries:: GroupEnsemble uses G groups of object queries during inference, with the original group plus G −1 additional groups, each containing N queries.
  • 1) Additional Groups of Object Queries:: The decoder attention mask blocks inter-group interactions while allowing full intra-group interactions, enabling independent simultaneous transformation of each query group.

2) Attention Mask:

GroupEnsemble reuses diverse learned query groups to produce overlapping detection sets independently and simultaneously, enabling ensemble-based uncertainty estimation. The query groups have similar detection capability and diverse reference points, supporting their use for this purpose.

  • 2) Attention Mask:: GroupEnsemble transforms each query group independently and simultaneously, producing G overlapping detection sets for one image in a single forward pass.The overlap between individual detection sets enables ensemble-based uncertainty estimation.
  • 2) Attention Mask:: Each detection cluster groups predictions with high mutual IoU and the same class, then aggregates them into a final detection with semantic and spatial uncertainty estimates.The method uses BSAS clustering to combine detections across query groups.
  • 2) Attention Mask:: Query groups must combine strong detection capability with diversity because reliable ensemble uncertainty estimation depends on diverse individual predictions.The groups share the same model, so diversity must come from the query groups themselves.
  • 2) Attention Mask:: Group DETR separately trains each query group to predict a complete detection set, while standard inference uses only the first group.GroupEnsemble instead leverages the additional learned groups for uncertainty estimation.
  • 2) Attention Mask:: With five query groups, per-group detection performance is similar on Cityscapes, and visualized reference points for the same objects are diverse.These observations support both the detection capability and diversity requirements.
  • 2) Attention Mask:: Group DETR meets the requirements for reliable ensemble-based uncertainty estimation and allows GroupEnsemble to inherit applicability across DETR-like models.Group DETR is described as a universal training method applicable to almost any DETR-like architecture.

C. Attention Mask

GroupEnsemble uses a self-attention mask to block interactions between query groups while preserving interactions within each group. This lets the decoder produce independent detection sets simultaneously and avoids the self-attention memory explosion associated with concatenating all groups.

  • C. Attention Mask: GroupEnsemble retains all learned query groups instead of discarding them, creating redundant detection sets needed for ensemble-based uncertainty estimation.The method builds on Group DETR’s multiple high-quality query groups.
  • C. Attention Mask: The decoder receives W = G·N concatenated queries, where G is the number of groups and N is the number of queries per group.The groups are arranged in group order before applying the mask.
  • C. Attention Mask: The self-attention mask blocks interactions between queries from different groups while fully enabling interactions within each group.Cross-attention still lets each query attend individually to the image features.
  • C. Attention Mask: Masked groups are transformed independently and simultaneously, producing G individual detection sets in a single forward pass.This preserves the intended ensemble structure without sequentially repeating inference.
  • C. Attention Mask: Processing query groups in the batch dimension avoids the O(W^2) self-attention memory explosion and improves efficiency.The batch-dimension implementation provides the same functionality as the explicit attention mask.

D. Clustering and Aggregation

GroupEnsemble clusters overlapping detections from multiple query-group outputs by spatial and semantic agreement, then aggregates each cluster into a final detection with confidence and spatial uncertainty estimates.

  • Clustering: BSAS clusters detections when bounding-box IoU exceeds threshold θ and predicted class labels match.Detections are sorted by class-prediction confidence before clustering.
  • Clustering: The standard BSAS method is retained because restricting clusters to one detection per query group increased false positives.The increase arose from repeated detections of a single target within the same detection set.
  • Aggregation: Each cluster’s confidence scales its maximum detection confidence by α_m = min(|C_m|, G)/G.The scaling lowers confidence when few query groups detect an object and preserves the maximum when all groups detect it.
  • Aggregation: Confidence-weighted averaging produces the final bounding box, while weighted variance estimates spatial uncertainty from prediction disagreement.The weighting prioritizes high-confidence detections and reduces the influence of noisy outliers.
  • Aggregation: Each cluster yields a detection containing a bounding box, spatial covariance, class label, and confidence score, optionally filtered by confidence threshold.The resulting outputs support probabilistic object detection by representing both semantic and spatial uncertainty.

A. Experiment Setup

The experiments compare detection quality, uncertainty quality, calibration, and computational efficiency across autonomous-driving and general object-detection datasets using controlled ensembling implementations.

  • Metrics: Evaluation covers detection accuracy, uncertainty quality, calibration, and computational efficiency.The metrics are mAP, PDQ, D-ECE, latency, and parameter count, respectively.
  • Dataset: Experiments use Cityscapes, Foggy Cityscapes, and COCO to cover urban driving, domain shift, and general detection scenarios.Foggy Cityscapes includes visibility levels of 600m, 300m, and 150m.
  • Implementation Details: The Group DETR baseline uses a ResNet-50 backbone, five object-query groups, and dropout probability 0.1 during training.Cityscapes fine-tuning uses eight driving-related classes at higher image resolution.
  • Implementation Details: All methods share a deterministic encoder and BSAS clustering with θ empirically set to 0.7, isolating ensemble differences to decoder-side components.Deep Ensembles use five models with independently trained decoders and task heads.
  • Implementation Details: MC-GroupEnsemble combines GroupEnsemble with MC-Dropout using five different query groups to increase stochasticity.This differs from standard MC-Dropout, which duplicates a fixed query group five times.
  • Aggregation: Scaled maximum confidence addresses the calibration problems of mean and unscaled maximum aggregation.Mean confidence produces under-confident detections, whereas unscaled maximum confidence produces slightly over-confident detections.

B. Experimental Results

GroupEnsemble bridges uncertainty-estimation performance and runtime, matching MC-Dropout while avoiding the extreme resource demands of Deep Ensembles. The hybrid MC-GroupEnsemble further improves benchmark performance at low model-size overhead.

  • Runtime and performance: 66% faster processing makes both MC-Dropout and GroupEnsemble more suitable for real-time inference than sequential counterparts.The speedup comes from parallelizing query-group processing through the transformer decoder.
  • Runtime and performance: GroupEnsemble achieves competitive performance to MC-Dropout across datasets and outperforms deterministic baselines across all metric categories.Deterministic Group DETR has lower computational cost but cannot estimate spatial uncertainty, resulting in poor PDQ scores.
  • Runtime and performance: MC-GroupEnsemble achieves the best PDQ and mAP on Cityscapes and Foggy Cityscapes, with only a +0.7% increase in model size over standard GroupEnsemble.The results include the challenging Foggy Cityscapes domain-shift scenario.
  • Ablation setup: Table II evaluates how the number of query groups affects GroupEnsemble uncertainty-estimation quality.The table reports an ablation over query-group count, with decoder-only measurements marked separately and best results bolded.
  • Ablation setup: Table III compares alternative confidence-score aggregation methods for GroupEnsemble detections.The ablation identifies the best-performing aggregation results across the reported metrics.

C. Ablation Study

The ablations show that more query groups improve uncertainty estimates, while confidence aggregation materially affects detection and calibration quality. These findings support GroupEnsemble’s design choices and its broader efficiency-performance trade-off.

  • Query-group count: Increasing the number of query groups raises PDQ scores, indicating more accurate uncertainty estimates.More groups produce more detection sets and more diverse detections for estimating spatial variance.
  • Query-group count: Decoder parallelism processes all query groups simultaneously, preserving the efficiency advantage as query-group count increases.The supplied passage links simultaneous processing to the transformer decoder’s parallelism.
  • Aggregation strategy: Scaling maximum confidence by detections-per-group improves results over the deterministic Group DETR baseline.Mean confidence performs worse across all three metrics, while unscaled maximum confidence improves PDQ and mAP but worsens D-ECE.
  • Overall implication: GroupEnsemble is designed as a single-pass uncertainty-estimation method that complements MC-Dropout in the paper’s broader evaluation.The conclusion passage reports comparable performance to MC-Dropout and stronger hybrid results than Deep Ensembles on certain metrics at lower cost.
Loading 2603.01847v1…