PaperlayerView exact revision on arXiv

Source-linked AI summary

Localisation-Aware Uncertainty for Pretrained Object Detection

Charmaine Barker, Daniel Bethell, Simos Gerasimou

arXiv:2610.01409v1cs.CV

TL;DR

Object detectors need uncertainty estimates that identify unreliable localisations under distribution shift and adversarial perturbation, but existing approaches can require retraining, architectural changes, or repeated inference. GRACE adds a saliency-guided evidential meta-model while keeping the detector frozen. Across four datasets it generally provides stronger localisation-aware uncertainty, and under attacks its TP-FP AUROC is 0.825 ± 0.020 versus 0.676 ± 0.000 for MetaDetect.

  • Problem

    Object detection uncertainty methods may require retraining, detector changes, or repeated inference, while reliable deployment calls for identifying unreliable localisations under shift and perturbation.

    Source: §1 · p. 1

  • Method

    GRACE trains a post-hoc evidential meta-model using saliency-guided corruption and targets combining localisation error, corruption, and prediction instability, while leaving the detector frozen.

    Source: §1 · pp. 1–2

  • Results

    Across four datasets, GRACE generally provides stronger localisation-aware uncertainty; under attacks, TP-FP AUROC is 0.825 ± 0.020 versus MetaDetect’s 0.676 ± 0.000.

    Source: §5.3 · p. 9, §6 · p. 10

  • Takeaways & Limitations

    GRACE estimates when each predicted localisation should be trusted while preserving the detector and its original box predictions.

    Source: §1 · p. 2, §6 · p. 10

    The stated setting is supervised object detection trained on a finite labelled set with structured box-and-class annotations.

    Source: §3 · p. 2

Abstract

from arXiv · show

Reliable uncertainty estimation is essential for deploying object detectors when distribution/covariate shift and adversarial attacks may occur. Existing approaches often require detector retraining, architectural modification, or repeated inference, which may be infeasible or incur significant overheads. We introduce a lightweight post-hoc evidential meta-model that learns when object localisations should be considered uncertain while keeping the base detector frozen. Our approach automatically identifies localisation-relevant features and uses saliency-guided modification to construct an increasingly challenging curriculum. Detection-level targets combine localisation error, modification level, and prediction instability to guide an evidential meta-model to estimate uncertainty for each predicted bounding box. Our approach requires no changes to the detector and preserves its original localisation outputs. Across adversarial attacks and evaluated strengths, GRACE improves TP-FP AUROC by 22% relative to the strongest comparator in some cases while maintaining in-distribution detection performance.

1 INTRODUCTION

Reliable object detection needs uncertainty at the detection and localisation level, especially under shift or perturbation. GRACE adds a post-hoc evidential meta-model that estimates uncertainty for each predicted box without changing the frozen detector.

1 INTRODUCTION

  • Detection uncertainty must distinguish a high-confidence class prediction from a poorly localised box, spurious detection, or location drift under perturbation.The introduction argues for detection-level, localisation-aware uncertainty rather than only image-level or class-confidence estimates.Source: §1 · p. 1
  • Existing uncertainty approaches may require stochastic inference, repeated passes, retraining, architectural changes, or access to unavailable training pipelines.Detection-specific methods also may rely on existing confidence structure and do not explicitly guide when localisation becomes unreliable.Source: §1 · p. 1
  • GRACE uses saliency calibration and progressively corrupted views to train a lightweight evidential meta-model to flag unreliable localisations while preserving the detector’s box predictions.Its uncertainty targets reflect when retained detections become poorly localised, affected by corruption, or unstable under perturbation.Source: §1 · pp. 1–2

2 RELATED WORK

Prior uncertainty methods estimate observation noise, model uncertainty, or prediction variability, while post-hoc detection methods seek to avoid modifying pretrained detectors. The related work identifies localisation-aware robustness under shift as an underexplored need.

Uncertainty Quantification

  • Uncertainty methods distinguish irreducible, input-dependent aleatoric uncertainty from epistemic uncertainty associated with limited model knowledge.Bayesian methods, ensembles, and other approaches estimate uncertainty, while model-agnostic test-time augmentation captures variability without explicitly separating its sources.Source: §2 · p. 2

Post-hoc Uncertainty

  • Post-hoc detection methods derive uncertainty from frozen representations, detector outputs, or box characteristics, often using lightweight auxiliary models.Related saliency-guided frozen-representation methods are classification-oriented and do not directly address detection-level localisation uncertainty.Source: §2 · p. 2
  • The related work identifies additional inference costs and limited or underexplored robustness to covariate shift and adversarial perturbations as gaps in post-hoc approaches.It motivates an approach that preserves the frozen detector while estimating localisation-aware uncertainty as inputs become unreliable.Source: §2 · p. 2

3 PRELIMINARIES

The paper frames object detection as supervised prediction over labelled images and structured box-and-class annotations. It also motivates uncertainty estimation by noting that pretrained deterministic detectors can be overconfident on unfamiliar or corrupted inputs.

3 PRELIMINARIES

  • The stated setting is supervised object detection trained on a finite labelled set, with variable-sized annotations comprising object boxes and class labels.The input images and annotations are described as independently and identically drawn from a joint distribution.Source: §3 · p. 2
  • A pretrained deterministic detector may be overconfident on out-of-distribution or adversarial inputs because its box predictions are optimised for detection, not epistemic uncertainty.The passage says this can yield spuriously confident detections on unfamiliar or corrupted inputs.Source: §3 · p. 2
  • The overview identifies saliency calibration and uncertainty-guided training stages, with evidential branches predicting uncertainty from Dirichlet evidence.The detector structure and weights remain unchanged in the described post-hoc approach.Source: §3 · p. 3, Figure 1 · p. 3

4 GRACE

GRACE connects localisation-relevant detector features to a lightweight evidential meta-model while keeping the pretrained detector frozen. Saliency-guided corruption and detection-level targets teach the meta-model to increase uncertainty as localisation reliability decreases.

4 GRACE

  • GRACE combines relevance-based saliency, evidential meta-modelling, and curriculum noise, using soft-target supervision to reduce confidence as reliability decreases.The detector remains frozen; the curriculum progressively perturbs inputs and trains the Normal-Inverse-Gamma model to become less confident.Source: §4 · p. 3

4.1 SALIENCY CALIBRATION

  • Layer relevance is propagated backward with the LRP-ϵ rule, and average relevance magnitude ranks layers for selection.The smallest layer subset covering a chosen fraction of total relevance is retained; for η = 0.9, it captures 90% of the relevance mass.Source: §4.1 · pp. 3–4
  • Localisation-wise attribution maps are merged into a spatial weight map, where larger values mark regions more relevant to retained localisation outputs.The map is used to prioritise input perturbations during curriculum generation.Source: §4.1 · p. 4

4.2 UNCERTAINTY GUIDED TRAINING

  • The evidential meta-model pools selected-layer features over each predicted box and estimates localisation uncertainty without altering the retained box prediction.Its outputs parameterise aleatoric and epistemic uncertainty for the localisation channels, while the pretrained detector parameters remain frozen.Source: §4.2 · p. 5
  • A monotonic corruption schedule targets salient pixels first at low noise and spreads corruption across the image at higher noise levels.A shared mask ensures stochastic corruption while preserving the expected noise budget and making each later view no less noisy than its predecessor.Source: §4.2 · p. 5
  • Uncertainty targets combine localisation error, corruption exposure inside the box, and prediction instability, including detections that drift or disappear.Their union-style aggregation is high if any factor signals unreliability and remains low only for accurate, lightly corrupted, stable detections.Source: §4.2 · pp. 5–6
  • Training shifts the corruption-level sampling distribution from cleaner views early to stronger corruption later, rather than requiring each image to pass through every level.The evidential regression objective gives greater fit weight to accurate, stable, lightly corrupted detections and greater regularisation weight to unreliable ones.Source: §4.2 · pp. 6–7

5 EXPERIMENTAL EVALUATION

Across four datasets, GRACE is strongest overall on CelebA, COCO, and VisDrone, while EMM leads on TT100K. Under covariate shifts and adversarial attacks, GRACE’s uncertainty also tracks localisation degradation and detection failure.

5 EXPERIMENTAL EVALUATION

  • The evaluation compares post-hoc uncertainty methods across four datasets using incorrect-detection identification, selective prediction, uncertainty-IoU alignment, and added inference time.The comparisons include the base model, EMM, ModelNet, and MetaDetect.Source: §5.1 · p. 7, §5 · p. 7

5.1 CORE RESULTS

  • GRACE performs best overall on CelebA, COCO, and VisDrone; on TT100K, EMM’s uncertainty-IoU correlation is −0.48 ± 0.01 versus GRACE’s −0.44 ± 0.02.On CelebA, GRACE’s TP-FP AUROC is 0.83 ± 0.02; on COCO, it leads every uncertainty-quality metric, including TP-FP AUROC of 0.55 ± 0.03.Source: §5.1 · p. 7, Table 1 · p. 8

5.2 COVARIATE SHIFT RESULTS

  • At severity t = 8, GRACE’s uncertainty-error correlations are consistently positive, with mean Pearson r from approximately 0.37 under compression to 0.65 under resize-crop.EMM remains close to zero across most perturbations, while ModelNet is frequently negatively correlated.Source: §5.2 · p. 8, Figure 3 · p. 9
  • At severity t = 8, GRACE has the highest disappearance-prediction AUROC across all five shifts, reaching approximately 0.87–0.91 for four shifts and around 0.65 under noise.Comparative approaches are generally lower; MetaDetect is typically around 0.59–0.72 and EMM close to 0.50.Source: §5.2 · p. 8, Figure 4 · p. 9

5.3 ADVERSARIAL ATTACK RESULTS

  • 0.825 ± 0.020 adversarial TP-FP AUROC across attacks and strengths exceeds MetaDetect (0.676 ± 0.000), EMM (0.519 ± 0.028), and ModelNet (0.501 ± 0.241).GRACE also achieves the highest adversarial FP AUPRC, 0.850 ± 0.025.Source: Figure 5 · p. 9, §5.3 · p. 9

6 CONCLUSIONS

GRACE is a lightweight post-hoc method for estimating unreliable localisations with the base detector frozen. It generally improves localisation-aware uncertainty across datasets and remains effective under covariate shift and adversarial attacks.

6 CONCLUSIONS

  • GRACE learns when predicted localisations are unreliable while keeping the base detector frozen.The method is post-hoc and lightweight.Source: §6 · p. 10
  • Across four datasets, GRACE generally provides stronger localisation-aware uncertainty than comparative approaches and remains effective under covariate shift and adversarial attacks.Source: §6 · p. 10
  • Future work will investigate video object detection by incorporating temporal information into uncertainty estimation.Source: §6 · p. 10

A IMPLEMENTATION DETAILS

GRACE trains an evidential meta-model while keeping the pretrained detector frozen. Saliency guides feature selection and corruption during training, and the resulting uncertainty is computed around the detector’s original box coordinates.

A IMPLEMENTATION DETAILS

  • The pretrained detector remains frozen throughout the procedure; only the evidential meta-model parameters are updated.Source: A IMPLEMENTATION DETAILS · p. 13
  • GRACE uses localisation relevance to select salient detector layers and construct input weight maps for saliency-guided corruption.The algorithm propagates relevance through the detector, accumulates layer relevance scores, and selects the smallest salient layer set meeting its threshold.Source: A IMPLEMENTATION DETAILS · p. 13
  • Training samples curriculum stages, applies saliency-weighted corruption, matches retained clean and corrupted detections, and extracts box-conditioned frozen features.Source: A IMPLEMENTATION DETAILS · p. 13
  • For each box coordinate, the evidential head predicts NIG parameters while fixing the predictive mean to the detector’s original coordinate; detection uncertainty is the mean total uncertainty across four coordinates.GRACE therefore estimates uncertainty around the original localisation without modifying the predicted box.Source: A IMPLEMENTATION DETAILS · pp. 13–14

B THEORETICAL ANALYSIS

The appendix analyzes the uncertainty-target aggregation used by GRACE. Its stated bound is 1-Lipschitz with respect to the constituent uncertainty terms.

LEMMA 1

  • The uncertainty target in Equation 15 is 1-Lipschitz with respect to the ℓ1 distance between its constituent uncertainty terms.Source: LEMMA 1 · pp. 14–15
  • Because the constituent uncertainty terms lie in [0, 1], their complementary factors also lie in [0, 1], supporting the stated bound.Source: LEMMA 1 · p. 14
  • The bound means changes in localisation, corruption exposure, or prediction stability are not amplified by the aggregation when forming the final uncertainty target.Source: LEMMA 1 · p. 15

C ADDITIONAL EXPERIMENTS

The appendix adds analyses of metrics, adversarial attacks, thresholds, and GRACE hyperparameters.

C ADDITIONAL EXPERIMENTS

FULL RESULTS

GRACE is generally effective at ranking poorly localised detections as uncertain, including under adversarial attacks, though its relative performance varies by dataset and metric. It adds modest inference overhead, and its uncertainty is systematically related to box size.

FULL RESULTS

  • The frozen YOLO11n detector provides different baselines across datasets: TT100K has mAP50:95 of 0.52±0.00, while CelebA has AP75 of 0.04 ± 0.00.All uncertainty approaches are evaluated on these fixed detection outputs.Source: FULL RESULTS · p. 15
  • Calibration and localisation alignment differ: MetaDetect calibrates detection-success probabilities strongly on several datasets, while its localisation alignment varies by dataset.EMM and ModelNet also show dataset-dependent results, and ModelNet’s uncertainty–IoU correlations are positive on CelebA, COCO, and TT100K.Source: FULL RESULTS · pp. 15–16
  • GRACE provides the most consistent localisation-aware uncertainty, with strong uncertainty–IoU alignment across datasets despite not leading every metric.It has the strongest reported correlations on CelebA, COCO, and VisDrone, and a negative correlation on TT100K.Source: FULL RESULTS · pp. 16–17
  • GRACE adds modest inference overhead compared with ModelNet and MetaDetect, while EMM is slightly faster on TT100K and VisDrone.GRACE adds 2.66 ± 0.48ms on CelebA and 8.01±4.76ms on COCO; its TT100K overhead is effectively comparable to EMM.Source: FULL RESULTS · p. 18, Table 4 · p. 19
  • GRACE uncertainty is negatively correlated with box size across all four datasets, so smaller detections tend to receive greater uncertainty.The effect is strongest on TT100K and VisDrone, where small and distant objects are common; box size should be considered when interpreting uncertainty.Source: FULL RESULTS · pp. 18–19, Figure 10 · p. 19

ADVERSARIAL ATTACK RESULTS

  • Across COCO, TT100K, and VisDrone attacks, relative performance varies: MetaDetect is strongest on COCO, GRACE and EMM separate on TT100K, and GRACE leads on VisDrone.GRACE reaches approximately 0.75 under the strongest TOG Vanishing attack on TT100K and approximately 0.75 and 0.70 under TOG Vanishing and TOG Fabrication on VisDrone.Source: FULL RESULTS · p. 20, Figure 11 · p. 20, Figure 12 · p. 20
  • GRACE leads adversarial results on CelebA and VisDrone, while MetaDetect leads on COCO and EMM on TT100K; GRACE is best in 8 of 16 comparisons.On CelebA it achieves TP-FP AUROC 0.825 ± 0.020 and FP AUPRC 0.850 ± 0.025; on VisDrone, TP-FP AUROC 0.634 ± 0.003 and FP AUPRC 0.717 ± 0.003.Source: FULL RESULTS · pp. 20–21, Table 5 · p. 21

DISTRIBUTIONAL SHIFT RESULTS

Under covariate shift, GRACE’s clean-detection uncertainty tracks later localisation degradation and predicts disappearance, with performance varying across datasets and perturbations. Ablations show that curriculum severity and saliency coverage affect the quality–cost trade-off.

DISTRIBUTIONAL SHIFT RESULTS

  • On CelebA, GRACE has the strongest positive uncertainty–localisation-error correlations across perturbations and severities; on COCO, the relationship is positive but weaker.On CelebA, correlations reach approximately 0.56 under blur and 0.65 under resize-crop at t = 8; COCO brightness remains below approximately 0.10.Source: DISTRIBUTIONAL SHIFT RESULTS · pp. 22–23, Figure 14 · p. 22, Figure 15 · p. 23
  • TT100K shows stronger shift predictability but competition from EMM, while VisDrone shows severity-related increases for GRACE and its strongest correlations under resize-crop.On TT100K, GRACE remains particularly effective under blur and resize-crop; on VisDrone, resize-crop correlations stay around 0.46–0.50.Source: DISTRIBUTIONAL SHIFT RESULTS · pp. 23–24, Figure 16 · p. 24, Figure 17 · p. 25
  • Across covariate shifts, GRACE’s clean uncertainty tracks later localisation degradation and predicts disappearance, with especially consistent disappearance prediction on CelebA, COCO, and VisDrone.Source: DISTRIBUTIONAL SHIFT RESULTS · pp. 24, 26
  • For disappearance prediction, GRACE is strong on CelebA and remains the strongest approach across nearly all COCO perturbations and severities.At t = 8 on CelebA, it reaches approximately 0.88 under blur and 0.92 under resize-crop; COCO values are generally lower.Source: DISTRIBUTIONAL SHIFT RESULTS · pp. 24–25, Figure 18 · p. 26, Figure 19 · p. 27
  • On TT100K, GRACE and EMM both predict disappearance well, with EMM stronger on some shifts; on VisDrone, GRACE is strong across perturbations and severities.GRACE reaches approximately 0.84 on TT100K blur and resize-crop at t = 8; VisDrone noise is an exception, with approximately 0.61 at t = 8.Source: DISTRIBUTIONAL SHIFT RESULTS · pp. 25–26, Figure 20 · p. 28, Figure 21 · p. 29

ABLATION STUDY

  • Curriculum length and corruption rate jointly control the final corruption level; shorter or more gradual schedules generally provide stronger uncertainty quality than excessively severe schedules.At T = 20, γ = 0.10 and γ = 0.15 reach noise targets of approximately 0.86 and 0.95, with TP-FP AUROC approaching 0.50 and correlations moving toward zero.Source: ABLATION STUDY · pp. 27–28, Figure 22 · p. 30, Figure 23 · p. 30
  • Performance is relatively stable across saliency coverage thresholds, while added time per box increases as η grows; η = 0.70 is selected as the default trade-off.Added time rises from approximately 0.8ms at η = 0.60 to nearly 1.9ms at η = 0.90, with limited uncertainty-quality improvement from retaining more layers.Source: ABLATION STUDY · p. 29

D ADDITIONAL DISCUSSION

GRACE’s relative gains vary by dataset: it remains ahead of EMM on TT100K, while its gains are clearer on CelebA and VisDrone. Its uncertainty is intended to track localisation quality, which differs from calibration of binary detection success.

D ADDITIONAL DISCUSSION

  • GRACE outperforms EMM on TT100K, where EMM is particularly competitive, while GRACE’s gains are clearer on CelebA and VisDrone.The discussion attributes TT100K’s smaller additional benefit to stronger localisation-reliability signals in the frozen detector representations; GRACE still adds benefit there.Source: D ADDITIONAL DISCUSSION · pp. 30–31
  • MetaDetect has lower Brier and success-score calibration errors on several datasets, while GRACE more consistently assigns uncertainty to poorly localised detections.The paper treats these outcomes as reflecting complementary objectives: binary detection-success calibration versus uncertainty aligned with localisation quality.Source: D ADDITIONAL DISCUSSION · p. 31
  • On TT100K and VisDrone, GRACE assigns greater uncertainty to smaller predicted objects, consistent with their greater localisation difficulty.The discussion says this size dependence is not inherently undesirable and does not by itself indicate an independent estimator bias.Source: D ADDITIONAL DISCUSSION · p. 31

E EXPERIMENTAL SETUP

The experimental setup evaluates GRACE across four detection benchmarks, using frozen YOLO11n detectors and post-hoc uncertainty comparators. It specifies training, attack and metric protocols, and analyzes computational costs.

E EXPERIMENTAL SETUP

  • The setup describes datasets, models, comparative methods, training, and task-specific implementation details to make the evaluation protocol reproducible.The section proceeds from datasets through implementation details for each experimental setting.Source: E EXPERIMENTAL SETUP · p. 31

DATASETS

  • COCO, CelebA, VisDrone, and TT100k cover varied image domains, object classes, resolutions, and standard train, calibration, and test splits.The examples are shown in Figure 25; the benchmarks include 80 COCO classes, one CelebA face class, 10 VisDrone classes, and 221 TT100k classes.Source: DATASETS · pp. 31–32, Figure 25 · p. 31

MODELS

  • YOLO11n is the base detector; dataset-specific detectors are fine-tuned where applicable, then frozen during GRACE training.COCO uses pretrained weights, while CelebA, VisDrone, and TT100k use detection heads adapted to their classes.Source: MODELS · p. 32

COMPARATIVE APPROACHES

  • Comparators include EMM, ModelNet, and MetaDetect; EMM uses clean detections without GRACE’s saliency selection or noise-driven curriculum.MetaDetect’s classifier and regressor use histogram-based gradient boosting, with true positives defined using an IoU threshold of 0.5.Source: COMPARATIVE APPROACHES · pp. 32–33

TRAINING DETAILS

  • Detector parameters remain frozen during uncertainty-model training, and GRACE is trained separately for each detector–dataset pair.GRACE uses saliency coverage threshold η = 0.7 and a 20-epoch meta-model schedule; results are reported over five independent random-seed runs.Source: TRAINING DETAILS · p. 33

METRICS AND ATTACKS

  • The protocol includes success-score calibration error and six detection-specific attacks, spanning white-box gradient and black-box square perturbations.The attacks include PGD under ℓ∞ and ℓ2 constraints, TOG-Vanishing, TOG-Fabrication, and Square Attack under an ℓ∞ constraint.Source: METRICS AND ATTACKS · p. 33

RUNTIME AND COMPLEXITY

  • Calibration is dominated by approximately O(N J̄ A), while inference costs O(F + J̄D), or O(J̄) additional overhead for fixed feature dimensions D.The curriculum samples one corruption level per example, avoiding an O(ENTF) training cost.Source: RUNTIME AND COMPLEXITY · pp. 33–34
Loading 2610.01409v1…