PaperlayerView exact revision on arXiv

Source-linked AI summary

Probabilistic Anchor Assignment with IoU Prediction for Object Detection

Kang Kim, Hee Seok Lee

arXiv:2007.08103v2cs.CV

TL;DR

Anchor assignment strongly affects object detection, but IoU-based heuristics do not account for the model’s learned cues. The paper introduces probabilistic, model-adaptive assignment and IoU-aware box ranking, reporting new COCO test-dev records with efficient models.

  • Problem

    IoU-based anchor assignment can ignore the actual content of intersecting regions, motivating model-dependent positive and negative selection.

  • Method

    PAA fits a two-modality probability distribution to model-conditioned anchor scores, while predicted IoU is combined with classification confidence for NMS ranking and score voting.

  • Results

    The models achieve new state-of-the-art performance on COCO test-dev, with score voting improving each method in ablation studies.

  • Takeaways & Limitations

    The approach adaptively selects positives without a predefined positive count or IoU threshold and aligns classification and localization across detector procedures.

  • Takeaways & Limitations

    The analysis assumes a binary classification task, with extension to multi-class described as straightforward.

Abstract

from arXiv · show

In object detection, determining which anchors to assign as positive or negative samples, known as anchor assignment, has been revealed as a core procedure that can significantly affect a model's performance. In this paper we propose a novel anchor assignment strategy that adaptively separates anchors into positive and negative samples for a ground truth bounding box according to the model's learning status such that it is able to reason about the separation in a probabilistic manner. To do so we first calculate the scores of anchors conditioned on the model and fit a probability distribution to these scores. The model is then trained with anchors separated into positive and negative samples according to their probabilities. Moreover, we investigate the gap between the training and testing objectives and propose to predict the Intersection-over-Unions of detected boxes as a measure of localization quality to reduce the discrepancy. The combined score of classification and localization qualities serving as a box selection metric in non-maximum suppression well aligns with the proposed anchor assignment strategy and leads significant performance improvements. The proposed methods only add a single convolutional layer to RetinaNet baseline and does not require multiple anchors per location, so are efficient. Experimental results verify the effectiveness of the proposed methods. Especially, our models set new records for single-stage detectors on MS COCO test-dev dataset with various backbones. Code is available at https://github.com/kkhoot/PAA.

1 Introduction

The paper treats anchor assignment as a model-dependent probabilistic decision and aligns training and testing around classification and localization quality. The resulting methods improve detection while remaining efficient.

  • IoU-threshold assignment can select anchors containing noisy background, nearby objects, or few meaningful target parts.
  • PAA adapts positive and negative anchor selection to the model’s learning status rather than relying only on anchor–GT IoU.
  • Anchor scores combine classification and localization losses, and a two-modality probability distribution assigns anchors according to their probabilities.
  • Predicted IoU is multiplied by classification confidence to rank boxes during NMS, aligning localization and classification objectives across detector procedures.
  • Score voting further improves each method in ablation studies.
  • All models achieve new state-of-the-art performance on COCO test-dev, while adding one convolutional layer, using one anchor per location, and requiring no test-time computation.

2 Related Work

Prior work established anchor assignment as an important determinant of detector performance and explored dynamic anchors, scored selection, and localization-quality prediction. PAA differs by probabilistically modeling anchor selection and connecting it with box ranking.

  • Anchor assignment has been identified as a crucial factor affecting object-detection performance.
  • MetaAnchor dynamically predicts anchor-function parameters and takes anchor shapes as an argument during training and testing.
  • MAL and NoisyAnchor score anchors using classification and localization losses but select a fixed number of top-scoring positives.
  • These fixed-selection mechanisms cannot flexibly adapt the number of positives to the model’s learning status and input.
  • Earlier methods predicted IoU or objectness for localization quality, but did not unify classification and IoU scores while relating NMS to anchor assignment.

3 Proposed Methods

The proposed methods make anchor assignment adaptive to the model’s learning status and align training, testing, and post-processing around combined classification and localization quality.

  • 3.1 Probabilistic Anchor Assignment Algorithm: The anchor score multiplies classification quality by localization quality raised to λ, with predicted-box IoU used as the localization score.Taking the negative logarithm connects this score to classification and IoU losses.
  • 3.1 Probabilistic Anchor Assignment Algorithm: PAA models anchor scores for each ground-truth box probabilistically, then separates anchors into positive and negative samples according to their estimated probabilities.The score reflects classification and localization quality and depends on the model parameters.
  • 3.1 Probabilistic Anchor Assignment Algorithm: A two-component Gaussian mixture model is fit to anchor scores with EM, and probability-based boundary schemes separate anchors into positive, negative, and ignored groups.Candidate anchors are selected from pyramid levels before fitting the mixture and applying a separation rule.
  • 3.1 Probabilistic Anchor Assignment Algorithm: PAA adaptively determines the number of positive samples from the model-conditioned probability distribution rather than using an IoU threshold or fixed sample-count heuristic.The training objective uses positive and negative anchor probabilities, approximated as binary assignments for optimization.
  • 3.2 IoU Prediction as Localization Quality: IoU prediction adds a convolutional prediction head so NMS can rank boxes with a unified classification-localization score instead of classification alone.The predicted IoU is trained with an IoU prediction loss and incorporated into the unified score during NMS.
  • 3.3 Score Voting: Score voting updates boxes after NMS using overlap-based weights combined with the unified classification-localization score, which improves performance over using the overlap weight alone.The unified score avoids assigning large weights to boxes with high classification confidence but poor localization.

4 Experiments

Experiments evaluate the proposed methods on MS COCO using standard training and evaluation settings, including ablations on minival and comparisons on test-dev.

  • Experimental setup: Experiments use trainval35k for training, minival for ablations, and test-dev for comparisons with prior methods.Accuracy is computed with the official COCO evaluation code.
  • Evaluation design: Ablation studies are reported on COCO minival with a Res50 backbone, comparing separation boundaries, FNP, FSR, and individual methods.Table 1 organizes boundary comparisons on the left and individual-method effects on the right.
  • Training details: The implementation follows a COCO setting with GroupNorm, Focal Loss, GIoU Loss, trainable regression scalars, and K=9 candidate anchors.The method does not use centerness prediction or center sampling.
  • Evaluation design: Comparisons with previous methods use various backbones and longer training than the ablation studies.Ablations use 135k iterations, while comparisons use 180k iterations.

4.2 Ablation Studies

Ablations show that probabilistic anchor separation outperforms simpler fixed rules, while IoU prediction, unified scoring, and score voting provide additional gains.

  • Anchor separation: PAA outperforms fixed numbers of positives and fixed positive score ranges in anchor-separation experiments.FSR with a threshold above 0.3 fails early because the model cannot find anchors within the specified score range.
  • Anchor separation: PAA adaptively determines separation boundaries without hyperparameters requiring careful hand-tuning.This adaptability is presented as an advantage over fixed separation rules.
  • Individual modules: Changing from IoU-based hard assignment to PAA improves AP by 5.3%.The reported ablation measures the change in AP after replacing the assignment strategy.
  • Individual modules: Adding an IoU prediction head and unified NMS score raises performance to 40.8%, while score voting further increases it to 41.0%.The unified score combines classification and localization quality; centerness does not improve PAA in this comparison.
  • Individual modules: IoU prediction is more effective than centerness prediction for ATSS, achieving 39.8% versus 39.4%.The comparison is reported for ATSS on the same evaluation setting.

Effects of individual modules

IoU prediction remains accurate across tested backbones, supporting its use as an additional localization-quality signal.

  • Accuracy of IoU prediction: Average IoU-prediction errors are below 0.1 for all evaluated backbones.The authors report this result as evidence that IoU prediction is plausible with an additional convolutional head.

Accuracy of IoU prediction

PAA’s positive and negative assignments become more distinct during training, but their separation is not determined solely by anchor IoU.

  • Sample visualization: Positive anchors do not necessarily have larger IoUs with the target box than negative anchors.At 30k and 50k iterations, many negative anchors still have high IoU values.
  • Sample visualization: As training proceeds, the distinction between PAA’s positive and negative samples becomes clearer.The visualization tracks this change across training iterations.

Visualization of anchor assignment

PAA’s anchor assignments become more clearly separated during training, while its positive-sample count varies adaptively across iterations rather than following a fixed or linear schedule.

  • PAA’s positive-sample counts are averaged over a GPU for visualization, while individual values range from 1 to 40.
  • Compared with PAA, FreeAnchor penalizes many candidate anchors except the single best one, which can confuse training.
  • PAA selects a different number of positive samples at each iteration, unlike fixed-count or linearly decaying approaches.

Statistics of positive samples

The experiments compare PAA with prior methods across multiple backbones on COCO test-dev, reporting higher accuracy and new records in the tested settings.

  • 44.8% accuracy with Res101 surpasses the previous best 43.6% from methods using the same backbone.
  • With ResNext101, PAA reaches 46.6% single-scale and 49.4% multi-scale accuracy, exceeding previous bests of 45.9% and 47.0%.
  • Deformable-backbone models outperform ATSS counterparts by 1.1% and 1.3%.
  • The deformable ResNext152 models achieve 50.8% single-scale and 53.5% multi-scale accuracy.
  • The table reports COCO test-dev results, with asterisks denoting multi-scale testing and bold text marking the best comparable performance.

5 Conclusions

The paper concludes that probabilistic anchor assignment, IoU-based localization-quality prediction, and score voting improve dense object detection performance while aligning training and testing scores.

  • PAA assigns positives and negatives by likelihood optimization over a probability distribution conditioned on model-computed anchor scores.
  • IoU prediction supplies localization quality for a unified classification-and-localization score during NMS, addressing a training–testing objective discrepancy.
  • Score voting is presented as a simple post-processing method applicable to most dense object detectors.
  • Experiments report significantly improved detection performance and superiority over previous methods on COCO test-dev.

6 Appendix

The appendix describes training settings, the modified single-anchor architecture, robustness checks, and visualizations of assignment and detection behavior.

  • 6.1 Training Details: Training uses 8 GPUs with two images per GPU, frozen Batch Normalization parameters, ImageNet-pretrained backbones, and scheduled learning-rate decay.
  • 6.1 Training Details: Multiplying localization losses by predicted IoUs or centerness helps train faster and improves performance.
  • 6.2 Network architecture: The architecture modifies RetinaNet with one anchor per spatial location and an additional head that predicts IoUs instead of centerness.
  • 6.4 Ablation Studies: Ablations report that PAA is robust to different topk K values and default anchor scales.
  • 6.5 Visualization: The appendix visualizes anchor assignment during training on COCO and non-COCO images, alongside detection results on COCO minival.
  • 6.4 Ablation Studies: With default anchor scale 8 and feature-pyramid strides from 8 to 128, anchor sizes range from 64 to 1024.
Loading 2007.08103v2…