Source-linked AI summary
FreeAnchor: Learning to Match Anchors for Visual Object Detection
Xiaosong Zhang, Fang Wan, Chang Liu, Rongrong Ji, Qixiang Ye
TL;DR
CNN-based detectors commonly assign objects to anchors using an IoU restriction, but this can fail for varied or crowded object geometries. FreeAnchor formulates anchor matching and detector training as maximum likelihood estimation with a detection-customized likelihood, and experiments on COCO report improved detection performance over counterpart detectors.
Problem
Hand-crafted IoU-based anchor assignment can fail for acentric, slender, or crowded objects, while a systematic way to model anchor-object correspondence during training is lacking.
Method
FreeAnchor formulates object-anchor matching as maximum likelihood estimation, selecting representative anchors from bags while jointly optimizing classification and localization.
Results
FreeAnchor significantly improved object-detection performance over the baseline detector and reached 47.3% AP with multi-scale testing.
Takeaways & Limitations
FreeAnchor provides a flexible, plug-and-play alternative to hand-crafted anchor assignment for CNN-based object detectors.
Abstract
from arXiv · showhide
Modern CNN-based object detectors assign anchors for ground-truth objects under the restriction of object-anchor Intersection-over-Unit (IoU). In this study, we propose a learning-to-match approach to break IoU restriction, allowing objects to match anchors in a flexible manner. Our approach, referred to as FreeAnchor, updates hand-crafted anchor assignment to "free" anchor matching by formulating detector training as a maximum likelihood estimation (MLE) procedure. FreeAnchor targets at learning features which best explain a class of objects in terms of both classification and localization. FreeAnchor is implemented by optimizing detection customized likelihood and can be fused with CNN-based detectors in a plug-and-play manner. Experiments on COCO demonstrate that FreeAnchor consistently outperforms their counterparts with significant margins.
1 Introduction
FreeAnchor replaces hand-crafted IoU-based anchor assignment with learning-based matching formulated as maximum likelihood estimation. It jointly optimizes classification and localization so objects can select anchors that best explain them.
- Motivation: IoU-based assignment can fail for acentric, slender, or crowded objects because spatial alignment may miss representative features or proper matches.A generic rule is difficult to design for objects with varied geometric layouts.
- Design objectives: The design also targets NMS compatibility by aligning higher classification scores with more accurate localization.Otherwise, accurately localized anchors with low classification scores could be suppressed.
- Approach: The method selects the most representative anchor from each anchor bag by defining its likelihood as the largest anchor confidence within the bag.Maximizing this likelihood ensures at least one anchor has high classification and localization confidence.
- Approach: FreeAnchor formulates detector training as maximum likelihood estimation and removes the IoU restriction on object-anchor matching.Objects flexibly select anchors under the principle of maximum likelihood.
- Approach: FreeAnchor jointly optimizes object classification and localization through a detection-customized likelihood in an end-to-end training mechanism.The resulting loss drives both CNN detector training and object-anchor matching.
2 Related Work
Prior detectors use anchors or anchor-free alternatives to address object-feature correspondence, but existing methods lack a systematic way to learn anchor-object matching during training. FreeAnchor instead lets each object flexibly choose an anchor from a bag.
- Anchor-based methods: Anchor-based detectors use anchors as reference points for localization, but rely on heuristics that spatially aligned anchors suit both classification and localization.Acentric features can cause detectors to miss the best anchors and features.
- Alternative approaches: Anchor-free approaches replace or reduce predefined-anchor dependence using pixel-level supervision, center-ness regression, key points, or semantic guidance.Examples include CornerNet, CenterNet, MetaAnchor, and GuidedAnchoring.
- Remaining gap: Existing methods still lack a systematic model of anchor-object correspondence during detector training, limiting feature-selection and feature-learning optimization.FreeAnchor addresses this correspondence through flexible matching from an anchor bag.
3 The Proposed Approach
FreeAnchor replaces fixed IoU-based anchor assignment with learned object-anchor matching formulated through maximum likelihood. Its customized likelihood and loss jointly address recall, precision, NMS compatibility, and gradual anchor selection during training.
- 3 The Proposed Approach: FreeAnchor formulates detector training as maximum likelihood estimation to jointly optimize object classification, localization, and object-anchor matching.The detection customized likelihood is converted into a loss optimized end-to-end.
- 3.2 Detection Customized Likelihood: For each object, FreeAnchor forms a candidate bag from top-ranked anchors and maximizes the highest joint classification-localization confidence in that bag.This maximum operation ensures at least one strong prediction while allowing other poorly localized anchors to be treated as background.
- 3.2 Detection Customized Likelihood: The matching probability increases monotonically with localization IoU and uses a saturated linear function to preserve compatibility with NMS.The likelihood combines recall and precision objectives while retaining the required matching behavior.
- 3.3 Anchor Matching Mechanism: The approach is implemented by converting detection customized likelihood into a loss and integrating it into CNN detector training with anchor-bag construction and gradient updates.Algorithm 1 selects n top-ranked anchors by IoU, calculates the loss, and updates network parameters with stochastic gradient descent.
- 3.3 Anchor Matching Mechanism: Mean-max matching uses nearly all anchors early in training and gradually approaches single-best-anchor selection as confidence improves.The loss replaces the max function with Mean-max, adds balance factors, and applies focal loss to the precision term.
4 Experiments
Experiments on COCO evaluate FreeAnchor as a RetinaNet-based detector, including its learning-to-match behavior, NMS compatibility, hyperparameters, and comparisons with baselines and one-stage detectors.
- Experimental setup: FreeAnchor was implemented by replacing RetinaNet’s loss with the proposed detection customized loss and evaluated on COCO 2017.Models used ResNet or ResNeXt backbones; results were evaluated on validation and test-dev sets.
- Model effect: FreeAnchor significantly outperformed RetinaNet on slender objects while reporting comparable performance on square objects.The reported explanation is that learning-to-match selects representative features rather than requiring spatial alignment.
- Model effect: As image crowdedness increased, FreeAnchor’s advantage over RetinaNet became more obvious.The authors attribute this to selecting more suitable anchors for objects in crowded scenes.
- NMS compatibility: FreeAnchor reported higher NMS recall than RetinaNet, indicating higher compatibility with NMS.NMS recall was averaged over IoU thresholds from 0.50 to 0.90 at intervals of 0.05.
- Detection performance: ~3.0% AP was the maximum improvement over the RetinaNet baseline in Table 2, with negligible training-time cost.The experiments used an anchor bag size of 50, background IoU threshold 0.6, α = 0.5, γ = 2.0, and β = 0.75.
- Detection performance: 44.9% vs. 44.9% AP was reported for FreeAnchor and CenterNet, while FreeAnchor achieved higher AP50 with a lighter backbone and fewer training iterations.FreeAnchor also outperformed RetinaNet and several anchor-free one-stage detectors in the comparison.
5 Conclusion
FreeAnchor replaces hand-crafted anchor assignment with learned object-anchor correspondence through maximum likelihood estimation, improving object detection performance over the baseline.
- FreeAnchor replaces hand-crafted anchor assignment with “free” object-anchor correspondence using maximum likelihood estimation.The method is designed to learn convolutional features that best explain object classes.
- The detection-customized likelihood facilitates learning convolutional features that best explain a class of objects.
- FreeAnchor significantly improves object detection performance in contrast with the baseline detector.