Source-linked AI summary
Soft Anchor-Point Object Detection
Chenchen Zhu, Fangyi Chen, Zhiqiang Shen, Marios Savvides
TL;DR
Anchor-point detectors are faster than key-point detectors but have lagged in accuracy, motivating better training. SAPD jointly optimizes anchor points within and across pyramid levels using soft weighting and soft level selection, achieving a 47.4% single-model single-scale AP on COCO while advancing the speed-accuracy trade-off.
Problem
Anchor-point detectors have a speed advantage over key-point detectors but traditionally cannot match their accuracy because of ineffective training within and across feature pyramid levels.
Method
SAPD jointly optimizes anchor points using soft-weighted anchor points and soft-selected pyramid levels to address false attention and feature selection.
Results
47.4% AP is achieved by the accurate SAPD variant on single-model single-scale COCO evaluation, surpassing TridentNet's 46.8% while running more than 3× faster.
Takeaways & Limitations
The training strategy creates a new upper envelope of the speed-accuracy trade-off for anchor-point detection.
Abstract
from arXiv · showhide
Recently, anchor-free detection methods have been through great progress. The major two families, anchor-point detection and key-point detection, are at opposite edges of the speed-accuracy trade-off, with anchor-point detectors having the speed advantage. In this work, we boost the performance of the anchor-point detector over the key-point counterparts while maintaining the speed advantage. To achieve this, we formulate the detection problem from the anchor point's perspective and identify ineffective training as the main problem. Our key insight is that anchor points should be optimized jointly as a group both within and across feature pyramid levels. We propose a simple yet effective training strategy with soft-weighted anchor points and soft-selected pyramid levels to address the false attention issue within each pyramid level and the feature selection issue across all the pyramid levels, respectively. To evaluate the effectiveness, we train a single-stage anchor-free detector called Soft Anchor-Point Detector (SAPD). Experiments show that our concise SAPD pushes the envelope of speed/accuracy trade-off to a new level, outperforming recent state-of-the-art anchor-free and anchor-based detectors. Without bells and whistles, our best model can achieve a single-model single-scale AP of 47.4% on COCO.
1 Introduction
Anchor-free detectors avoid anchor boxes, but anchor-point methods traditionally trail key-point methods in accuracy despite their speed advantages. SAPD addresses ineffective training with soft-weighted anchor points and soft-selected pyramid levels, improving the speed-accuracy trade-off.
- Anchor-free detectors simplify architecture and reduce training memory, while anchor-point detectors additionally offer faster training and inference than key-point detectors.
- SAPD identifies false attention within pyramid levels and feature selection across levels as the main training issues limiting anchor-point detectors.
- Soft-weighted anchor points reweight loss contributions according to geometric relation to instance boundaries, reducing attention to poorly aligned points.
- Soft-selected pyramid levels learn instance-dependent participation weights for feature levels through a lightweight feature-selection network used during training.
- 2.1% AP improves the COCO ResNet-50 FSAF baseline without inference slowdown, while SAPD variants extend the speed-accuracy trade-off.
2 Related Work
Related work spans anchor-free detection and feature selection in detection. Prior methods use anchor points, key points, or scale-based and loss-based pyramid-level assignment to address object localization and scale variation.
- Anchor-free detectors include single-stage key-point methods such as CornerNet, ExtremeNet, and CenterNet, and anchor-point methods such as FSAF, FCOS, and FoveaBox.
- Feature pyramids address scale variation, making suitable feature-level selection important for detection.
- Anchor-based methods select levels implicitly through anchor matching, while most anchor-free approaches assign instances according to scale.
- FSAF dynamically assigns each instance to the pyramid level with minimal instance-dependent loss, but selects only one level per instance.
3 Soft Anchor-Point Detector
SAPD formulates anchor-point detection around joint optimization within and across feature-pyramid levels. Its training strategy reduces false attention among anchor points and performs soft, instance-dependent pyramid-level selection.
- SAPD is a concise single-stage anchor-point detector whose training strategy addresses false attention within levels and feature selection across levels.
- A vanilla detector uses a backbone, feature pyramid, and one classification-localization head per pyramid level.Each level predicts class probabilities and four distances from anchor points to nearby instance boundaries.
- Soft-Weighted Anchor Points: Equal treatment of anchor points can give poorly localized boxes high confidence, causing NMS to retain them and lowering strict-IoU AP.Boundary-near points have less aligned features because their receptive fields include more background.
- Soft-Weighted Anchor Points: Soft weighting reduces each positive anchor point’s loss contribution according to its geometric proximity to the instance boundaries.Closer boundary distance receives less attention weight because localization is harder there.
- Soft-Selected Pyramid Levels: Feature responses across adjacent pyramid levels are similar, while details fade at smaller resolutions, making single-level assignment waste network capacity.
- Soft-Selected Pyramid Levels: A feature-selection network uses RoIAlign features concatenated across pyramid levels to predict a probability distribution used as instance-specific soft-selection weights.The network is jointly trained with the detector using cross-entropy against the minimum-loss pyramid level.
4 Experiments
Experiments evaluate SAPD through ablations, comparisons with anchor-based and anchor-free detectors, and accuracy-speed measurements on COCO. The results show consistent gains from soft-weighted anchor points, soft-selected pyramid levels, augmented feature pyramids, and the complete SAPD design.
- Experimental setup: Ablations use COCO val2017 with ResNet-50 and examine soft-weighted anchor points, soft-selected pyramid levels, and augmented feature pyramids.The ablations are trained and tested at an image scale of 800 pixels.
- Ablation studies: Up to 1.1% AP improvement and 1.6% AP75 improvement show that soft-weighted anchor points improve localization over baseline FSAF.The method is insensitive to various hyperparameters and suppresses poorly localized boxes.
- Ablation studies: Approximately 1.0% absolute AP improvements show that soft-selected pyramid levels benefit from assigning each instance to multiple feature levels.The selected levels are the top 3 levels with minimal instance-dependent losses; larger instances tend to receive higher-level weights.
- Ablation studies: Softly selecting multiple levels, rather than joint training of the feature selection network alone, provides the major performance contribution.Joint training without using predicted weights reaches 37.1% AP, only 0.1% above the preceding configuration and 0.9% below the soft-selection configuration.
- Comparison to state of the art: SAPD increases AP by 1.6%, 1.7%, and 1.5% over anchor-based plus FSAF detectors on ResNet-50, ResNet-101, and ResNeXt-101-64x4d, respectively.The comparisons use the COCO val2017 set and the same GTX 1080Ti testing setup.
- Comparison to state of the art: 14.9 FPS at 41.7% AP is achieved by the fastest ResNet-50 SAPD without DCN, while the DCN version forms an upper envelope among compared detectors.Against the best key-point detector, CenterNet, SAPD reaches 47.4% versus 44.9% AP with up to 5× faster inference.
5 Conclusion
The conclusion identifies joint optimization of anchor points within and across feature pyramid levels as the key insight. The resulting training strategy addresses false attention and feature selection, producing a new speed-accuracy envelope.
- Conclusion: Joint optimization of anchor points within and across feature pyramid levels is the key insight of the work.The strategy addresses false attention within levels and feature selection across levels.
- Conclusion: Applying the training strategy to a simple anchor-point detector produces a new upper envelope of the speed-accuracy trade-off.
A Discussion
Additional experiments investigate the proposed SAPD training strategy beyond the main-paper ablations. These experiments use the same ablation setting and ResNet-50 backbones.
- Discussion: Additional experiments extend the main-paper ablations to further understand the proposed SAPD training strategy.
- Discussion: All additional models use the ResNet-50 backbone and follow the ablation-study setting from Section 4.1.
A.1 Soft-Weighting during Training or Testing?
The discussion compares soft-weighting during training with center-ness weighting during testing. Training-time loss reweighting is more effective because it directly reduces contributions from poorly aligned anchor points.
- Training versus testing: Training-time soft-weighting down-weights anchor points' contributions to network loss, whereas FCOS applies center-ness to confidence scores during testing.FCOS predicts center-ness masks from extra branches; SAPD uses the function to alter training loss contributions.
- Training versus testing: 1.1% AP gain from training-time soft-weighting exceeds the 0.6% gain from the best center-ness variant applied during testing.The comparison is performed on the baseline FSAF module.
- Training versus testing: Soft-weighting alone does not improve after adding center-ness weighting, indicating that reweighting anchor loss is closer to suppressing poorly localized detections than reshaping inference confidence.
A.2 Which Loss to Reweight?
The soft-weighting scheme performs best when applied jointly to classification and localization losses, rather than to either loss alone. Reweighting only one loss can bias optimization toward the other task and reduce detection performance.
- Applying soft-weighting to only the classification or localization loss performs worse than the baseline.Down-weighting one loss can cause the network to focus on optimizing the other unweighted loss.
- Soft-weighting both classification and localization losses provides more balanced optimization for detection.The detection problem requires both proper classification and localization abilities.
- The evaluated choices are classification-only, localization-only, and joint classification-plus-localization reweighting for soft-weighted anchor points.
B Visualization of Feature Selection Network
The feature selection network predicts instance-specific participation weights across pyramid levels, using multiple levels rather than selecting only one. These weights are visualized for levels P3 through P7 across varied object categories.
- The feature selection network predicts a participation degree for each pyramid level and instance.The selection is agnostic to instance class and applies across animals, humans, food, vehicles, furniture, and other objects.
- Using features from multiple pyramid levels is better than FSAF's online strategy, which assigns each instance to a single level during training.
- Figure 8 visualizes weights from 0 to 1 for pyramid levels P3 through P7 for each instance.More filled red bars indicate higher predicted weights.