Source-linked AI summary
Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection
Shifeng Zhang, Cheng Chi, Yongqiang Yao, Zhen Lei, Stan Z. Li
TL;DR
The paper asks whether the performance gap between anchor-based and anchor-free detectors comes from their regression representation or from training-sample definitions. It proposes ATSS, which selects samples from object statistics, and reports that it bridges the gap while achieving state-of-the-art performance without overhead.
Problem
The paper investigates why anchor-based and anchor-free detectors differ in performance, focusing on whether their positive and negative sample definitions are the essential distinction.
Method
ATSS automatically selects positive and negative training samples according to the statistical characteristics of each object.
Results
ATSS bridges the gap between anchor-based and anchor-free detectors and achieves state-of-the-art AP 50.7% on MS COCO without additional overhead.
Takeaways & Limitations
Appropriate positive-sample selection can make detector performance comparable regardless of whether regression starts from a box or a point.
Takeaways & Limitations
The paper concludes that multiple anchors per location may be unnecessary under ATSS and says the operation needs further study.
Abstract
from arXiv · showhide
Object detection has been dominated by anchor-based detectors for several years. Recently, anchor-free detectors have become popular due to the proposal of FPN and Focal Loss. In this paper, we first point out that the essential difference between anchor-based and anchor-free detection is actually how to define positive and negative training samples, which leads to the performance gap between them. If they adopt the same definition of positive and negative samples during training, there is no obvious difference in the final performance, no matter regressing from a box or a point. This shows that how to select positive and negative training samples is important for current object detectors. Then, we propose an Adaptive Training Sample Selection (ATSS) to automatically select positive and negative samples according to statistical characteristics of object. It significantly improves the performance of anchor-based and anchor-free detectors and bridges the gap between them. Finally, we discuss the necessity of tiling multiple anchors per location on the image to detect objects. Extensive experiments conducted on MS COCO support our aforementioned analysis and conclusions. With the newly introduced ATSS, we improve state-of-the-art detectors by a large margin to $50.7\%$ AP without introducing any overhead. The code is available at https://github.com/sfzhang15/ATSS
1. Introduction
The paper argues that positive/negative sample definition, rather than box-versus-point regression, explains the performance gap between anchor-based and anchor-free detectors. It proposes ATSS to select samples adaptively and reports state-of-the-art MS COCO performance without additional overhead.
- Detector comparison: Anchor-based detectors tile preset boxes and refine them, whereas anchor-free detectors locate objects through keypoints or centers without preset anchors.Center-based anchor-free detectors are especially similar to anchor-based detectors because points act as preset samples instead of anchor boxes.
- Core analysis: The comparison between RetinaNet and FCOS isolates anchor count, sample selection, and regression starting status as candidate sources of their performance gap.The paper investigates these differences while ruling out implementation inconsistencies.
- Core analysis: Anchor-based and anchor-free detectors differ essentially in how they define positive and negative training samples.When they use the same sample definitions, the paper reports no obvious final-performance gap regardless of whether regression starts from a box or point.
- Core contribution: ATSS automatically selects positive and negative samples according to object statistical characteristics.The method is introduced to bridge the performance gap between anchor-based and anchor-free detectors.
- Discussion: Tiling multiple anchors per location is presented as unnecessary under the proposed method.The paper treats this conclusion as supported by experiments and identifies it as a topic for further study.
- Results: State-of-the-art performance is achieved on MS COCO without introducing additional overhead.The introduction lists this as a contribution of the proposed approach.
2. Related Work
Related work spans anchor-based detectors and two main families of anchor-free detectors. The paper positions center-based anchor-free methods as especially comparable to anchor-based detectors because both use location-associated samples for classification and regression.
- Taxonomy: CNN-based object detection includes anchor-based detectors and anchor-free detectors, with anchor-based methods divided into two-stage and one-stage families.Anchor-free methods are categorized into keypoint-based and center-based families.
- Anchor-based methods: Two-stage anchor-based detection uses a region proposal network followed by region-wise prediction.Faster R-CNN established the dominant position of two-stage anchor-based detectors, whose refinements include architecture, context, training, and proposal improvements.
- Anchor-free methods: Keypoint-based anchor-free methods locate predefined or learned keypoints and generate bounding boxes from them.Examples include CornerNet, CornerNet-Lite, and grid-based or center-based localization approaches.
- Anchor-free methods: Center-based methods define positives around object centers and predict distances to object boundaries.DenseBox and related methods use center regions or pixels as positives for localization and box prediction.
3. Difference Analysis of Anchor-based and Anchor-free Detection
The experiments align RetinaNet and FCOS implementations to isolate sample selection and regression starting status. Results show that positive/negative sample definitions account for the meaningful performance difference, while box-versus-point regression is not decisive when sampling is matched.
- 3.1. Experiment Setting: The study uses RetinaNet with one square anchor per location and FCOS under aligned MS COCO experimental settings.Experiments use MS COCO with a ResNet-50 and five-level feature pyramid; RetinaNet uses one square anchor per location.
- 3.1. Experiment Setting: The analysis separately examines the number of anchors tiled per location, which is excluded from the initial one-anchor comparison.The original RetinaNet uses nine anchors per location, while the analyzed RetinaNet configuration uses one square anchor.
- 3.2. Inconsistency Removal: 37.1% vs. 32.5% AP separates FCOS from RetinaNet (#A=1) before implementation improvements are aligned.Subsequent universal improvements raise RetinaNet (#A=1) to 37.0% AP and FCOS to 37.8% AP, leaving a 0.8% gap.
- 3.3. Essential Difference: RetinaNet selects positives with IoU across spatial and scale dimensions, whereas FCOS applies spatial and scale constraints sequentially.Figure 1 contrasts direct IoU selection with FCOS candidate-positive selection followed by scale-based final-positive selection.
- 3.3. Essential Difference: Using spatial and scale constraints raises RetinaNet AP from 37.0% to 37.8%, while using IoU lowers FCOS AP from 37.8% to 36.9%.These changes support sample-definition strategy as an essential difference between the detectors.
- 3.3. Essential Difference: RetinaNet regresses from an anchor box with four offsets, while FCOS regresses from an anchor point with four distances to object boundaries.The comparison concerns the regression starting status after positive and negative samples have been determined.
4. Adaptive Training Sample Selection
ATSS adaptively selects positive and negative samples from object-specific statistics, reducing sensitivity to fixed selection rules. It improves both anchor-based and anchor-free detectors while showing that multiple anchors per location are unnecessary when sample selection is appropriate.
- Description: ATSS replaces sensitive fixed IoU thresholds and scale ranges with nearly hyperparameter-free selection based on each object's statistical characteristics.The method is intended to avoid neglecting outer objects caused by fixed rules.
- Description: ATSS selects candidate anchors near each ground-truth center on every pyramid level, then sets an object-specific IoU threshold as mean plus standard deviation.Candidates with IoU at least tg and centers inside the ground-truth box become final positives.
- Description: About 0.2 × kL positive samples are selected per object, remaining approximately invariant to scale, aspect ratio, and location.RetinaNet and FCOS tend to assign more positive samples to larger objects, creating unfairness between objects.
- Verification: 2.3% AP improvement is obtained for anchor-based RetinaNet, with gains across AP50, AP75, APS, APM, and APL without additional overhead.The reported gains are 2.4% AP50, 2.9% AP75, 2.9% APS, 2.1% APM, and 2.7% APL.
- Verification: ATSS improves full-version FCOS by 1.4% AP, while full ATSS outperforms lite center sampling across the reported metrics.The full version adaptively selects positives along the scale dimension, whereas the lite version retains scale-range hyperparameters.
- Discussion: Under ATSS, changing the number of anchor scales or aspect ratios per location leaves results almost unchanged, unlike traditional IoU-based selection.With traditional selection, tiling more anchors improves RetinaNet; with ATSS, RetinaNet configurations using different anchor counts perform similarly.
5. Conclusion
The paper identifies training-sample definition as the essential difference between anchor-based and center-based anchor-free detectors and proposes adaptive selection to address it. It also questions the usefulness of tiling multiple anchors per location, while achieving state-of-the-art performance without additional overhead.
- The paper identifies positive and negative training-sample definitions as the essential difference between anchor-based and center-based anchor-free detectors.
- Adaptive Training Sample Selection automatically divides positive and negative training samples according to statistical characteristics of objects.
- The authors question the usefulness of tiling multiple anchors per location for object detection.
- Extensive MS COCO experiments show state-of-the-art performance without introducing additional overhead.