FoveaBox: Beyond Anchor-based Object Detector

Tao Kong; Fuchun Sun; Huaping Liu; Yuning Jiang; Lei Li; Jianbo Shi

Source-linked AI summary

FoveaBox: Beyond Anchor-based Object Detector

Tao Kong, Fuchun Sun, Huaping Liu, Yuning Jiang, Lei Li, Jianbo Shi

arXiv:1904.03797v2Apr 8, 2019cs.CV

TL;DR

Anchor-based detectors depend on predefined boxes whose design and matching can limit generality and introduce hyper-parameters. FoveaBox instead predicts category-sensitive object possibility and category-agnostic box boundaries directly, using feature pyramids, and reports state-of-the-art single-model results on COCO and Pascal VOC.

Problem
Anchor-based detectors rely on predefined locations, scales, aspect ratios, and IoU matching, making performance and generalization sensitive to anchor design.
Method
FoveaBox uses a backbone and two-branch fovea head to predict per-position category confidences and category-agnostic bounding boxes without anchors.
Results
FoveaBox achieves state-of-the-art single-model performance on COCO and Pascal VOC, including a 2.2 AP gain over RetinaNet on COCO.
Takeaways & Limitations
FoveaBox provides a simple anchor-free detection baseline that avoids anchor-related computation and hyper-parameters.

Abstract

from arXiv · show

We present FoveaBox, an accurate, flexible, and completely anchor-free framework for object detection. While almost all state-of-the-art object detectors utilize predefined anchors to enumerate possible locations, scales and aspect ratios for the search of the objects, their performance and generalization ability are also limited to the design of anchors. Instead, FoveaBox directly learns the object existing possibility and the bounding box coordinates without anchor reference. This is achieved by: (a) predicting category-sensitive semantic maps for the object existing possibility, and (b) producing category-agnostic bounding box for each position that potentially contains an object. The scales of target boxes are naturally associated with feature pyramid representations. In FoveaBox, an instance is assigned to adjacent feature levels to make the model more accurate.We demonstrate its effectiveness on standard benchmarks and report extensive experimental analysis. Without bells and whistles, FoveaBox achieves state-of-the-art single model performance on the standard COCO and Pascal VOC object detection benchmark. More importantly, FoveaBox avoids all computation and hyper-parameters related to anchor boxes, which are often sensitive to the final detection performance. We believe the simple and effective approach will serve as a solid baseline and help ease future research for object detection. The code has been made publicly available at https://github.com/taokong/FoveaBox .

I. INTRODUCTION

FoveaBox addresses limitations of anchor-based detection by directly predicting object presence and box boundaries without predefined anchors. Combined with feature pyramids, it achieves strong COCO results while simplifying training and inference.

Anchor-based detectors require dataset-dependent anchor design, IoU-based matching, and anchor targets for training.
FoveaBox is a completely anchor-free framework that predicts object locations and boundaries directly from visual feature maps.
For each potentially occupied position, FoveaBox predicts category confidences and a bounding box, with training targets generated directly from ground-truth boxes.
Feature pyramid networks provide the multiscale detection framework, avoiding the fixed-scale image pyramids required by DenseBox.
2.2 AP gains over anchor-based RetinaNet on COCO are reported without bells and whistles.

II. RELATED WORK

Related work spans two-stage proposal-driven and one-stage proposal-free anchor-based detectors, whose common design relies on anchors for box prediction.

Anchor-based object detectors are broadly divided into two-stage proposal-driven and one-stage proposal-free methods.
Faster R-CNN established a dominant two-stage framework combining a region proposal network with region-wise prediction.
Both detector families use anchors as regression references and classification candidates for generating proposals or final bounding boxes.
One-stage methods such as SSD place multiscale anchor boxes on convolutional feature maps to predict categories and box offsets efficiently.

B. Anchor-Free Explorations

Earlier anchor-free detectors explored grid, point, corner, and pixel-based representations, but FoveaBox combines direct position and boundary prediction with feature pyramids.

YOLOv1 predicts boxes from a 7 × 7 grid without anchors but suffers from low recall, motivating anchors in YOLOv2 and YOLOv3.
Other anchor-free methods represent objects using boundary points, deformable sample points, or per-pixel predictions with additional assignment or suppression mechanisms.
FoveaBox uses a backbone and fovea head, whose two branches perform per-pixel classification and position-wise box prediction.
Feature pyramid levels detect objects at specific scales, while FoveaBox directly predicts object possibility and boundaries at potentially occupied positions.
The comparison figure contrasts three anchors per position and IoU-based labels with ground-truth-defined samples and direct boundary prediction.

1) Object Occurrence Possibility:

FoveaBox constructs object-occurrence targets on feature pyramids by mapping each ground-truth box to a shrunk positive area. Each positive cell receives the target class label.

Each valid ground-truth box is mapped to a target feature-pyramid level using the level’s down-sample factor.
The positive region on the score map is designed as a shrunk version of the original ground-truth box.
Cells inside the positive area receive the corresponding target class label, while the remaining feature-map area is negative.
The score branch outputs C category channels over an H×W map, with each channel representing class possibility as a binary mask.
Focal Loss trains the occurrence branch because the positive area usually occupies only a small portion of the feature map.

2) Scale Assignment:

FoveaBox assigns object scales across feature-pyramid levels rather than using a single level, allowing adjacent pyramids to predict each instance.

2) Scale Assignment:: On each FPN level, separate subnetworks classify cells and predict object-box coordinates, producing score maps and boxes before NMS.Score probability is represented by color density in the output map.
2) Scale Assignment:: An object may be detected by multiple adjacent pyramid levels instead of being assigned to only one feature pyramid.Adjacent levels are used to improve optimization and training stability.
2) Scale Assignment:: Adjacent-level assignment produces 3.1 AP gains compared with single-level training and prediction.

3) Box Prediction:

FoveaBox predicts normalized offsets from positive feature-map locations to all four ground-truth box boundaries, using direct ground-truth targets rather than anchor matching.

3) Box Prediction:: Each ground-truth box is represented by coordinates (x1, y1, x2, y2), and each positive point predicts offsets to its four boundaries.
3) Box Prediction:: The coordinate transformation maps each feature-map point to the input image and computes its normalized offset from the ground-truth box.The offsets are regularized in log space using the pyramid level’s basic scale rl.
3) Box Prediction:: Table I reports single-model detection results on COCO test-dev using an 800-pixel input scale, including FoveaBox-Align.
3) Box Prediction:: FoveaBox uses Smooth L1 loss for box prediction, with four output channels for tx1, ty1, tx2, and ty2.

4) Network Architecture:

FoveaBox combines an FPN backbone with a shared detection head on every pyramid level and evaluates multiple architectures under a simple training and inference setup.

4) Network Architecture:: The framework distinguishes the convolutional backbone from the network head and evaluates multiple architecture instantiations and head variants.More complex designs are left outside the work’s focus.
4) Network Architecture:: The FPN contains levels P3–P7 at progressively reduced resolutions, each with 256 channels and a shared Fovea head.
4) Network Architecture:: Training uses synchronized SGD on four GPUs with 16 images per minibatch, a 12-epoch schedule, horizontal flipping, and standard weight decay and momentum.
4) Network Architecture:: Inference applies a confidence threshold, keeps 1000 scoring boxes per prediction layer, performs class-wise NMS at 0.5, and selects the top 100 predictions per image.Additional post-processing and test-time augmentation are omitted for simplicity and fair baseline comparison.
4) Network Architecture:: FoveaBox can generate region proposals through a class-agnostic model head, but applying this to improve two-stage detectors is beyond the paper’s focus.
4) Network Architecture:: Experiments cover bounding-box detection on MS COCO and Pascal VOC, with ablations evaluated on COCO minival and main results reported on COCO test-dev.
4) Network Architecture:: All FoveaBox instantiations outperform baseline variants of previous state-of-the-art models in the reported comparison.
4) Network Architecture:: With ResNet-101, FoveaBox outperforms all single-stage detectors across the reported evaluation metrics and surpasses most two-stage detectors.The comparison includes CornerNet, ExtremeNet, FPN, Mask R-CNN, and IoU-Net.

B. Ablation Study

The ablation study examines qualitative detection, anchor density, and FoveaBox’s anchor-free design. FoveaBox predicts accurate boxes without candidate anchors while reducing output-space and optimization ambiguities.

Qualitative Results: FoveaBox produces predicted boxes close to ground truth despite multiple active points for each object.The qualitative outputs use points and boxes with class probability above 0.5 before NMS.
Various Anchor Densities and FoveaBox: Anchor-based detectors face ambiguous position definitions because multiple anchors at one location share features while receiving different labels.Over-dense anchors also increase foreground-background optimization difficulty.
Various Anchor Densities and FoveaBox: FoveaBox predicts one target at each position, reducing the output space to 1/A of the anchor-based method and simplifying optimization.The method also avoids the ambiguity caused by assigning different anchor labels at the same position.
Various Anchor Densities and FoveaBox: FoveaBox improves qualitative results for classes with large aspect ratios compared with RetinaNet.The comparison is shown in Fig. 6.
Various Anchor Densities and FoveaBox: FoveaBox is more flexible because it avoids extensive anchor design when selecting a relatively better configuration.This design choice is presented as an advantage over anchor-based schemes.

3) FoveaBox Is More Robust to Box Distribution:

FoveaBox is evaluated across aspect-ratio groups, proposal generation, and model configurations. The reported results indicate robustness to box distributions and strong region-proposal performance.

FoveaBox Is More Robust to Box Distribution: FoveaBox’s aspect-ratio evaluation divides validation boxes into three groups using their ground-truth aspect ratios.Performance is compared with RetinaNet at different aspect-ratio thresholds.
FoveaBox Is More Robust to Box Distribution: FoveaBox outperforms RetinaNet as ground-truth aspect-ratio thresholds increase, although both methods perform best at lower thresholds.The comparison is reported across aspect-ratio groups in Table III.
Experimental Settings: The feature-alignment and group-normalization configuration is evaluated using ResNet-50 with an 800-scale input.These settings correspond to the configuration named in the experiment tables.
Generating High-Quality Region Proposals: 52.9 AR with the top 100 region proposals outperforms the RPN baseline by 8.4 points.This result is reported on the minival set for the class-agnostic proposal variant.

5) Across Model Depth and Scale:

Across backbones, input resolutions, feature alignment, and label-assignment choices, FoveaBox maintains strong performance and supports faster inference than RetinaNet counterparts.

Across Model Depth and Scale: 0.9 ∼1.4 higher AP is achieved by FoveaBox across different backbone networks and input resolutions under the same baseline settings.The corresponding FoveaBox models are also about 1.1 ∼1.3 times faster than RetinaNet counterparts.
Label Assignment: +0.4 AP is obtained by the shrunk label-assignment version compared with IoU-based assignment.The comparison is reported in Table IX.
Better Head and Feature Alignment: 40.1 AP is achieved with ResNet-50 when using a heavier head together with feature alignment and group normalization.The experiment is presented as evidence of generality to network design.
Better Head and Feature Alignment: Using the last box-branch layer gives the best AP, while front features improve speed in feature-alignment experiments.Table X compares performance and speed across different box-branch layers.
Across Model Depth and Scale: FoveaBox improves AP in most COCO classes over RetinaNet with the same ResNet-50-FPN backbone and 800 input scale.The vertical axis in Fig. 8 is AP_FoveaBox-AP_RetinaNet.
Across Model Depth and Scale: Inference-time comparisons are averaged over 10 runs on a single Nvidia V100 GPU.The reported speed field can differ from actual runtime because implementations differ.

C. Pascal VOC Dataset

The Pascal VOC experiments evaluate FoveaBox on 20 object categories using mAP at IoU = 0.5. The paper presents these experiments as evidence of effectiveness on a standard benchmark.

C. Pascal VOC Dataset: Pascal VOC performance is measured by mean average precision at IoU = 0.5 across 20 object categories.Models are trained on VOC2007 trainval and VOC2012 trainval and tested on VOC2007 test.
C. Pascal VOC Dataset: FoveaBox is evaluated against RetinaNet using ResNet-50 on the Pascal VOC dataset.The comparison uses the training and testing protocol described for the VOC experiments.
C. Pascal VOC Dataset: The paper concludes that FoveaBox is an effective, simple, and completely anchor-free framework for generic object detection.Its effectiveness is demonstrated on standard benchmarks with extensive experimental analysis.