PaperlayerView exact revision on arXiv

Source-linked AI summary

FCOS: Fully Convolutional One-Stage Object Detection

Zhi Tian, Chunhua Shen, Hao Chen, Tong He

arXiv:1904.01355v5cs.CV

TL;DR

Object detection has largely relied on sensitive, complex anchor-box designs. FCOS instead performs anchor-free, proposal-free per-pixel detection and achieves state-of-the-art one-stage performance with substantially less design complexity.

  • Problem

    Anchor-based detectors depend on sensitive anchor-box sizes, aspect ratios, and counts, motivating per-pixel object detection analogous to semantic segmentation.

  • Method

    FCOS performs anchor-free, proposal-free per-pixel detection with multi-level prediction and a center-ness branch to suppress low-quality boxes.

  • Results

    FCOS achieves state-of-the-art performance among one-stage detectors while avoiding anchor-box computation and hyper-parameters.

  • Takeaways & Limitations

    FCOS provides a simpler, flexible alternative to mainstream anchor-based object detectors and unifies detection with dense prediction tasks.

  • Takeaways & Limitations

    The reported configuration is not optimal, as later work showed improved MS-COCO AP when center-ness is placed parallel to the regression branch.

Abstract

from arXiv · show

We propose a fully convolutional one-stage object detector (FCOS) to solve object detection in a per-pixel prediction fashion, analogue to semantic segmentation. Almost all state-of-the-art object detectors such as RetinaNet, SSD, YOLOv3, and Faster R-CNN rely on pre-defined anchor boxes. In contrast, our proposed detector FCOS is anchor box free, as well as proposal free. By eliminating the predefined set of anchor boxes, FCOS completely avoids the complicated computation related to anchor boxes such as calculating overlapping during training. More importantly, we also avoid all hyper-parameters related to anchor boxes, which are often very sensitive to the final detection performance. With the only post-processing non-maximum suppression (NMS), FCOS with ResNeXt-64x4d-101 achieves 44.7% in AP with single-model and single-scale testing, surpassing previous one-stage detectors with the advantage of being much simpler. For the first time, we demonstrate a much simpler and flexible detection framework achieving improved detection accuracy. We hope that the proposed FCOS framework can serve as a simple and strong alternative for many other instance-level tasks. Code is available at:Code is available at: https://tinyurl.com/FCOSv1

1. Introduction

The introduction motivates replacing anchor-based object detection with fully convolutional, per-pixel prediction. FCOS is presented as an anchor-free and proposal-free framework that uses FPN to reduce assignment ambiguity and center-ness to suppress low-quality detections.

  • Motivation: Anchor-based detectors require carefully tuned hyperparameters and struggle with large shape variations, especially for small objects, limiting generalization across tasks.Their fixed scales and aspect ratios can require redesigning anchors for detection tasks with different object sizes or aspect ratios.
  • Motivation: FCOS asks whether object detection can adopt the fully convolutional, per-pixel prediction paradigm used by semantic segmentation and other dense prediction tasks.This aims to bring object detection closer to a unified framework for fundamental vision tasks.
  • Method: FCOS predicts a 4D bounding-box vector and a class category at each feature-map location, analogous to FCN-based detection frameworks.The 4D vector represents relative offsets from the location to the four sides of a bounding box.
  • Method: FPN largely eliminates location-to-box assignment ambiguity, while a center-ness branch suppresses low-quality detections from locations far from object centers.The introduction states that FPN enables comparable accuracy to traditional anchor-based detectors before adding center-ness to suppress such detections.
  • Advantages: The resulting framework unifies detection with FCN-solvable tasks and removes proposals and anchor boxes, reducing design parameters and simplifying training.The introduction emphasizes that anchor-free and proposal-free detection avoids heuristic tuning and many performance-oriented tricks.

2. Related Work

Prior detectors either use predefined anchor boxes with numerous shape and labeling hyper-parameters or adopt anchor-free strategies with recall and post-processing limitations. FCOS addresses these limitations with multi-level FPN prediction and a center-ness branch, enabling simpler generic detection.

  • Anchor-based Detectors: Anchor-based detectors treat predefined anchor boxes as sliding windows or proposals, classifying them and refining locations with bounding-box offset regression.Anchor labels are often assigned using intersection over union with ground-truth boxes, adding positive, ignored, and negative-sample hyper-parameters.
  • Anchor-based Detectors: Anchor-based methods require carefully tuned hyper-parameters for anchor shapes and sample-label assignment, complicating detector design.These parameters include anchor dimensions and IOU thresholds used to determine positive, ignored, or negative anchors.
  • Anchor-free Detectors: YOLOv1 predicts boxes only at points near object centers, but this design suffers from low recall, motivating YOLOv2 to employ anchor boxes.Center-near points are selected because they are considered more likely to produce higher-quality detections.
  • Anchor-free Detectors: CornerNet detects and groups bounding-box corner pairs, requiring more complicated post-processing and an additional learned distance metric.The grouping step associates corner pairs belonging to the same instance.
  • Anchor-free Detectors: DenseBox-based anchor-free detectors have difficulty with overlapping boxes and relatively low recall, while multi-level FPN prediction and a center-ness branch largely alleviate both problems.The paper presents these components as enabling a simpler detector to achieve better detection performance.

3. Our Approach

FCOS reformulates detection as per-pixel prediction without anchor boxes, directly assigning locations classification labels and bounding-box regression targets. Multi-level prediction addresses recall and overlap ambiguity, while a centerness branch suppresses low-quality boxes and improves performance.

  • Per-pixel prediction: FCOS directly treats feature-map locations as training samples, predicting class labels and distances to the four sides of a ground-truth box instead of using anchor boxes.Locations inside ground-truth boxes are positive samples; their regression targets are the distances to the left, top, right, and bottom sides.
  • Multi-level prediction: Multi-level prediction addresses the low recall caused by large feature-map strides and greatly reduces ambiguity from overlapping ground-truth boxes.The paper reports that FCN-based detection can achieve performance on par with, and sometimes better than, anchor-based detectors.
  • Multi-level prediction: FCOS uses five FPN levels, P3 through P7, to detect different object sizes and assigns each level a bounded regression range.Locations whose maximum regression target falls outside a level’s assigned range become negative samples and need not regress a box.
  • Centerness: A single-layer centerness branch predicts how close each location is to its responsible object’s center, suppressing low-quality boxes without new hyper-parameters.At test time, predicted centerness multiplies the classification score, down-weighting boxes produced far from object centers.

4. Experiments

Experiments on MS-COCO evaluate FCOS’s recall, ambiguity, center-ness, and detection performance against anchor-based and other detectors. FCOS achieves competitive or superior results while using the same or closely matched settings as RetinaNet.

  • Experimental Setup: Experiments use COCO trainval35k for training, minival for ablations, and test-dev for main results.The splits contain 115K, 5K, and 20K images, respectively.
  • Ablation Studies: With FPN, ambiguous samples decrease from 23.16% to 7.14% because overlapping objects are assigned to different feature levels.The reported percentage of ambiguous samples is 3.75% when excluding same-category ambiguities.
  • Ablation Studies: 37.1% AP is achieved with center-ness, up from 33.5%, outperforming anchor-based RetinaNet at 35.9%.The center-ness branch suppresses low-quality boxes predicted from locations far from object centers.
  • Comparison with RetinaNet: 36.3% vs 35.9% AP shows FCOS remains favorable to RetinaNet under exactly the same settings and without Group Normalization.The comparison uses C5 and RetinaNet’s hyper-parameters, including learning rate and NMS threshold.

5. Extensions on Region Proposal Networks

FCOS extends to two-stage detection by replacing anchor boxes in Faster R-CNN’s RPN with an FPN-based, anchor-free method. Experiments show significant improvements in proposal recall, further increased by the center-ness branch.

  • Extension to RPNs: FCOS replaces anchor boxes in FPN-based RPNs for Faster R-CNN, confirming its applicability beyond one-stage detection.The extension is evaluated experimentally against RPNs with FPN.
  • Experimental setup: The RPN comparison keeps all other settings identical to the official FPN-based RPN implementation, while adding GN to stabilize FPN-head training.The comparison also replaces the anchor boxes with the FCOS method.
  • Results: Even without center-ness, FCOS significantly improves both AR100 and AR1k over RPNs with FPN.The passage reports significant gains for both proposal-recall metrics before adding the center-ness branch.
  • Results: 52.8% AR100 and 60.3% AR1k are achieved with the center-ness branch, further boosting both metrics.These values are reported after adding the proposed center-ness branch.

6. Conclusion

FCOS is an anchor-free and proposal-free one-stage detector that performs object detection through per-pixel prediction. Experiments show favorable comparisons with popular anchor-based one-stage detectors while reducing design complexity and eliminating anchor-box computation and hyperparameters.

  • Conclusion: FCOS is proposed as an anchor-free and proposal-free one-stage object detector.The framework removes both predefined anchor boxes and proposals.
  • Conclusion: FCOS compares favorably with RetinaNet, YOLO, and SSD while requiring much less design complexity.These are identified as popular anchor-based one-stage detectors in the experiments.
  • Conclusion: FCOS avoids all anchor-box-related computation and hyperparameters by solving detection through per-pixel prediction, similar to semantic segmentation.This casts object detection as a dense prediction task.

Appendix · 7. Class-agnostic Precision-recall Curves

The appendix evaluates class-agnostic precision-recall behavior for FCOS and RetinaNet at IoU thresholds of 0.50, 0.75, and 0.90. FCOS outperforms RetinaNet, with a larger advantage at stricter IoU thresholds, while both detectors’ best recalls remain well below 90%.

  • 7. Class-agnostic Precision-recall Curves: FCOS’s improvement over RetinaNet becomes larger as the IoU threshold increases.This pattern indicates stronger performance under stricter localization requirements.
  • 7. Class-agnostic Precision-recall Curves: The evaluation presents class-agnostic precision-recall curves on split minival.Curves are shown in Figures 4, 5, and 6.
  • 7. Class-agnostic Precision-recall Curves: The curves use IoU thresholds of 0.50, 0.75, and 0.90, respectively.Table 7 reports the AP values corresponding to these three curves.
  • 7. Class-agnostic Precision-recall Curves: FCOS achieves better class-agnostic detection performance than the anchor-based RetinaNet.The comparison uses the same models as Table 4 in the main paper.
  • 7. Class-agnostic Precision-recall Curves: The larger high-IoU improvement suggests that FCOS has a more accurate bounding-box regressor.The paper attributes this in part to FCOS leveraging more foreground samples to train the regressor.
  • 7. Class-agnostic Precision-recall Curves: The detectors’ best recalls in the precision-recall curves are much lower than 90%.Thus, the small BPR difference between FCOS and RetinaNet is unlikely to affect final detection performance substantially.
  • 7. Class-agnostic Precision-recall Curves: 98.40% vs. 99.23% is the reported gap in best possible recall between FCOS and RetinaNet.The passage states that this small BPR gap hardly harms final detection performance.

8. Visualization for Center-ness

The center-ness branch suppresses low-quality bounding boxes by down-weighting scores before post-processing, improving detection precision. Visualization shows that high-confidence boxes with low IOU are filtered more effectively after center-ness is applied.

  • Effect: The center-ness branch improves detection performance by suppressing low-quality detected bounding boxes.The section confirms the large performance margin attributed to this suppression effect.
  • Mechanism: Center-ness down-weights low-quality bounding-box scores so post-processing such as NMS can filter them out.Low-quality boxes have low IOU with their corresponding ground-truth boxes; retaining them can harm precision.
  • Visualization: Before center-ness, many low-IOU bounding boxes have high confidence scores and cannot be eliminated during post-processing.These boxes appear as points under the line y = x and lower detector precision.

9. Qualitative Results

Qualitative results show that FCOS detects a wide range of objects, including crowded, occluded, highly overlapped, extremely small, and very large objects.

  • FCOS detects crowded, occluded, highly overlapped, extremely small, and very large objects.These qualitative results are shown in Fig. 8.

10. More discussions

The discussion clarifies FCOS’s center-ness design, including how it suppresses low-quality boxes, handles assignment ambiguity, and differs from IoUNet and anchor-based RetinaNet. It also reports that FCOS’s recall upper bound nearly matches RetinaNet’s and that center-ness enables broader regressor supervision during training.

  • Center-ness: Center-ness is simpler than IoUNet because it uses one jointly trained detector layer rather than a separate network predicting box–ground-truth IoU.The passage states that both approaches aim to suppress low-quality predictions.
  • Recall and regression targets: 98.4% vs. 99.23%: FCOS’s recall upper bound is reported as close to anchor-based RetinaNet’s upper bound.The comparison is presented to clarify Table 1’s purpose, not to compare different recall definitions.
  • Assignment ambiguity: During training, FCOS resolves same-level assignment ambiguity by choosing the ground-truth box with minimal area.At test time, overlapping same-class objects remain correct if a location predicts either object, while different-class overlaps can produce class–box mismatches.
  • Center-ness: Center-ness suppresses low-quality detections by substantially reducing their scores after classification scores are multiplied by center-ness scores.Figure 7 shows low-quality boxes below y = x being pushed to the plot’s left side.
  • Anchor-based comparison: Center-ness cannot directly support RetinaNet with multiple anchors per location because one feature-map location has only one score for anchors requiring different center-ness values.For anchor-based RetinaNet, anchor–ground-truth IoU may instead serve as an alternative.
  • Training and testing: During training, all locations inside ground-truth boxes are positive, allowing FCOS to use more foreground locations for regressor training and produce more accurate bounding boxes.Center-ness is applied only during testing.
Loading 1904.01355v5…