Source-linked AI summary
FCOS: A simple and strong anchor-free object detector
Zhi Tian, Chunhua Shen, Hao Chen, Tong He
TL;DR
Object detection commonly relies on anchor boxes whose hyper-parameters and training computations add design complexity, while FCN-style per-pixel detection faces scale and overlap challenges. FCOS addresses these issues with an anchor-free, proposal-free fully convolutional detector using FPN and center-ness. The paper reports competitive or state-of-the-art one-stage performance with simpler training and broader reuse across instance-level tasks.
Problem
Anchor-based detectors depend on sensitive anchor designs and IoU-based training computations, while earlier FCN-based detectors struggled with varying object sizes and overlapping boxes.
Method
FCOS directly predicts detection outputs at feature-map locations, uses FPN for different object sizes, and adds center-ness to suppress low-quality predictions.
Results
FCOS achieves state-of-the-art performance among one-stage detectors while avoiding anchor-box computations and hyper-parameters.
Takeaways & Limitations
FCOS provides a simpler anchor-free alternative that can reuse fully convolutional dense-prediction ideas and extend to other instance-level recognition tasks.
Takeaways & Limitations
Earlier FCN-based detectors were mainly applied to specialized domains because highly overlapping boxes created assignment ambiguity in generic object detection.
Abstract
from arXiv · showhide
In computer vision, object detection is one of most important tasks, which underpins a few instance-level recognition tasks and many downstream applications. Recently one-stage methods have gained much attention over two-stage approaches due to their simpler design and competitive performance. Here we propose a fully convolutional one-stage object detector (FCOS) to solve object detection in a per-pixel prediction fashion, analogue to other dense prediction problems such as semantic segmentation. Almost all state-of-the-art object detectors such as RetinaNet, SSD, YOLOv3, and Faster R-CNN rely on pre-defined anchor boxes. In contrast, our proposed detector FCOS is anchor box free, as well as proposal free. By eliminating the pre-defined set of anchor boxes, FCOS completely avoids the complicated computation related to anchor boxes such as calculating the intersection over union (IoU) scores during training. More importantly, we also avoid all hyper-parameters related to anchor boxes, which are often sensitive to the final detection performance. With the only post-processing non-maximum suppression (NMS), we demonstrate a much simpler and flexible detection framework achieving improved detection accuracy. We hope that the proposed FCOS framework can serve as a simple and strong alternative for many other instance-level tasks. Code and pre-trained models are available at: https://git.io/AdelaiDet
1 INTRODUCTION
FCOS reframes object detection as fully convolutional per-pixel prediction, eliminating anchor boxes and proposals while addressing ambiguity through feature pyramids and center-ness. The resulting detector simplifies training and achieves strong performance compared with anchor-based and other one-stage detectors.
- Motivation: Anchor-based detectors require sensitive shape and labeling hyper-parameters, complicate training, and generalize poorly across object-scale and aspect-ratio variations.Anchor settings can affect RetinaNet performance by up to 4% AP on COCO, while dense anchor placement also creates many negative samples.
- Motivation: FCOS asks whether object detection can use the neat per-pixel prediction framework already successful for semantic segmentation.The paper presents an affirmative answer and reports that this simpler FCN-based detector can outperform anchor-based counterparts.
- Prior FCN-based detection: Earlier FCN-based detectors struggled with varying object sizes and highly overlapping boxes, requiring image pyramids or facing ambiguity over which box each pixel should regress.DenseBox used crop-and-resize preprocessing and image pyramids, while overlapping objects created ambiguous assignments in generic detection.
- FCOS design: FCOS uses FPN to largely eliminate overlapping-box ambiguity and adds a center-ness branch that down-weights low-quality predictions far from object centers during NMS.The center-ness branch runs in parallel with regression and improves detection performance with negligible computational overhead.
- FCOS advantages: The framework is anchor free and proposal free, reducing design parameters and avoiding anchor IoU computation and matching during training.Its standard FCN structure also makes ideas developed for dense prediction tasks directly reusable for FCOS detection.
- Results: FCOS reports state-of-the-art results among one-stage detectors and outperforms strong baselines including Faster R-CNN, RetinaNet, YOLOv3, and SSD.The paper also argues that its reduced design complexity supports extensions to other instance-level recognition tasks.
2 OUR APPROACH
FCOS reformulates detection as per-pixel prediction without predefined anchors, using FPN levels to handle scale and ambiguity and center-ness to suppress low-quality boxes.
- Fully Convolutional One-Stage Object Detector: FCOS directly regresses bounding-box coordinates and predicts class scores at feature-map locations, treating locations as training samples instead of anchor boxes.Its outputs include an 80D classification vector and a 4D coordinate vector.
- Fully Convolutional One-Stage Object Detector: Locations inside a ground-truth box’s center area are positive samples, with COCO using a center-area radius hyper-parameter r=1.5.The center area is clipped to remain within the original box.
- Multi-level Prediction with FPN for FCOS: FPN uses five feature levels to detect different object sizes and resolves large-stride recall limitations and overlapping-box assignment ambiguity.The levels are P3 through P7, with strides 8, 16, 32, 64, and 128.
- Multi-level Prediction with FPN for FCOS: FCOS assigns level-specific regression ranges directly, removing the coupling between anchor-box sizes and target object sizes.Locations outside a level’s permitted regression range become negative samples and need not regress a box.
- Center-ness for FCOS: A center-ness branch predicts a normalized center distance, and its score is combined with classification confidence during testing to rank detections for NMS.The resulting score is the square root of predicted center-ness multiplied by classification confidence.
- Center-ness for FCOS: Center-ness down-weights boxes predicted far from object centers, allowing NMS to filter many low-quality detections and improve detection performance.The method uses center-ness to suppress boxes with low localization quality but high classification confidence.
3 EXPERIMENTS
The experiments evaluate FCOS on COCO using standard training and validation splits, with main results reported on test-dev and comparisons including RetinaNet’s BPR.
- Experimental Setup: Experiments use COCO train2017 for training, val2017 for ablation studies, and test-dev for the main reported results.The splits contain 115K, 5K, and 20K images, respectively.
- Best Possible Recall: FCOS has similar BPR to the best anchor-based RetinaNet and higher recall than Detectron’s official implementation under its stated matching rule.The comparison is reported on the COCO val2017 split.
- Experimental Setup: The implementation uses ResNet-50 and RetinaNet-matched hyperparameters, training with SGD for 90k iterations.The learning rate is reduced at iterations 60k and 80k.
3.1 Analysis of FCOS
FCOS analysis examines recall, ambiguity, center-ness, and design choices. FPN and center sampling reduce ambiguous samples, while center-ness suppresses low-quality predictions and improves AP.
- Ambiguous Samples in FCOS: With center sampling and FPN, ambiguous samples fall below 3% of positive samples.Without FPN, the ambiguous-sample ratio is 23.40%; FPN reduces it to 7.42%, and further center sampling lowers it.
- Best Possible Recall (BPR) of FCOS: 98.95 vs 99.32 BPR shows that FCOS and RetinaNet achieve similar maximum recall with FPN.The reported gap is less than 0.5%.
- The Effect of Center-ness: 38.9% AP is achieved with center-ness, up from 38.0% without it.The separate center-ness branch adds negligible computational time, whereas computing center-ness from the predicted regression vector does not improve performance.
- The Effect of Center-ness: Center-ness largely eliminates boxes with low IoU but high confidence scores.These boxes are identified as points under the line y = x after center-ness is applied.
- Other Design Choices: max(l*, t*, r*, b*) performs best for assigning objects to FPN levels because it keeps complete objects within the effective receptive field.This makes the range hyper-parameters mainly dependent on network architecture rather than dataset-specific tuning.
3.2 FCOS vs. Anchor-based Counterparts
FCOS is compared with RetinaNet under matched improvements and achieves higher AP while using an anchor-free design. The comparison supports reconsidering the necessity of anchor boxes.
- FCOS vs. Anchor-based Counterparts: 38.0% vs 37.6% in AP shows FCOS outperforming improved RetinaNet with nine anchors per location, even without center-ness.Adding center-ness raises FCOS to 38.9% AP.
- FCOS vs. Anchor-based Counterparts: 38.0% vs 35.2% in AP shows FCOS outperforming improved RetinaNet with one anchor per location.The comparison indicates FCOS is not equivalent to single-anchor RetinaNet because FCOS does not use anchor–ground-truth IoU scores for training labels.
- FCOS vs. Anchor-based Counterparts: FCOS combines superior performance with a simpler design and fewer hyper-parameters than anchor-based detectors.The authors therefore encourage reconsidering the necessity of anchor boxes.
3.3 Comparison with State-of-the-art Detectors on COCO
On COCO test-dev, FCOS is evaluated against one-stage and two-stage detectors using ResNet-101-FPN. It outperforms the original RetinaNet and Faster R-CNN by substantial AP margins.
- Comparison with State-of-the-art Detectors on COCO: 43.2% vs 39.1% AP means FCOS outperforms original RetinaNet with the same ResNet-101-FPN backbone by 4.1% AP.The comparison uses the COCO test-dev split.
- Comparison with State-of-the-art Detectors on COCO: 43.2% vs 36.2% AP means FCOS also surpasses Faster R-CNN by a large margin.The paper describes this as an anchor-free detector outperforming anchor-based detectors without bells and whistles.
- Comparison with State-of-the-art Detectors on COCO: FCOS also outperforms SSD, DSSD, CornerNet, and CenterNet in the reported comparisons.The paper characterizes FCOS as simpler than CornerNet and CenterNet because those methods require special corner-grouping designs.
- Comparison with State-of-the-art Detectors on COCO: 50.4% AP is obtained after test-time multi-scale and horizontal-flip augmentation.Augmented predictions are merged using NMS.
3.4 Real-time FCOS
FCOS-RT adapts FCOS for real-time inference by reducing input resolution and feature levels while using more aggressive training. The resulting models provide competitive speed–accuracy trade-offs.
- Real-time FCOS: 40.2% AP at 38 FPS is achieved by FCOS-RT with ResNet-50 on one 1080Ti GPU.The real-time configuration reduces input dimensions and removes P6 and P7.
- Real-time FCOS: 40.3% AP at 46 FPS is achieved after replacing ResNet-50 with DLA-34.The paper reports this as a better speed/accuracy trade-off.
- Real-time FCOS: 52 FPS is reached by sharing classification and regression towers, with a 1.2% AP decrease.Even with this trade-off, the model outperforms CenterNet by 1.7% AP at the same speed.
- Real-time FCOS: 42.1% AP is obtained by replacing FPN with BiFPN at similar speed.The reported improvement is 1.8% AP from the 40.3% AP model.
3.5 FCOS on CrowdHuman
On CrowdHuman, FCOS addresses crowded scenes where locations may correspond to multiple ground-truth boxes, using target-selection and multiple-instance strategies while comparing favorably with anchor-based detectors.
- On COCO, FCOS selects the minimal-area object for ambiguous locations, whereas CrowdHuman uses the object whose center is nearest to the location.The two schemes perform similarly on COCO, while the distance-based scheme performs better on CrowdHuman.
- 84.47% of positive samples are unambiguous on CrowdHuman, while 13.63% correspond to two boxes and 1.69% to three boxes.The remaining samples correspond to more than three ground-truth boxes.
- FCOS compares favorably with RetinaNet on AP and JI under crowded detection, although its higher MR−2 indicates more high-confidence false positives.Lower MR−2 is better, and center-ness can reduce this error.
- The CrowdHuman experiments are intended to show that anchor boxes are unnecessary, not to pursue state-of-the-art benchmark performance.
- 59.04% to 51.34%: the reported reduction improves FCOS across AP, MR−2, and JI on CrowdHuman.
- FCOS can incorporate multiple instance prediction and Set NMS when one location is associated with multiple objects.Set NMS skips suppression for boxes originating from the same location.
4 CONCLUSION
The paper concludes that FCOS is an anchor-free and proposal-free one-stage detector that performs competitively with much less design complexity. It presents FCOS as a simple alternative to mainstream anchor-based detectors, including real-time models with state-of-the-art performance and inference speed.
- FCOS is an anchor-free and proposal-free one-stage detector that performs object detection through per-pixel prediction.
- FCOS compares favorably with RetinaNet, YOLO, and SSD while avoiding anchor-box computation and hyper-parameters.
- FCOS includes real-time models with state-of-the-art performance and inference speed.