PaperlayerView exact revision on arXiv

Source-linked AI summary

Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection

Xiang Li, Wenhai Wang, Lijun Wu, Shuo Chen, Xiaolin Hu, Jun Li, Jinhui Tang, Jian Yang

arXiv:2006.04388v1cs.CV

TL;DR

Existing dense detectors separately use localization quality and classification during training and inference, while rigid box distributions struggle with ambiguous boundaries. GFL jointly represents quality and classification, models box locations as general distributions, and extends Focal Loss to continuous labels, achieving 45.0% AP with ResNet-101 on COCO test-dev and 48.2% AP at 10 FPS in its best model.

  • Problem

    Dense detectors face training–inference inconsistency from separate quality and classification usage and limited Dirac delta representations for ambiguous localization.

  • Method

    GFL merges localization quality with classification, represents box locations as general distributions, and generalizes Focal Loss to continuous labels through QFL and DFL.

  • Results

    45.0% AP with ResNet-101 on COCO test-dev surpasses SAPD (43.5%) and ATSS (43.6%), while the best model reaches 48.2% AP at 10 FPS.

  • Takeaways & Limitations

    GFL provides a joint quality-classification representation and flexible distributed box estimation without introducing additional overhead.

Abstract

from arXiv · show

One-stage detector basically formulates object detection as dense classification and localization. The classification is usually optimized by Focal Loss and the box location is commonly learned under Dirac delta distribution. A recent trend for one-stage detectors is to introduce an individual prediction branch to estimate the quality of localization, where the predicted quality facilitates the classification to improve detection performance. This paper delves into the representations of the above three fundamental elements: quality estimation, classification and localization. Two problems are discovered in existing practices, including (1) the inconsistent usage of the quality estimation and classification between training and inference and (2) the inflexible Dirac delta distribution for localization when there is ambiguity and uncertainty in complex scenes. To address the problems, we design new representations for these elements. Specifically, we merge the quality estimation into the class prediction vector to form a joint representation of localization quality and classification, and use a vector to represent arbitrary distribution of box locations. The improved representations eliminate the inconsistency risk and accurately depict the flexible distribution in real data, but contain continuous labels, which is beyond the scope of Focal Loss. We then propose Generalized Focal Loss (GFL) that generalizes Focal Loss from its discrete form to the continuous version for successful optimization. On COCO test-dev, GFL achieves 45.0\% AP using ResNet-101 backbone, surpassing state-of-the-art SAPD (43.5\%) and ATSS (43.6\%) with higher or comparable inference speed, under the same backbone and training settings. Notably, our best model can achieve a single-model single-scale AP of 48.2\%, at 10 FPS on a single 2080Ti GPU. Code and models are available at https://github.com/implus/GFocal.

1 Introduction

Dense detectors commonly use separate localization-quality estimation and rigid box representations, creating training–inference inconsistency and difficulty representing ambiguous boundaries. GFL addresses these issues with joint quality-classification and distributed box representations optimized through a continuous generalization of Focal Loss.

  • Introduction: Separate quality and classification branches are trained independently but combined during inference, creating a training–test gap.Negative samples supervised only for quality estimation can produce unusually high quality predictions and rank ahead of positives.
  • Introduction: Existing Dirac delta box representations cannot adequately express arbitrary localization distributions in ambiguous scenes.Occlusion, shadow, and blur can make object boundaries unclear, while Gaussian alternatives remain too simple or symmetric.
  • Introduction: The proposed classification-IoU joint representation merges localization quality into the class prediction vector for end-to-end training and direct inference use.Negative samples receive zero quality scores, improving the reliability of quality predictions and their correlation with classification.
  • Introduction: Generalized Focal Loss extends Focal Loss from discrete {1, 0} labels to continuous targets, supporting the new representations.Its specializations include Quality Focal Loss for joint classification-quality learning and Distribution Focal Loss for distributed box locations.

2 Related Work

Prior work represents localization quality with separate IoU or centerness branches and boxes with Dirac delta or Gaussian assumptions. These representations are respectively inconsistent or unreliable across training and inference, and too rigid or simplified for complex real distributions.

  • Representation of localization quality: Separate IoU or centerness branches estimate localization quality in methods including Fitness NMS, IoU-Net, MS R-CNN, FCOS, and IoU-aware.The paper identifies separate quality estimation as a source of training–test inconsistency and unreliable predictions.
  • Representation of bounding boxes: Dirac delta representations dominate bounding-box modeling, while Gaussian approaches introduce predicted variance to learn uncertainty.The paper characterizes both existing approaches as insufficient for reflecting complex underlying box-location distributions.

3 Method

GFL redesigns dense-detector representations by jointly encoding classification and localization quality, modeling box locations as flexible distributions, and extending Focal Loss to continuous labels. Its components are QFL for quality-aware classification and DFL for distributed box regression, unified under a generalized focal-loss perspective.

  • Overview: GFL combines Quality Focal Loss (QFL) and Distribution Focal Loss (DFL) to optimize quality-aware classification and distributed box regression.The method presents QFL and DFL as improved representations and unifies them as Generalized Focal Loss.
  • Quality Focal Loss: QFL jointly represents classification and localization quality by assigning each category prediction a continuous target based on IoU.Negative samples have quality target 0, while positive samples use the predicted-box/ground-truth IoU in [0,1].
  • Quality Focal Loss: QFL extends Focal Loss to continuous labels by combining complete cross entropy with an absolute-distance modulating factor |y −σ|^β.The modulating factor emphasizes inaccurate quality estimates and down-weights well-estimated examples; β = 2 performs best in the reported experiments.
  • Distribution Focal Loss: DFL replaces Dirac delta or Gaussian assumptions with a discretized General distribution over bounding-box regression targets.The continuous target range is discretized into evenly spaced values and represented with a softmax distribution.
  • Distribution Focal Loss: DFL focuses probability mass on values neighboring the target, producing an estimate arbitrarily close to the regression label.Its optimum assigns probabilities to the two adjacent discretized values, which preserves the target expectation while encouraging compact predictions.
  • Ablations: Joint representations optimized by QFL yield about 0.6–0.8% AP gains without additional inference overhead, while DFL performance is robust across n and favors small Δ.The reported experiments identify β = 2 as best for QFL; for DFL, Δ = 1 is described as sufficient in practice.

4 Experiment

Experiments on COCO evaluate QFL, DFL, and their combination in dense detectors, showing improved accuracy with negligible practical overhead and competitive speed-accuracy trade-offs.

  • QFL: QFL joint representations consistently outperform separate or implicit localization-quality representations, and IoU performs better than centerness.QFL also improves other one-stage detectors by about 0.6–0.8% AP without additional inference overhead.
  • DFL: DFL with General distributions achieves superior or comparable results to alternative box representations and provides more accurate locations, especially under considerable occlusion.The distributed representation also reflects bounding-box uncertainty through its shape.
  • Ablation: 1% AP improvement over the strong ATSS baseline results from jointly using QFL and DFL, whose contributions are orthogonal.The combined model introduces negligible additional inference overhead on a single GeForce RTX 2080Ti GPU.
  • COCO test-dev: 45.0% AP at 14.6 FPS is achieved by GFL with ResNet-101 on COCO test-dev, exceeding SAPD at 43.5% and ATSS at 43.6% with the same backbone.The comparison uses single-model, single-scale testing with multi-scale training and a 2x learning schedule.
  • COCO test-dev: 48.2% AP at 10 FPS is obtained by GFL with ResNeXt-101-32x4d-DCN under single-model, single-scale testing.The speed-accuracy visualization places GFL at a high accuracy-speed boundary.

5 Conclusion

The paper proposes GFL to learn qualified and distributed bounding boxes by extending Focal Loss from discrete to continuous labels. QFL unifies classification with localization quality, while DFL models box locations as General distributions.

  • Contribution: GFL generalizes Focal Loss from {1, 0} discrete supervision to continuous labels for dense object detection.It is specialized into Quality Focal Loss and Distribution Focal Loss.
  • Contribution: QFL learns a joint representation of classification and localization quality, while DFL models box locations as General distributions for more informative and precise estimates.The conclusion presents both components as the core mechanisms for qualified and distributed bounding boxes.
  • Conclusion: Extensive experiments validate GFL’s effectiveness as a simple baseline for dense object detectors.The paper positions GFL as a baseline for future community use.

A More Discussions about the Distributions

The paper contrasts rigid Dirac delta and Gaussian assumptions with a more flexible General distribution for bounding-box locations. Simulations indicate that General distributions are less sensitive to feature perturbations.

  • Distribution assumptions: Gaussian optimization is a dynamically weighted L2 loss, making it similar to Dirac delta optimization at the edge level.Both assumptions have limitations in modeling real data distributions.
  • Distribution assumptions: The General distribution can take an arbitrary shape, unlike the fixed-point Dirac delta and relatively rigid Gaussian distributions.This comparison concerns the flexibility of the assumed location distributions.
  • Perturbation sensitivity: A feature disturbance with norm 0.1 produces more regression errors for Dirac delta, with error increasing as the regression target grows.The General distribution remains stable and insensitive to the same disturbance for targets 1.5, 2.5, and 3.5.
  • Perturbation sensitivity: Dirac delta and Gaussian box regression are more sensitive to feature perturbations, whereas General distributions provide greater robustness to noise.The simulation is presented as evidence for the General distribution’s advantage over the other representations.

B Global Minimum of GFL(pyl, pyr)

The global-minimum analysis shows that GFL changes the optimal classification target from one-hot labels toward the accurate IoU between predicted and ground-truth boxes, benefiting NMS ranking.

  • GFL's global minimum forces predictions toward the accurate IoU between estimated boxes and ground-truth boxes.
  • IoU- or centerness-guided variants do not substantially change the original classification loss's global minimum, whose optimal targets remain one-hot labels.
  • This distinction makes GFL beneficial for the ranking process of NMS.

C FL, QFL and DFL are special cases of GFL

GFL subsumes FL, QFL, and DFL as special cases obtained by selecting particular label, probability, and β settings.

  • GFL specializes to the forms of FL, QFL, and DFL through appropriate parameter substitutions.
  • FL: FL is recovered with β = γ, labels y_l = 0 and y_r = 1, probabilities p_y_r = p and p_y_l = 1 − p, and binary y ∈ {1, 0}.
  • QFL: QFL is recovered by setting y_l = 0, y_r = 1, p_y_r = σ, and p_y_l = 1 − σ.
  • DFL: DFL is recovered with β = 0, adjacent labels y_i and y_i+1, and probabilities S_i and S_i+1.

D Details of Experimental Settings

The experiments use ImageNet-pretrained FPN backbones, standardized image resizing, SGD training for ablations, and thresholded per-class NMS at inference.

  • ImageNet-pretrained models with FPN serve as the experimental backbones.
  • Training resizes images to an 800-pixel shorter side and at most 1333 pixels on the longer side, using a 90K-iteration SGD ablation schedule.
  • Inference applies a 0.05 background filter, keeps the top 1000 candidates per feature pyramid, and uses per-class NMS at IoU threshold 0.6 for 100 final detections per image.

E Why is IoU-branch always superior than centerness-branch?

IoU provides more reliable localization-quality supervision than centerness because centerness often produces very small labels, making positive boxes harder to recall.

  • Most IoU labels exceed 0.4, whereas centerness labels tend to be much smaller and can approach 0.
  • Small centerness labels can reduce final NMS scores and prevent some ground-truth boxes from being recalled.
  • Figure 11 compares ground-truth and predicted boxes with positive points on a stride-8 feature-pyramid layer.
  • Figure 12 shows label distributions across all positive COCO training samples from a pretrained ResNet-50 GFL detector.

F More Examples of Distributed Bounding Boxes

GFL produces distributed bounding-box predictions that reflect ambiguity and uncertainty in complex scenes, while remaining sharp for clear boundaries.

  • GFL sometimes predicts more reasonable box coordinates than the ground-truth boxes when object boundaries are ambiguous.The examples include a slim, nearly invisible backpack strap and a partially occluded umbrella handle.
  • Ambiguous boundaries can yield flattened General distributions and, in some cases, two peaks corresponding to different likely object boundaries.Examples include uncertainty over whether an occluded umbrella handle belongs inside the box.
  • Clear boundaries produce relatively sharp General distributions and highly accurate box estimates.
Loading 2006.04388v1…