Source-linked AI summary
Gradient Harmonized Single-stage Detector
Buyu Li, Yu Liu, Xiaogang Wang
TL;DR
Single-stage detectors are efficient but suffer from positive-negative and easy-hard training disharmonies. The paper interprets these problems through gradient distributions and proposes GHM-C and GHM-R to harmonize classification and regression gradients. The combined model surpasses modern two-stage detectors on COCO under the reported evaluation.
Problem
Single-stage detectors face severe imbalance between positive and negative examples and between easy and hard examples, which complicates training.
Method
The paper proposes GHM and embeds it into classification and regression as GHM-C and GHM-R, adapting example weighting to gradient density and changing data distributions.
Results
The combined GHM-C and GHM-R detector achieves state-of-the-art performance on COCO and can surpass modern two-stage detectors with the same backbone.
Takeaways & Limitations
GHM provides a single gradient-based framework for addressing classification imbalance and regression outliers in single-stage detection.
Takeaways & Limitations
The optimal target distribution of gradients remains undefined and requires further research.
Abstract
from arXiv · showhide
Despite the great success of two-stage detectors, single-stage detector is still a more elegant and efficient way, yet suffers from the two well-known disharmonies during training, i.e. the huge difference in quantity between positive and negative examples as well as between easy and hard examples. In this work, we first point out that the essential effect of the two disharmonies can be summarized in term of the gradient. Further, we propose a novel gradient harmonizing mechanism (GHM) to be a hedging for the disharmonies. The philosophy behind GHM can be easily embedded into both classification loss function like cross-entropy (CE) and regression loss function like smooth-$L_1$ ($SL_1$) loss. To this end, two novel loss functions called GHM-C and GHM-R are designed to balancing the gradient flow for anchor classification and bounding box refinement, respectively. Ablation study on MS COCO demonstrates that without laborious hyper-parameter tuning, both GHM-C and GHM-R can bring substantial improvement for single-stage detector. Without any whistles and bells, our model achieves 41.6 mAP on COCO test-dev set which surpasses the state-of-the-art method, Focal Loss (FL) + $SL_1$, by 0.8.
Introduction
Single-stage detectors are efficient but face severe imbalance among positive and negative examples and among easy and hard examples. The paper reframes these disharmonies through gradient distributions and introduces GHM-based classification and regression losses.
- Motivation: Single-stage detectors are efficient and elegant, but easy and background examples can overwhelm training through severe positive-negative and easy-hard imbalance.Example-mining methods discard most examples, while proposal-driven two-stage detectors avoid these problems.
- Gradient View: The two imbalance problems can be summarized as disharmony in gradient norm distribution across examples.Easy well-classified examples produce small gradients, whereas misclassified examples produce larger gradients and attract model attention.
- Proposed Mechanism: GHM estimates gradient density for examples with similar attributes and assigns each example a harmonizing parameter according to that density.The mechanism targets the harmony of gradient contributions rather than discarding examples.
- Proposed Mechanism: GHM-C and GHM-R embed gradient harmonization into classification and regression losses, respectively, with GHM-C adapting to batch and model-distribution changes.GHM-C is formulated with few hyper-parameters and GHM-R extends the mechanism to box regression.
Related Work
Related work contrasts efficient, simple one-stage detectors with accurate two-stage proposal-driven systems. Prior methods address classification imbalance through example mining or loss reformulation, while smooth L1 remains standard for box regression.
- Two-Stage Detectors: Two-stage detectors use R-CNN-style proposal generation to reduce nearly infinite candidate regions to a manageable set of regions of interest.The network then evaluates each region of interest.
- One-Stage Detectors: One-stage detectors offer simple structures and high speed but have historically struggled to surpass two-stage detector accuracy.RetinaNet improves one-stage performance by focusing on class imbalance during training.
- Classification Losses: Hard-example mining methods address one-stage classification imbalance by discarding many examples, but they are inefficient and insufficient.Focal Loss instead reformulates cross-entropy so easy negatives are down-weighted.
- Regression Losses: Smooth L1 is widely used for stable box regression because it reduces the impact of outliers.It became the default regression loss in nearly all subsequent detection works cited here.
- Gradient Harmonization: GHM harmonizes example contributions using gradient distributions and adapts weights to changing mini-batch data distributions.The approach is positioned to address both class imbalance and outliers without relying solely on fixed example selection.
Gradient Harmonizing Mechanism
The paper interprets example imbalance through gradient-norm distributions and introduces GHM to harmonize gradient contributions. GHM is implemented as adaptive classification and regression losses that reduce dominant easy-example and outlier contributions.
- Gradient Density: Gradient norm g measures an example’s attribute and its impact on the global gradient.It is defined as the norm of the gradient with respect to the model output x.
- Gradient Density: Very easy examples are extremely numerous, while very hard examples can outnumber medium-difficulty examples and behave as outliers.The distribution is shown on a log scale because example counts differ by orders of magnitude.
- Gradient Harmonizing Mechanism: GHM assigns each example a harmonizing parameter based on the density of neighboring gradient norms, down-weighting examples in dense regions.The parameter can be rewritten as βi = 1 / (GD(gi)/N), so higher local density receives lower weight.
- GHM-C Loss: GHM-C embeds these density-based weights into classification loss and dynamically adapts them to each mini-batch and model state.Its reformulated gradient has a trend similar to Focal Loss while also down-weighting outlier contributions.
- Efficient Approximation: The approximate density computation uses histogram regions with M units, reducing practical calculation cost and enabling parallel computation.The paper notes that direct density calculation can require O(N^2), whereas the histogram method has O(MN) complexity.
- GHM-R Loss: For regression, ASL1 preserves quadratic behavior for small errors and linear behavior for large errors, enabling GHM-R to balance easy and hard examples.GHM-R up-weights important easy examples and down-weights outliers, outperforming SL1 and ASL1 in experiments.
Experiments
The approach is evaluated on the COCO bounding box detection benchmark using established training splits, with main results reported on test-dev and ablations on minival.
- COCO experiments use a 35k validation subset plus the 80k training set for trainval35k, while ablations use the separate 5k minival subset.
Implementation Details
Experiments use RetinaNet with ResNet-FPN backbones, standardized anchors and input scale, SGD optimization, and fixed training schedules and augmentation.
- RetinaNet experiments use ResNet-FPN backbones, three anchor scales, three aspect ratios, and 800-pixel input images.Ablations use ResNet-50, while the final test-dev model uses ResNeXt-101.
- Models are trained with SGD for 14 epochs using an initial learning rate of 0.01, scheduled reductions, weight decay, momentum, and horizontal flipping.Training uses eight GPUs with an effective mini-batch size of 16.
GHM-C Loss
GHM-C is evaluated for classification against cross-entropy under fixed smooth L1 regression, with unit-region sensitivity and training efficiency examined alongside comparisons to other losses.
- GHM-C experiments fix smooth L1 regression with δ = 1/9 to isolate classification-loss effects.
- The cross-entropy baseline requires focal-loss-style bias initialization and a 20× classification-loss weight to avoid divergence and maintain a reasonable initial loss.
- GHM-C performance improves as the number of unit regions increases when M is small, while M = 30 already provides a large improvement over baseline.
- Region-unit approximation accelerates GHM-C training by magnitudes with negligible performance harm, while its slowdown versus cross-entropy remains acceptable.The loss does not change inference time, but its current implementation is not fully GPU-based.
- GHM-C performs slightly better than focal loss in the reported comparison after focal loss is retrained at the same 800-pixel input scale.
GHM-R Loss
GHM-R is evaluated for box regression against smooth L1 and adaptive smooth L1, including IoU-threshold behavior, COCO comparisons, and transfer to two-stage detection.
- GHM-R gains 0.7 mAP over the adaptive smooth L1 baseline in the regression-loss comparison.
- The combined approach achieves state-of-the-art performance on the COCO test-dev set and outperforms focal loss in most reported metrics.
- GHM-R slightly lowers AP@IoU=0.5 but improves at higher IoU thresholds, emphasizing the relevance of easy regression examples for accurate localization.
- GHM-R also works with a Faster R-CNN ResNet-50-FPN baseline, showing applicability beyond one-stage detectors.
Main Results
The evaluated RetinaNet uses GHM-C for classification and GHM-R for box regression. On COCO test-dev, the approach achieves excellent performance and outperforms focal loss in most metrics.
- The model uses a 32x8d FPN-ResNext101 backbone with RetinaNet, GHM-C classification, and GHM-R box regression.
- The main results are reported on the COCO test-dev set.
- The approach outperforms focal loss in most metrics on the test-dev set.
Conclusion and Discussion
The paper frames single-stage detector training problems as gradient-density disharmony and proposes GHM-C and GHM-R for classification and regression. It notes that the optimal gradient distribution remains an open research question.
- Conclusion and Discussion: GHM summarizes single-stage detectors’ positive-negative and easy-hard imbalances as disharmony in gradient density.
- Conclusion and Discussion: GHM-C and GHM-R address disharmony in classification and bounding box regression, respectively.
- Conclusion and Discussion: With GHM, single-stage detectors can surpass modern state-of-the-art two-stage detectors using the same network backbone.
- Conclusion and Discussion: The optimal gradient distribution is not defined and requires further research.