PaperlayerView exact revision on arXiv

Source-linked AI summary

Disentangle Your Dense Object Detector

Zehui Chen, Chenhongyi Yang, Qiaofei Li, Feng Zhao, Zheng-Jun Zha, Feng Wu

arXiv:2107.02963v2cs.CV

TL;DR

Dense-detector training relies on three conjunctions whose validity is questioned: shared assignment, shared spatial features, and equal pyramid supervision. The paper proposes DDOD to disentangle them and reports improvements across dense-detector baselines, with state-of-the-art results on COCO and WIDER FACE.

  • Problem

    Dense detectors use training conjunctions involving shared label assignment, shared receptive fields, and equal supervision across pyramid layers, although these assumptions may be suboptimal.

  • Method

    DDOD disentangles label assignment, spatial features, and pyramid supervision using separate task assigners, adaptive feature mechanisms, and FPN re-weighting.

  • Results

    DDOD improves RetinaNet, FCOS, and ATSS by more than 2.0 mAP and achieves state-of-the-art results on COCO and WIDER FACE.

  • Takeaways & Limitations

    Disentangling these conjunctions improves detection accuracy across representative dense detectors and generalizes to COCO and WIDER FACE.

Abstract

from arXiv · show

Deep learning-based dense object detectors have achieved great success in the past few years and have been applied to numerous multimedia applications such as video understanding. However, the current training pipeline for dense detectors is compromised to lots of conjunctions that may not hold. In this paper, we investigate three such important conjunctions: 1) only samples assigned as positive in classification head are used to train the regression head; 2) classification and regression share the same input feature and computational fields defined by the parallel head architecture; and 3) samples distributed in different feature pyramid layers are treated equally when computing the loss. We first carry out a series of pilot experiments to show disentangling such conjunctions can lead to persistent performance improvement. Then, based on these findings, we propose Disentangled Dense Object Detector (DDOD), in which simple and effective disentanglement mechanisms are designed and integrated into the current state-of-the-art dense object detectors. Extensive experiments on MS COCO benchmark show that our approach can lead to 2.0 mAP, 2.4 mAP and 2.2 mAP absolute improvements on RetinaNet, FCOS, and ATSS baselines with negligible extra overhead. Notably, our best model reaches 55.0 mAP on the COCO test-dev set and 93.5 AP on the hard subset of WIDER FACE, achieving new state-of-the-art performance on these two competitive benchmarks. Code is available at https://github.com/zehuichen123/DDOD.

1 INTRODUCTION

Dense detectors offer fast, direct prediction for multimedia applications, but their training pipelines contain three underexamined conjunctions that can deteriorate performance. DDOD disentangles these conjunctions through separate task assignments, spatial features, and pyramid supervision.

  • Single-stage dense detectors directly predict bounding boxes, providing a simple pipeline and fast inference suitable for edge devices.
  • Shared receptive fields are suboptimal because classification favors semantic regions, whereas regression favors object edges.
  • P3 contains 256 times as many training samples as P7, causing shallow layers to dominate supervision.
  • Three training conjunctions concern label assignment, shared receptive fields, and equal supervision across FPN layers.
  • DDOD disentangles the conjunctions using separate classification and regression assigners, spatial feature mechanisms, and adaptive FPN supervision re-weighting.
  • The paper reports detailed experiments, a unified DDOD framework, and state-of-the-art results on COCO and WIDER FACE.

2 RELATED WORK

Related work addresses label assignment, feature alignment, and imbalance in object detection. DDOD targets the interaction of these three issues specifically in dense detectors with minimal added cost.

  • Label-assignment methods replace fixed IoU thresholds with strategies based on loss values or candidate bags.
  • Feature-alignment methods improve representations through cascaded re-alignment, separate heads, or different classification and localization RoIAlign offsets.
  • Whether classification-localization feature conjunctions also affect dense detectors remains an open problem in prior work.
  • Imbalanced learning spans sample, feature, and objective aspects, with Focal loss addressing positive-negative sample imbalance in dense detection.
  • DDOD disentangles three conjunctions in state-of-the-art dense detectors to improve accuracy with minimum cost.

3 PILOT EXPERIMENTS

Pilot experiments examine label assignment, spatial feature, and pyramid supervision conjunctions in dense detectors. Their findings motivate task-specific assignment and pyramid-level re-weighting.

  • Pilot experiments investigate three dense-detector training conjunctions and test whether disentanglement improves detection performance.
  • 3.1 Label Assignment Conjunction: A RetinaNet achieves 36.5 mAP when classification and regression use separate IoU thresholds of 0.5 and 0.4, respectively.
  • 3.2 Spatial Feature Conjunction: Classification and regression should attend different object regions because semantic areas aid classification while contours aid localization.
  • 3.2 Spatial Feature Conjunction: GradCAM visualizations show different sensitive regions for classification and regression on FPN outputs.
  • 3.3 Supervision Conjunction in FPN: Flattening predictions across pyramid levels gives lower-level samples disproportionate supervision because feature resolution grows quadratically.
  • 3.3 Supervision Conjunction in FPN: FPN hierarchical loss re-weights pyramid supervision, and reducing the weight from 3.0 to 1.0 validates the proposed disentanglement.

4 METHODOLOGIES

DDOD disentangles label assignment, spatial features, and pyramid supervision in dense detector training. These mechanisms assign task-specific samples, learn task-specific receptive regions, and reweight supervision across FPN layers.

  • 4.1 Label Assignment Disentanglement: DDOD uses separate label assigners for classification and regression to select task-specific training samples.A cost formulation balances foreground probability and regression quality through α, enabling distinct assignments for the two branches.
  • 4.2 Spatial Feature Disentanglement: Deformable convolution learns task-specific offsets so classification attends discriminative features while regression attends border features.The classification and regression branches therefore sample different spatial locations instead of sharing a regular receptive field.
  • 4.3 Pyramid Supervision Disentanglement: FPN hierarchical loss reweights samples according to their pyramid-level counts to address unequal supervision across layers.The method strengthens supervision for deeper layers, where fewer training samples occur; weights empirically range from 1.0 to 2.0, with P3 at 1.0 and P7 at 2.0.
  • Overview: Figure 3 contrasts DDOD’s three disentanglement mechanisms with the original implementation’s shared labels, offsets, and equal layer supervision.The proposed design produces different classification and regression labels, separate convolutional offsets, and different loss weights across FPN layers.

5 EXPERIMENTS

The experiments evaluate DDOD on COCO using representative dense detectors and controlled component studies. They include implementation details, label-assignment hyperparameter analysis, and comparisons of DDOD modules.

  • Experimental Setup: Experiments evaluate DDOD on COCO using RetinaNet, FCOS, and ATSS, with AP reported for single-stage detector comparisons.The study also includes component effects and hyperparameter analyses.
  • Implementation Details: The ATSS implementation uses an advanced FCOS implementation.
  • Implementation Details: COCO experiments train on trainval35k with 115K images and evaluate on minval with 5K images or test-dev with 20K images.Most experiments use a standard 1x schedule with ResNet-50 and MMDetection default parameters.
  • Ablation Studies: The study reports effects of DDOD components and different α values for classification and regression label assignment.It also examines deformable-convolution positions and FPN hierarchical-loss reweighting strategies.

5.2 Results on Dense Object Detectors

DDOD improves three representative dense detectors, with the largest gains reported for large objects. The results attribute these gains to spatial attention and stronger supervision on high-level feature maps.

  • Dense Detector Results: More than 2.0 mAP improvement is reported for DDOD across RetinaNet, FCOS, and ATSS baselines.The method is implemented on all three representative dense detectors.
  • Dense Detector Results: AP_L improves most across the three detectors.The paper relates this pattern to spatial disentanglement attending important parts of large objects and hierarchical loss compensating for insufficient high-level supervision.

5.3 Ablation Study

Ablations on ATSS show that each disentanglement contributes to performance, with their combination producing the largest overall gain. The component effects are especially pronounced for large-object AP and localization accuracy.

  • Label Assignment Disentanglement: 1.0 AP is gained from label assignment disentanglement over the 39.4 AP ATSS baseline.The improvement occurs across objects of all sizes.
  • Spatial Feature Disentanglement: 1.2 AP is added by spatial feature disentanglement, with AP_L benefiting most.Its improvement on AP75 reaches 1.5, linking the module’s effect to accurate bounding-box localization in the reported ablation.
  • Pyramid Supervision Disentanglement: 0.6 accuracy improvement follows adding pyramid supervision disentanglement, while AP_L increases by 3.0.The reported gain corresponds to stronger supervision for underrepresented deeper pyramid layers.

5.4 Discussion

Discussion experiments identify settings that strengthen DDOD’s three disentanglement mechanisms, including branch-specific assignment, early feature separation, and pyramid-level re-weighting.

  • Experimental Setup: The experiments use ResNet-50-based ATSS with the same settings as Section 5.1.This provides the common evaluation setup for the discussion experiments.
  • Label Assignment Disentanglement: α_cls=0.8 and α_reg=0.5 provide robust branch-specific label-assignment settings across scenarios.The differing optima support separate assignment criteria for classification and regression.
  • Spatial Feature Disentanglement: 40.6 mAP is achieved when spatial feature disentanglement replaces the first head convolution with deformable convolution.Moving disentanglement to later head layers decreases performance, while separating features directly from FPN eases learning.
  • Pyramid Supervision Disentanglement: 40.0 mAP is achieved by the proposed linear-interpolation re-weighting on the COCO validation subset.Moving-average weights stabilize training and slightly improve accuracy.

5.5 Comparison with State-of-the-Arts

DDOD outperforms other state-of-the-art detectors under matched backbones and reaches 55.0 mAP on COCO test-dev with additional testing and augmentation techniques.

  • COCO Test-Dev: 52.5 mAP is achieved by DDOD with a Res2Net-101-DCN backbone under single-scale testing.With multi-scale testing, Soft-NMS, and stronger augmentation, the model reaches 55.0 mAP and establishes state-of-the-art performance.

5.6 Results on WIDER FACE Dataset

DDOD generalizes to crowded face-detection scenes and improves the adapted ATSS baseline across WIDER FACE difficulty settings, reaching state-of-the-art hard-subset performance.

  • WIDER FACE: Approximately 0.5 AP improvement is obtained over the adapted ATSS baseline under easy, medium, and hard settings.The baseline reaches 93.0 overall AP after adding DCN, SSH, and DIoU loss.
  • WIDER FACE: 93.5% AP hard is achieved by DDOD-Face with a ResNet-50 backbone on the validation subset.The corresponding easy and medium results are 97.0% and 96.4%.

6 CONCLUSION

The paper presents DDOD as a training paradigm that decomposes conjunctions in one-stage dense detectors and demonstrates state-of-the-art results on COCO and WIDER FACE.

  • Conclusion: DDOD decomposes current one-stage detector conjunctions through label assignment, spatial feature, and pyramid supervision disentanglement.The paper presents these mechanisms as a unified training paradigm for dense object detectors.
  • Conclusion: State-of-the-art results on COCO and WIDER FACE validate DDOD’s effectiveness and generalization.The authors suggest extending DDOD to two-stage detectors as an important next step.
Loading 2107.02963v2…