PaperlayerView exact revision on arXiv

Source-linked AI summary

Localization Distillation for Dense Object Detection

Zhaohui Zheng, Rongguang Ye, Ping Wang, Dongwei Ren, Wangmeng Zuo, Qibin Hou, Ming-Ming Cheng

arXiv:2102.12252v4cs.CV

TL;DR

Object-detector knowledge distillation often emphasizes feature imitation, while localization knowledge and the value of separating semantic from localization transfer remain underexplored. The paper proposes localization distillation with valuable localization regions and reports improved COCO detection accuracy without inference overhead.

  • Problem

    Prior object-detection KD methods mainly imitate deep features or classification logits, while localization knowledge is insufficiently addressed and semantic and localization knowledge are mixed on feature maps.

  • Method

    The paper separately distills semantic and localization knowledge using localization distillation and a valuable localization region for dense object detectors.

  • Results

    The method improves GFocal-ResNet-50 from 40.1 to 42.1 AP and from 43.1 to 45.6 AP75 on MS COCO without inference overhead.

  • Takeaways & Limitations

    Logit mimicking can outperform feature imitation, and separately transferring classification and localization knowledge is important for distilling object detectors.

Abstract

from arXiv · show

Knowledge distillation (KD) has witnessed its powerful capability in learning compact models in object detection. Previous KD methods for object detection mostly focus on imitating deep features within the imitation regions instead of mimicking classification logit due to its inefficiency in distilling localization information and trivial improvement. In this paper, by reformulating the knowledge distillation process on localization, we present a novel localization distillation (LD) method which can efficiently transfer the localization knowledge from the teacher to the student. Moreover, we also heuristically introduce the concept of valuable localization region that can aid to selectively distill the semantic and localization knowledge for a certain region. Combining these two new components, for the first time, we show that logit mimicking can outperform feature imitation and localization knowledge distillation is more important and efficient than semantic knowledge for distilling object detectors. Our distillation scheme is simple as well as effective and can be easily applied to different dense object detectors. Experiments show that our LD can boost the AP score of GFocal-ResNet-50 with a single-scale 1x training schedule from 40.1 to 42.1 on the COCO benchmark without any sacrifice on the inference speed. Our source code and trained models are publicly available at https://github.com/HikariTJU/LD

1. Introduction

Object detection distillation must address localization ambiguity, not only semantic classification. The paper separates semantic and localization transfer and introduces localization distillation to improve dense detectors without inference overhead.

  • Motivation: Localization ambiguity makes some object edges difficult to locate and is especially problematic for lightweight detectors.Examples include the bottom edge of an elephant and the right edge of a surfboard.
  • Motivation: Previous detection KD methods mainly imitate deep features because classification logit mimicking transfers semantics while neglecting localization knowledge.Feature maps mix semantic and localization knowledge, making the benefit of transferring each type at each location difficult to determine.
  • Method: The proposed divide-and-conquer strategy separately transfers semantic knowledge with classification KD and localization knowledge with localization distillation.LD switches bounding boxes to probability distributions rather than treating teacher outputs as additional regression targets.
  • Results: A best ResNeXt-101-32x4d-DCN model reaches 50.5 AP under single-scale testing and surpasses existing detectors under matched settings.The matched comparison uses the same backbone, neck, and test settings.

2. Related Work

Related work spans bounding-box representations, localization-quality estimation, and knowledge distillation. The paper differs from feature-imitation approaches by separately transferring classification and localization knowledge through logits and valuable localization regions.

  • Bounding Box Regression: Bounding-box localization has evolved from Dirac delta to Gaussian and probability-distribution representations.Probability distributions describe bounding-box uncertainty more comprehensively than earlier representations.
  • Localization Quality Estimation: Localization Quality Estimation predicts a score measuring the localization quality of a detector’s bounding box.It can support training consistency with classification or joint classification-localization decisions during NMS.
  • Knowledge Distillation: Knowledge distillation trains compact student models under guidance from teacher networks, including through intermediate feature imitation.Object-detection KD has used proposal regions, close anchor locations, discriminative patches, and different object-background weights.
  • Knowledge Distillation: Unlike feature-imitation methods, the proposed approach separately transfers classification and localization knowledge using logits and valuable localization regions.The method is designed to make distillation more efficient by matching each knowledge type to a separate distillation region.

3. Proposed Method

The proposed method separately distills semantic and localization knowledge, representing bounding-box edges as probability distributions and matching teacher–student localization outputs. It also selects valuable localization regions to focus distillation where localization information is useful.

  • 3. Proposed Method: The method uses divide-and-conquer distillation: classification KD transfers semantic knowledge, while LD transfers localization knowledge separately.The two forms of knowledge are distilled through their respective detector heads and preferred regions.
  • 3.1. Preliminaries: Probability-distribution bounding boxes model localization uncertainty more comprehensively than conventional Dirac delta representations.Ambiguous and clear box edges correspond to flat and sharp probability distributions, respectively.
  • 3.2. Localization Distillation: LD applies temperature-scaled SoftMax to teacher and student localization logits, then measures each edge’s distribution similarity with KL divergence.The four-edge localization loss is the sum of the per-edge KL-divergence losses.
  • 3.2. Localization Distillation: LD is presented as the first logit-mimicking approach for distilling localization knowledge in object detection, contrasting with prior emphasis on feature imitation.The method combines probability-distribution box representations with KL-divergence loss.
  • 3.3. Valuable Localization Region: The valuable localization region selects locations whose DIoU with ground-truth boxes lies between γαpos and αpos, complementing the main positive region.γ controls the region range: γ = 0 includes all locations below the positive threshold, while γ → 1 shrinks the region toward empty.
  • 3.4. Overall Distillation Process: The overall loss combines standard classification, regression, and distribution focal losses with separately masked LD and classification KD losses.The distribution focal loss can be disabled because LD is described as providing sufficient guidance ability.

4. Experiment

Experiments on MS COCO evaluate LD’s temperature, distillation regions, comparison with feature imitation, localization errors, student backbones, and detector architectures. LD consistently improves detection performance while introducing no inference overhead.

  • Ablation Studies and Analysis: LD with τ = 10 is fixed for subsequent experiments after varying temperatures consistently improves results over the no-distillation baseline.The teacher is ResNet-101 with 44.7 AP, and the student is ResNet-50.
  • Ablation Studies and Analysis: +0.4 AP and +0.7 AP75 are obtained by teacher bounded regression with ε = 0.1, whereas LD reaches 41.1 AP and 44.9 AP75 using probability-distribution bounding boxes.The paper attributes the difference to probability distributions containing localization knowledge absent from coarse bounding-box representations.
  • Ablation Studies and Analysis: 42.1 AP is achieved by the separate LD scheme, compared with 41.5 AP for GI imitation, and adding GI imitation raises the baseline gains to +2.3 AP and +3.1 AP75.The scheme uses logits rather than features and remains orthogonal to feature imitation methods.
  • Ablation Studies and Analysis: LD lowers box probability distribution errors, while adding classification KD reduces both classification-score and box-distribution errors.Main LD and Main LD + VLR LD have comparable or larger classification-score errors than feature imitation but lower box-distribution errors.
  • Ablation Studies and Analysis: LD reduces per-location localization-head logit errors more observably than GI imitation at the P5 and P6 FPN levels, alleviating localization ambiguity.The visualization uses Main LD + VLR LD, a model with slightly better AP than GI imitation.
  • Ablation Studies and Analysis: +1.7, +2.1, and +2.0 AP are obtained for ResNet-18, ResNet-34, and ResNet-50 students, while LD consistently improves RetinaNet, FCOS, and ATSS by approximately 2 AP.These experiments use a ResNet-101 teacher with 44.7 AP and evaluate dense detectors on MS COCO.

5. Conclusion

The paper proposes flexible localization distillation and valuable localization regions for separately transferring classification and localization knowledge in dense object detection. It reports that logit mimicking can outperform feature imitation, while noting extensions to sparse detectors and other fields as future research.

  • The method combines localization distillation with valuable localization regions in a separate distillation region manner for dense object detection.
  • The paper reports that logit mimicking can be better than feature imitation when distilling object detectors.
  • The separate distillation region strategy is presented as important for transferring classification and localization knowledge.
  • Applying LD to DETR-series sparse detectors, instance segmentation, object tracking, and 3D object detection is identified as future research.

A1. Implementation Details

The experiments use standard 1× and 2× training schedules with specified multi-GPU training, warm-up, and learning-rate decay settings.

  • Training uses 8 GPUs with 2 images per GPU and runs for 12 epochs under the 1× schedule or 24 epochs under the 2× schedule.
  • The initial learning rate is 0.01, with linear warm-up during the first 500 iterations.
  • For 1× training, the learning rate drops after epochs 8 and 11; for 2× training, it drops after epochs 16 and 22.

A2. More Method Studies

Additional studies evaluate individual LD, self-distillation, efficiency, and performance across detectors and datasets. They show localization supervision benefits students, usually adds little computational cost, and improves high-IoU detection quality.

  • Individual LD: 36.4 AP and 39.3 AP75 are achieved by ResNet-18 using only LD in the main distillation region without ground-truth box losses.Adding the regression and distribution focal losses increases AP by only 0.1, suggesting teacher probability distributions provide strong localization supervision.
  • Self-LD: +0.3 AP is obtained from one-round self-LD on the main distillation region.The text attributes the choice of one round to the balance between regularization and possible under-fitting from continued self-distillation.
  • Inference Speed: LD adds negligible model-size and FLOPs increases for FCOS and ATSS, slightly reduces RetinaNet FPS, and provides a free improvement for GFocal.The RetinaNet effect is linked to its nine anchor boxes per location and the resulting larger localization-head expansion.
  • LD for Lightweight Detectors on PASCAL VOC: LD consistently improves the student ResNet-18 on PASCAL VOC, with especially stronger gains for high-IoU metrics such as AP90.The results are reported on the PASCAL VOC 2007 test set.

A3. More Visualization

Visualizations compare GFocal with LD under different NMS settings. They indicate that LD produces more accurate localization and makes redundant detections easier to suppress with default NMS.

  • LD produces more accurate localization boxes than the original GFocal in the visualized detection results.
  • With default NMS, LD makes redundant boxes easier to suppress because its detected boxes have improved localization quality.The comparison uses GFocal and LD with default NMS, alongside a 0.95 threshold setting.
Loading 2102.12252v4…