PaperlayerView exact revision on arXiv

Source-linked AI summary

Distilling Object Detectors with Fine-grained Feature Imitation

Tao Wang, Li Yuan, Xiaopeng Zhang, Jiashi Feng

arXiv:1906.03609v1cs.CVcs.AIcs.LG

TL;DR

Object detection needs compact models, but classification-oriented knowledge distillation offers only minor gains and does not directly distill localization knowledge. The paper estimates informative near-object anchor locations from teacher feature-response discrepancies and imitates high-level features there, producing gains across lightweight detectors and Faster R-CNN settings.

  • Problem

    Knowledge distillation methods mainly target classification, while object detection additionally requires localization and contains highly imbalanced foreground and background instances.

  • Method

    Fine-grained feature imitation uses inter-location discrepancies in teacher feature responses to identify near-object anchor locations and guide student feature imitation there.

  • Results

    The method improves lightweight and Faster R-CNN students, including up to 15% mAP over a non-imitated counterpart and recovery of up to 74% of performance lost from a shallow backbone.

  • Takeaways & Limitations

    Fine-grained imitation transfers detection knowledge without relying on softened teacher outputs and avoids noisy, less-informative background regions.

Abstract

from arXiv · show

State-of-the-art CNN based recognition models are often computationally prohibitive to deploy on low-end devices. A promising high level approach tackling this limitation is knowledge distillation, which let small student model mimic cumbersome teacher model's output to get improved generalization. However, related methods mainly focus on simple task of classification while do not consider complex tasks like object detection. We show applying the vanilla knowledge distillation to detection model gets minor gain. To address the challenge of distilling knowledge in detection model, we propose a fine-grained feature imitation method exploiting the cross-location discrepancy of feature response. Our intuition is that detectors care more about local near object regions. Thus the discrepancy of feature response on the near object anchor locations reveals important information of how teacher model tends to generalize. We design a novel mechanism to estimate those locations and let student model imitate the teacher on them to get enhanced performance. We first validate the idea on a developed lightweight toy detector which carries simplest notion of current state-of-the-art anchor based detection models on challenging KITTI dataset, our method generates up to 15% boost of mAP for the student model compared to the non-imitated counterpart. We then extensively evaluate the method with Faster R-CNN model under various scenarios with common object detection benchmark of Pascal VOC and COCO, imitation alleviates up to 74% performance drop of student model compared to teacher. Codes released at https://github.com/twangnh/Distilling-Object-Detectors

1. Introduction

Object detectors are difficult to compress with standard distillation because detection requires localization and focuses on local object regions amid highly imbalanced foreground and background instances. The paper proposes fine-grained feature imitation, which identifies informative near-object anchor locations and transfers teacher feature responses there.

  • State-of-the-art detectors are cumbersome for low-computation devices, while quantization and pruning may require dedicated hardware or software customization for practical speedup.
  • Vanilla knowledge distillation provides only a minor boost for detection students because classification-oriented softened outputs do not capture localization knowledge and detection has few classes.
  • Directly imitating full high-level features hurts detection performance by introducing noise from background and other uncared regions.
  • Fine-grained feature imitation estimates knowledge-dense near-object anchor locations from inter-location feature-response discrepancies and makes the student imitate the teacher there.
  • Fine-grained imitation transfers information before classification and localization heads, addressing class discrimination, localization, and noisy background responses.
  • 15% mAP is the reported maximum boost for the lightweight student over its non-imitated counterpart, while Faster R-CNN experiments show broad applicability across PASCAL VOC and MSCOCO.

2. Related Works

Object-detection research has progressed from region-based CNN detectors to one-stage detectors designed for real-time inference. The paper also develops a lightweight detector partly for mobile implementation.

  • R-CNN detects objects by extracting and classifying each region of interest, with later works extending the framework for improved performance.
  • One-stage detectors were proposed to meet the requirement for real-time inference.
  • The authors design a lightweight detector partly for implementation on mobile devices.

3. Method

Fine-grained feature imitation trains a student detector to match teacher feature responses at informative anchor locations near ground-truth objects. The method estimates these regions from anchor–box overlap and combines imitation with detection supervision.

  • Method motivation: Near-object anchor locations are selected because their feature-response discrepancies convey how the teacher detects object instances.The method distills inter-location discrepancy rather than relying on softened teacher outputs.
  • Imitation region estimation: For each ground-truth box, anchor overlaps form a W × H × K IOU map, which is thresholded and combined into an imitation mask I.The threshold is F = ψ ∗ M, where M is the largest IOU value.
  • Imitation region estimation: ψ = 0.5 is used in all experiments because it provides the best distillation performance in the reported ablation study.The adaptive threshold avoids bias toward objects at particular scales, unlike a fixed IOU threshold.
  • Feature imitation: A convolutional adaptation layer aligns student and teacher feature channels before their responses are compared.The authors also report that direct feature matching can yield only minor gains even when channel dimensions are compatible.
  • Feature imitation: The student minimizes feature discrepancy over masked near-object locations, together with the ground-truth detection loss.The overall objective balances detection loss Lgt and imitation loss with λ.
  • Evaluation setup: Table 1 compares pruned baselines with proposed imitation, full-feature imitation, scaled-box regions, vanilla distillation, and combined losses.The 1× model is the base detector, while 0.5× and 0.25× models are pruned baselines trained with ground-truth supervision.

4. Experiments

The method is evaluated first on a lightweight toy detector with KITTI and then on Faster R-CNN under varied network settings and common detection benchmarks.

  • Evaluation scope: Experiments cover a lightweight toy detector on KITTI and Faster R-CNN under various network settings.The benchmarks include widely used common object detection datasets, while the toy detector captures the basic principle of anchor-based detection.

4.1. Lightweight detector

The lightweight detector refactors ShuffleNet for detection by preserving higher-resolution features, reducing computation, and adding detection-specific capacity.

  • Base architecture: The detector is based on ShuffleNet, which provides classification performance with limited FLOPs and parameters.Directly adapting the original classification architecture produced poor detection results because its top feature map had stride 32.
  • Architecture refactoring: Changing Conv1 stride from 2 to 1 preserves higher-resolution top features for the detection heads.The modification enables use of convolution layers while retaining higher spatial resolution.
  • Efficiency changes: Reducing Conv1 output channels from 24 to 16 lowers memory footprint and computation.The refactoring also reduces stage-3 blocks from 8 to 6, with slightly lower pre-training precision but no reported detection-performance harm.
  • Detection head: Two additional ShuffleNet blocks adapt high-level features before the regression and classification heads.The model uses a simple RPN-like detector with full-convolution detection heads, which the authors report significantly improves accuracy.

4.2. Imitation with lightweight detectors

On the lightweight toy detector, fine-grained imitation substantially improves severely compressed student models, while full feature imitation and vanilla distillation are ineffective or harmful.

  • 2.5 mAP improvement is achieved by the 0.5× student, while the 0.25× student gains 6.6 mAP with imitation.The 0.25× gain is 14.7% of the non-imitated model’s mAP.
  • Full feature imitation reduces the original 0.25× model’s performance by 8.9 mAP, despite performing well for classification.The authors attribute this to background noise overwhelming informative teacher supervision.
  • Scaling ground-truth boxes avoids background noise but misses supervision from near-object locations, producing less gain than the proposed method.
  • Adapting vanilla knowledge distillation to detection increases mAP by only 0.9, while combining its loss with imitation performs worse than imitation alone.

4.3. Imitation with Faster R-CNN

Fine-grained imitation improves compressed Faster R-CNN students across halved, shallow-backbone, and multi-layer settings on Pascal VOC and COCO.

  • The Faster R-CNN evaluation covers halved students, shallow students, and multi-layer imitation.
  • Halved student model: Halving the Faster R-CNN teacher produces a significant performance drop, but imitation adds 2.8 absolute mAP in Pascal-style average precision.Experiments use ResNet101-based Faster R-CNN on COCO and Pascal VOC07.
  • Shallow student network: Shallow-backbone students improve significantly with imitation, with the VGG11-based model gaining 8.0 absolute mAP.This nearly recovers 74% of the performance drop caused by the shallow backbone.
  • Multi-layer imitation: Multi-layer imitation with FPN yields 3.2 mAP gain in Pascal-style average precision and 3.6 mAP gain with COCO-style average precision.The student is a halved ResNet50 FPN-based Faster R-CNN model.

4.4. Analysis

The analysis examines how imitation changes detection behavior, error composition, region selection, and feature variance. Results indicate that selective imitation improves localization and discrimination while avoiding the noise associated with full feature imitation.

  • Region selection and feature variance: Figure 3 compares proposed masks at ψ = 0.2, 0.5, and 0.8 with constant-threshold masks at 0.5 and 0.8.Masks are calculated on the last feature map and enlarged to overlay the input image.
  • Qualitative performance gain from imitation: Imitated students improve discrimination, localization, duplicate suppression, background handling, and grouped-object detection relative to raw students.Qualitative examples show fewer sofa confusions, more accurate person boxes, fewer repeated tvmonitor detections, fewer background false positives, and fewer grouped detections.
  • Quantitative performance gain from imitation: Imitation increases correct detections and reduces Localization, Similar, Other, and Background errors across vehicle, animal, and furniture class groups.The largest reported reduction is especially associated with the Loc error category.
  • Region selection and feature variance: ψ = 0 degenerates to full feature imitation, severely misguiding the student and producing lower mAP than ground-truth-only training.The thresholding-factor experiment compares selective region retention against the full-imitation setting associated with hint learning.
  • Region selection and feature variance: Teacher feature responses exhibit different per-channel variances inside and outside the selected imitation region on KITTI and COCO.The variance analysis is used to investigate why unrestricted feature imitation deteriorates performance.

5. Conclusion

The paper presents fine-grained feature imitation for transferring knowledge from cumbersome detectors to smaller models. Its analysis supports selective use of teacher responses near object anchors, and the method can complement pruning and quantization.

  • Conclusion: Fine-grained feature imitation distills teacher knowledge through inter-location discrepancies in feature responses near object anchor locations.The method is described as simple to implement and designed for smaller object detectors.
  • Conclusion: The method is reported effective across the paper’s experiments and analyses.The conclusion summarizes the experimental evidence without specifying a single benchmark result.
  • Conclusion: Fine-grained feature imitation is orthogonal to pruning and quantization and can be combined with them.This identifies a supported compatibility with other model-acceleration methods.
Loading 1906.03609v1…