Source-linked AI summary
Distilling Object Detectors via Decoupled Features
Jianyuan Guo, Kai Han, Yunhe Wang, Han Wu, Xinghao Chen, Chunjing Xu, Chang Xu
TL;DR
Object-detector distillation often ignores background features and treats regions with equal importance, despite their relevance to student performance. DeFeat decouples neck features and classification proposals by region type, and experiments show consistent gains over other distillation techniques, including COCO improvements for Faster R-CNN and RetinaNet.
Problem
Existing object-detection distillation methods commonly ignore background features and negative proposals or treat different regions equally.
Method
DeFeat separately distills object and background FPN features and positive and negative RoI-aligned proposals using region-specific processing.
Results
DeFeat consistently outperforms other distillation techniques across one-stage and two-stage detectors, improving ResNet50 based FPN from 37.4% to 40.9% mAP on COCO.
Takeaways & Limitations
Background regions and negative proposals provide useful complementary information for distilling object detectors.
Abstract
from arXiv · showhide
Knowledge distillation is a widely used paradigm for inheriting information from a complicated teacher network to a compact student network and maintaining the strong performance. Different from image classification, object detectors are much more sophisticated with multiple loss functions in which features that semantic information rely on are tangled. In this paper, we point out that the information of features derived from regions excluding objects are also essential for distilling the student detector, which is usually ignored in existing approaches. In addition, we elucidate that features from different regions should be assigned with different importance during distillation. To this end, we present a novel distillation algorithm via decoupled features (DeFeat) for learning a better student detector. Specifically, two levels of decoupled features will be processed for embedding useful information into the student, i.e., decoupled features from neck and decoupled proposals from classification head. Extensive experiments on various detectors with different backbones show that the proposed DeFeat is able to surpass the state-of-the-art distillation methods for object detection. For example, DeFeat improves ResNet50 based Faster R-CNN from 37.4% to 40.9% mAP, and improves ResNet50 based RetinaNet from 36.5% to 39.7% mAP on COCO benchmark. Our implementation is available at https://github.com/ggjy/DeFeat.pytorch.
1. Introduction
Object-detector distillation must account for both object and background information, because background regions can complement object regions and reduce background false positives. DeFeat decouples features at multiple detector levels to assign region-specific distillation importance.
- Object detection uses computationally intensive backbones, motivating knowledge distillation to build compact detectors without sacrificing performance.
- Existing distillation methods primarily select object regions, often treating background features as noisy and uninformative.
- Background-only distillation can remarkably enhance students, achieve results comparable to object-region distillation, and reduce background false positives.
- Background information complements object-region distillation because object likelihood depends on both object and background features.
- DeFeat decouples FPN features into object and background parts and RoI-aligned features into positive and negative parts for separate distillation.
- DeFeat improves ResNet50 based FPN from 37.4% to 40.9% mAP and ResNet50 based RetinaNet from 36.5% to 39.7% mAP on COCO.
2. Related Work
Prior detection-distillation frameworks focus on foreground or uniformly weighted features, whereas this work identifies useful roles for background features and negative proposals and decouples them during distillation.
- Deep detector backbones improve accuracy but are expensive computationally and difficult to deploy on mobile devices.
- Earlier methods distilled fine-grained features from foreground object regions while leaving background features underexplored.
- DeFeat addresses these omissions by decoupling intermediate features into object and background regions and classification proposals into positive and negative groups.
3. Distillation via Decoupled Features
DeFeat decouples intermediate features and region proposals so object and background information receive separate distillation treatment. It balances region-specific feature losses and separately processes positive and negative proposal predictions.
- 3. Distillation via Decoupled Features: DeFeat combines intermediate-feature distillation with soft-prediction distillation over proposals, alongside the detector’s classification, regression, and RPN losses.The overall student objective includes feature, classification, regression, and RPN loss components where applicable.
- 3.1. Decouple Intermediate Features in Distillation: Background-only feature distillation achieves results comparable to object-only distillation, challenging the assumption that background mainly contributes noise.Earlier methods either selected partial regions or treated all intermediate-feature locations equally.
- 3.1. Decouple Intermediate Features in Distillation: DeFeat decouples neck features into object and background regions using a ground-truth-based binary mask.Object locations receive mask value 1 and background locations value 0; FPN boxes are assigned to corresponding pyramid levels.
- 3.1. Decouple Intermediate Features in Distillation: Object and background feature regions use separate loss coefficients, enabling both types of regions to be distilled in a balanced manner.The coefficients αobj and αbg control the respective region losses.
- 3.2. Decouple Region Proposals in Distillation: Positive and negative proposals are distilled separately because their proposal counts and distillation losses can differ substantially.The positive-proposal distillation loss is consistently larger than the negative-proposal loss, motivating separate balancing.
4. Experiments
Experiments evaluate DeFeat across detectors, backbones, student capacities, benchmarks, region-selection masks, proposal choices, and distillation configurations. Results consistently show gains from decoupling neck features and classification proposals, with further improvements from selected feature components.
- Main Results: 40.9% mAP is achieved on COCO after adding backbone-feature distillation to decoupled neck features and proposals, improving the baseline by 3.5%.For ResNet152-R50-FPN, decoupled neck features improve 40.1% to 40.4%, decoupled proposals improve 40.5% to 40.8%, and backbone features yield 40.9%.
- Comparison with State-of-the-art Methods: DeFeat consistently improves student detectors across Faster R-CNN and RetinaNet, including shallower and narrower students on COCO and Pascal VOC.On COCO, it surpasses FGFI by 1.0% mAP and TADF by 0.8% mAP; on VOC, ResNet152-R50-FPN rises from 80.5% to 82.3% mAP and ResNet101-R50-FPN from 80.5% to 81.5%.
- Impact of Background Regions: Background-only neck distillation can improve students, while background regions complement object-region distillation across shallow and narrow student settings.The experiments compare object-only and background-only FPN regions using ResNet152-to-ResNet50 and ResNet50-to-Quartered-ResNet50 distillation.
- Impact of Background Regions: Background-region distillation has smaller training loss and gradients than object-region distillation, motivating a larger loss weight for background regions.Classification and regression losses are about four times larger than distillation loss on object regions, helping explain the best object-region performance at γ = 4.
- Ablation Study: Negative-only proposal distillation outperforms positive-only distillation, and decoupling proposals improves performance from 38.2% to 38.9% mAP.Randomly matching the number of negative proposals to positive proposals decreases mAP by 0.2%, while teacher proposals outperform student proposals for shared-proposal distillation.
- Ablation Study: DeFeat minimizes teacher–student per-channel distances across both object and background regions more effectively than object-only or all-region neck distillation.The analysis averages pixel distances per FPN channel over object/background regions for students trained without KD, with region-restricted KD, with all-region KD, and with DeFeat.
5. Conclusion
DeFeat separates object and background information at both FPN and RoI-aligned feature levels, applying distillation separately to each. Experiments validate its effectiveness across one-stage and two-stage object detectors.
- DeFeat splits features into object and background parts at FPN and RoI-aligned feature levels, then distills the two parts separately.
- DeFeat is general and can be used with both one-stage and two-stage detection frameworks.
- Extensive experiments show DeFeat consistently outperforms other distillation techniques.