Source-linked AI summary
Dense Relation Distillation with Context-aware Aggregation for Few-Shot Object Detection
Hanzhe Hu, Shuai Bai, Aoxue Li, Jinshi Cui, Liwei Wang
TL;DR
Few-shot object detection must detect novel classes from scarce bounding-box annotations without losing fine-grained object information. DCNet uses dense support-query relation distillation and context-aware aggregation within a meta-learning framework. The method achieves state-of-the-art results on PASCAL VOC and MS COCO.
Problem
Few-shot object detection must adapt to novel classes from only a few annotated examples, making fine-grained features easy to overlook.
Method
DCNet combines dense matching between query and support features with adaptive aggregation of features from different scales in a meta-learning framework.
Results
DCNet achieves state-of-the-art results on the PASCAL VOC and MS COCO benchmark datasets.
Takeaways & Limitations
Dense support guidance and multi-scale context aggregation provide a comprehensive feature representation for few-shot object detection.
Abstract
from arXiv · showhide
Conventional deep learning based methods for object detection require a large amount of bounding box annotations for training, which is expensive to obtain such high quality annotated data. Few-shot object detection, which learns to adapt to novel classes with only a few annotated examples, is very challenging since the fine-grained feature of novel object can be easily overlooked with only a few data available. In this work, aiming to fully exploit features of annotated novel object and capture fine-grained features of query object, we propose Dense Relation Distillation with Context-aware Aggregation (DCNet) to tackle the few-shot detection problem. Built on the meta-learning based framework, Dense Relation Distillation module targets at fully exploiting support features, where support features and query feature are densely matched, covering all spatial locations in a feed-forward fashion. The abundant usage of the guidance information endows model the capability to handle common challenges such as appearance changes and occlusions. Moreover, to better capture scale-aware features, Context-aware Aggregation module adaptively harnesses features from different scales for a more comprehensive feature representation. Extensive experiments illustrate that our proposed approach achieves state-of-the-art results on PASCAL VOC and MS COCO datasets. Code will be made available at https://github.com/hzhupku/DCNet.
1. Introduction
Few-shot object detection must generalize to novel classes from scarce annotations while preserving fine-grained, scale-aware object features. DCNet addresses these challenges with dense support-query relation distillation and context-aware feature aggregation, achieving state-of-the-art results on PASCAL VOC and MS COCO.
- Few-shot object detection seeks generalization to novel classes from only a few annotated examples, unlike conventional detection requiring large labeled datasets.
- Global pooling in prior methods can lose detailed local context, leaving appearance changes, occlusions, and scale variation as major detection challenges.
- Dense relation distillation densely matches query and support features to exploit local support information for novel-class detection.
- Context-aware feature aggregation uses multiple pooling resolutions and attention to combine scale-specific features into a more comprehensive representation.
- DCNet achieves consistent improvement and state-of-the-art performance on PASCAL VOC and MS COCO.
2. Related Work
Related work spans one-stage and two-stage object detectors, general few-shot learning methods, and approaches specifically designed for few-shot object detection. Existing few-shot detection research includes transfer learning, meta-learning, and fine-tuning strategies, while few-shot detection remains less explored than classification.
- Object detectors are broadly categorized as one-stage methods, which directly predict classes and boxes, and two-stage methods, which refine region proposals.
- Few-shot learning methods include Bayesian inference, metric learning, optimization-based adaptation, and cross-attention for transferring knowledge to new classes.
- Few-shot object detection targets novel-class detection with few annotations and uses transfer learning frameworks such as LSTD and RepMet to reduce overfitting.
- Meta-learning detection methods reweight features or perform class-specific prediction, whereas some approaches use two-stage fine-tuning or address scale variation.
3. Method
DCNet combines dense support-query relation distillation with context-aware feature aggregation in a meta-learning few-shot detection framework. It refines query features using support information and adaptively aggregates multiscale RoI features before detection.
- Dense Relation Distillation Module: Retrieved support values are combined with query values to produce a refined query feature activated by co-existing objects across query and support images.The module sums outputs from the N support features before passing the refined query feature to the RPN.
- Dense Relation Distillation Module: The method replaces global support-feature pooling with dense relations that preserve local details useful under appearance changes and occlusions.Prior holistic modulation can be misleading when same-class objects vary substantially or are partially unseen.
- Dense Relation Distillation Module: The dense relation distillation module performs cross-attention-style, pixel-wise matching between query and support features to retrieve relevant support information.Learned key maps measure similarities, while value maps store detailed recognition information for support-feature retrieval.
- Context-aware Feature Aggregation: After DRD refinement, Faster R-CNN generates proposals with RPN, and RoI Align extracts features for class prediction and bounding-box regression.CFA addresses information loss associated with the fixed pooling resolution used in the original implementation.
- Context-aware Feature Aggregation: Context-aware feature aggregation adaptively fuses features from different RoI pooling scales using attention weights normalized by softmax.The final representation is a weighted summation of three pooled features, with larger resolutions emphasizing local detail and smaller resolutions capturing holistic information.
- Learning Strategy: DCNet uses episodic meta-training and meta fine-tuning, training on base classes before jointly adapting with limited base and novel annotations.Novel classes contribute k labeled bounding boxes, while k boxes per base class balance the fine-tuning samples.
4. Experiments
Experiments on PASCAL VOC and MS COCO evaluate DCNet against prior methods and isolate the contributions of dense relation distillation and context-aware aggregation. Results show strong few-shot detection performance, especially in low-shot settings, with complementary benefits from both modules.
- PASCAL VOC Results: DCNet achieves state-of-the-art results on almost all PASCAL VOC splits and shot settings, outperforming previous methods by about 10% in split 1 and split 3 under 1-shot evaluation.The comparison uses AP50 on the VOC 2007 test set across three novel-class splits.
- Ablation Study: The dense relation distillation module consistently improves over the baseline across novel splits and shot numbers, with larger gains when support data is scarce.The ablation compares DCNet without CFA against a Meta R-CNN-based baseline.
- Ablation Study: The context-aware feature aggregation module improves baseline performance by preserving detailed information across object scales, while attentive aggregation further boosts performance.The attention mechanism balances contributions from features produced at different RoI pooling resolutions.
- Ablation Study: Aggregating features from all three RoI pooling resolutions produces the best performance, rather than using only a larger pooling resolution.The pooling-resolution study is conducted on VOC 2007 test split 1 for the 10-shot task.
- Qualitative Results: Qualitative visualizations show that dense relation distillation activates query-object features, while context-aware aggregation reduces false classifications and missed detections for large or small objects.The visualizations compare features before and after DRD and illustrate CFA effects on scale variation.
- MS COCO Results: On MS COCO, DCNet achieves state-of-the-art performance on most metrics for novel classes despite the dataset’s large number of categories.Results are reported for 10-shot and 30-shot setups using mean average precision and mean average recall over 20 novel classes.
5. Conclusions
The paper presents DCNet for few-shot object detection, combining dense query-support matching with adaptive multi-scale feature aggregation. Ablations support both components, and the method achieves state-of-the-art results on PASCAL VOC and MS COCO.
- Conclusions: DCNet combines dense matching between query and support features with adaptive aggregation of features from different scales.Dense relation distillation exploits support information, while context-aware aggregation produces a more comprehensive feature representation.
- Conclusions: Ablation experiments demonstrate the effectiveness of each DCNet component.
- Conclusions: DCNet achieves state-of-the-art results on the PASCAL VOC and MS COCO benchmark datasets.