Source-linked AI summary
LSTD: A Low-Shot Transfer Detector for Object Detection
Hao Chen, Yali Wang, Guoyou Wang, Yu Qiao
TL;DR
Object detection performance depends on large fully annotated datasets, but target tasks may have only a few annotated images. LSTD transfers a pretrained source-domain detector and applies TK and BD regularization during target fine-tuning; it outperforms other state-of-the-art approaches in challenging low-shot experiments.
Problem
Fully annotated target detection data may be limited, while general transfer strategies face task differences and detector overfitting in low-shot settings.
Method
LSTD combines SSD and Faster RCNN designs with source-domain pretraining and TK and BD regularization during target-domain fine-tuning.
Results
LSTD outperforms other state-of-the-art approaches on a number of challenging low-shot detection experiments.
Takeaways & Limitations
LSTD is presented as a preferable deep detector for low-shot detection scenarios.
Takeaways & Limitations
Weakly or semi-supervised detectors may have limited performance because their training images lack sufficient supervision.
Abstract
from arXiv · showhide
Recent advances in object detection are mainly driven by deep learning with large-scale detection benchmarks. However, the fully-annotated training set is often limited for a target detection task, which may deteriorate the performance of deep detectors. To address this challenge, we propose a novel low-shot transfer detector (LSTD) in this paper, where we leverage rich source-domain knowledge to construct an effective target-domain detector with very few training examples. The main contributions are described as follows. First, we design a flexible deep architecture of LSTD to alleviate transfer difficulties in low-shot detection. This architecture can integrate the advantages of both SSD and Faster RCNN in a unified deep framework. Second, we introduce a novel regularized transfer learning framework for low-shot detection, where the transfer knowledge (TK) and background depression (BD) regularizations are proposed to leverage object knowledge respectively from source and target domains, in order to further enhance fine-tuning with a few target images. Finally, we examine our LSTD on a number of challenging low-shot detection experiments, where LSTD outperforms other state-of-the-art approaches. The results demonstrate that LSTD is a preferable deep detector for low-shot scenarios.
1 Introduction
LSTD addresses low-shot object detection by transferring source-domain detector knowledge to scarce target data. It combines SSD and Faster RCNN designs with TK and BD regularization to support target-domain fine-tuning.
- Fully annotated target training sets may be limited, restricting the power of deep object detectors.
- Transfer learning remains challenging because small target sets cannot readily eliminate detection–classification task differences and detectors overfit more readily than classifiers.
- LSTD integrates SSD-style multi-convolutional bounding-box regression with Faster RCNN-style coarse-to-fine object classification.
- TK transfers source-object-label knowledge to target proposals, while BD uses target bounding-box knowledge as extra feature-map supervision to suppress background disturbances.
- LSTD outperforms other state-of-the-art approaches across challenging low-shot detection experiments.
2 Related Works
The related work contrasts standard deep detectors and existing low-shot methods with LSTD, which combines detector architectures and transfers object knowledge from source and target domains.
- Object Detection: Faster RCNN uses region proposals followed by Fast RCNN, whereas SSD is a one-stage detector with multi-layer bounding-box regression for varied object sizes.
- Object Detection: Directly applying Faster RCNN or SSD with few examples is problematic because their architectures lack low-shot considerations.
- Low-shot Learning: Existing low-shot learning approaches mainly address classification rather than object detection.
- LSTD: LSTD combines Faster RCNN and SSD advantages and uses regularization to integrate object knowledge from source and target domains.
3 Low-Shot Transfer Detector (LSTD)
LSTD combines SSD-style bounding-box regression with Faster RCNN-style coarse-to-fine classification, then transfers source and target object knowledge through TK and BD regularization for low-shot detection.
- Basic Deep Architecture of LSTD: Pretrained SSD regression parameters can initialize the target domain, avoiding random re-initialization of bounding-box regression with few target images.This reduces fine-tuning burdens when target data are scarce.
- Basic Deep Architecture of LSTD: LSTD combines SSD-style multi-layer bounding-box regression with Faster RCNN-style coarse-to-fine object classification in a unified architecture.The design separates regression and classification across relatively separate places to decompose low-shot learning difficulties.
- Regularized Transfer Learning for LSTD: LSTD fine-tunes a source-domain detector on the target domain with a regularization combining background-depression and transfer-knowledge terms.The total regularization uses coefficients λBD and λTK for LBD and LTK.
- Regularized Transfer Learning for LSTD: BD masks middle-level feature cubes using target ground-truth boxes and penalizes background activation, helping suppress background regions and focus on target objects.Figure 3 reports reduced background disturbances on feature heatmaps.
- Regularized Transfer Learning for LSTD: TK supplies source-object knowledge for target proposals despite different categories, using source-class soften-labels to transfer relations such as color or shape similarity.Figure 4 illustrates relevant source categories for Cow/Aeroplane proposals through Bear/Kite soften-labels.
1. Pre-training on Large-scale Source Domain
Source-domain LSTD is trained using a large-scale detection benchmark.
- Pre-training on Large-scale Source Domain: Source-domain LSTD is trained with a large-scale detection benchmark.
2. Initialization for Small-scale Target Domain
Target-domain LSTD reuses the source detector except for the object classifier, which is reinitialized because source and target object categories differ.
- Initialization for Small-scale Target Domain: The last layer of the (K + 1)-object classifier is randomly initialized because source and target object categories differ.All other parts of target-domain LSTD are initialized from source-domain LSTD.
3. Fine-tuning for Small-scale Target Domain
LSTD fine-tunes the source-pretrained detector on a small target set with BD and TK regularizations, integrating source-domain knowledge into target-domain learning.
- Fine-tuning for Small-scale Target Domain: Target-domain LSTD is fine-tuned with BD and TK regularizations using the small training set.
- Fine-tuning for Small-scale Target Domain: Target-domain proposals are passed through the source-domain LSTD ROI pooling and object classifier to generate source-domain knowledge vectors.
- Fine-tuning for Small-scale Target Domain: The source classifier converts proposal activations into soften-labels using softmax with temperature τ > 1, retaining richer label-relation information.
- Fine-tuning for Small-scale Target Domain: A source-object soften classifier is added to target-domain LSTD so each target proposal predicts source-object categories as an auxiliary task.
- Fine-tuning for Small-scale Target Domain: Cross-entropy between source-domain knowledge and target predictions forms the TK regularization, integrating source knowledge into target training.
4 Experiment
The experiments evaluate LSTD across transfer tasks with scarce target-domain annotations, comparing its architecture, regularizations, robustness, and state-of-the-art performance. LSTD improves low-shot detection over closely related and weakly/semi-supervised alternatives under the reported settings.
- Experimental setup: LSTD is evaluated on three transfer tasks using large-scale source training and 1/2/5/10/30 target training images per object class.The datasets include COCO, ImageNet2015, VOC2007, and VOC2010.
- Basic Deep Structure of LSTD: In Task 1, LSTD achieves competitive source-domain mAP and significantly outperforms SSD and Faster RCNN for one-shot target detection.The comparison uses models fine-tuned from source-domain pretrained models.
- Regularized Transfer Learning for LSTD: TK and BD regularizations significantly improve LSTD fine-tuning over the baseline, especially when the target training set is scarce, such as in one-shot detection.BD remains generally robust when implemented on different convolutional layers.
- Comparison with the State-of-the-art: LSTD outperforms SSD and Faster RCNN in Task 1 as target-shot counts change, and surpasses weakly- or semi-supervised detectors beyond two shots in Tasks 2 and 3.The weakly- and semi-supervised results are taken from their original papers on the same test set.
- Comparison with the State-of-the-art: LSTD outperforms competing detectors on Task 3 with 0.4% of training data and is competitive with fully supervised detectors using 11% of the training set.At 11%, the reported mAP values are LSTD:69.7, SSD:68.0, and Faster RCNN:69.9.
- Visualization: Visualization shows that one-shot LSTD can localize but sometimes misclassifies objects, while five-shot training largely clarifies these errors.For 2-shot LSTD on VOC 2007, Animal errors are reported as 23% localization, 71% similarity, and 4% background.
5 Conclusion
The conclusion presents LSTD as a low-shot transfer detector for object detection with few training images. It combines a flexible architecture with TK and BD regularization and reports superior performance across challenging experiments.
- LSTD addresses object detection with a few training images through a flexible deep architecture designed to reduce low-shot transfer difficulties.
- The architecture incorporates strengths of SSD and Faster RCNN within a unified deep framework.
- TK and BD terms form a regularized transfer-learning framework intended to generalize fine-tuning with few target images.
- LSTD outperforms other state-of-the-art approaches across challenging low-shot detection experiments.