Source-linked AI summary
Incremental Few-Shot Object Detection
Juan-Manuel Perez-Rua, Xiatian Zhu, Timothy Hospedales, Tao Xiang
TL;DR
Object detectors usually depend on abundant labels and offline batch training, limiting incremental accommodation of novel classes and complicating deployment. The paper formulates iFSD and proposes ONCE, an adapted CentreNet with a meta-learned class-specific code generator. ONCE registers novel classes feed-forward without base-class access, and experiments across object and landmark detection show the feasibility of iFSD.
Problem
Existing object detectors require abundant per-class annotation and offline batch training, limiting open-ended incremental accommodation of novel classes.
Method
ONCE adapts CentreNet by separating class-generic and class-specific components and meta-learning a code generator for feed-forward novel-class registration.
Results
ONCE shows superior performance over alternatives on both object detection and fashion landmark detection tasks in the incremental few-shot setting.
Takeaways & Limitations
The study provides a promising initial solution and first evidence for incremental few-shot detection without revisiting base-class training data.
Takeaways & Limitations
The incremental setting is more challenging and can yield inferior performance compared with non-incremental learning.
Abstract
from arXiv · showhide
Most existing object detection methods rely on the availability of abundant labelled training samples per class and offline model training in a batch mode. These requirements substantially limit their scalability to open-ended accommodation of novel classes with limited labelled training data. We present a study aiming to go beyond these limitations by considering the Incremental Few-Shot Detection (iFSD) problem setting, where new classes must be registered incrementally (without revisiting base classes) and with few examples. To this end we propose OpeN-ended Centre nEt (ONCE), a detector designed for incrementally learning to detect novel class objects with few examples. This is achieved by an elegant adaptation of the CentreNet detector to the few-shot learning scenario, and meta-learning a class-specific code generator model for registering novel classes. ONCE fully respects the incremental learning paradigm, with novel class registration requiring only a single forward pass of few-shot training samples, and no access to base classes -- thus making it suitable for deployment on embedded devices. Extensive experiments conducted on both the standard object detection and fashion landmark detection tasks show the feasibility of iFSD for the first time, opening an interesting and very important line of research.
1. Introduction
The paper defines Incremental Few-Shot Detection as learning novel classes from few examples without revisiting base data, addressing scalability, forgetting, and deployment constraints. It introduces ONCE, which adapts CentreNet and meta-learns class-specific codes for feed-forward novel-class registration.
- Problem: Existing few-shot detectors remain non-incremental because adding a class requires costly retraining with both old and new class data.This creates quadratic computation in the number of categories and raises data-privacy concerns.
- Problem: Incremental Few-Shot Detection targets online registration of novel classes from few annotated examples while retaining performance on previously learned classes.The setting also requires bounded memory, storage, and computation for resource-limited devices.
- Problem: Fine-tuning novel classes without old data causes catastrophic forgetting, making conventional detectors and existing few-shot methods unsuitable for iFSD.The paper identifies dramatic degradation on existing categories as the central failure mode.
- Approach: ONCE decomposes CentreNet into class-generic and class-specific components, training the former on base data and meta-learning a code generator for novel classes.Given a handful of novel-class images, the generator registers the class without base data or iterative updating.
- Evaluation: The study evaluates iFSD on standard object detection and fashion landmark detection, reporting significant performance advantages over existing alternatives.The experiments cover COCO, PASCAL VOC, and DeepFashion2.
2. Related Work
The related work spans conventional two-stage and one-stage detectors, few-shot learning, and few-shot object detection. The paper distinguishes iFSD by requiring incremental learning without repeatedly revisiting large base-class datasets.
- Object Detection: Two-stage detectors generally achieve stronger detection performance, but one-stage detectors are more efficient because they avoid region proposal inference and subsequent proposal classification.Both detector families typically assume many training images per class.
- Few-Shot Learning: Few-shot learning meta-learns data-efficient strategies that adapt novel classes from very limited per-class examples using abundant base-class examples.A major concern is learning from scarce data without overfitting.
- Few-Shot Object Detection: Recent few-shot object detection methods consider non-incremental learning, limiting scalability when large-scale base data or computational resources are unavailable.The paper positions iFSD as a more practical alternative for such settings.
- Few-Shot Object Detection: Incremental few-shot learning avoids repeatedly training on large base datasets, but the paper notes that this harder setting can produce inferior performance than non-incremental learning.Its application potential is tied to more representative human-like learning capabilities.
- Related Paradigms: Weakly supervised and zero-shot learning use different training data or prior knowledge and are complementary techniques that can be combined with iFSD when multiple input sources exist.Examples include unlabelled data and semantic class descriptors.
3. Methodology
ONCE adapts CentreNet for incremental few-shot detection by separating class-generic features from class-specific localization and meta-learning codes for novel classes. This design supports few-shot class enrollment without revisiting base data or updating the model.
- Incremental Few-Shot Detection: iFSD incrementally recognizes novel classes from few labelled examples while preserving base-class knowledge and limiting memory, storage, and computation.The setting assumes abundant base-class data and deployment where novel classes are enrolled over an unbounded stream.
- CentreNet Architecture: CentreNet represents detection through per-class heatmaps for object centres and sizes, enabling independent class predictions without softmax interactions.Its one-stage point-and-attribute formulation predicts aligned heatmaps and identifies detections from local maxima.
- ONCE Model: ONCE decomposes CentreNet into a shared feature extractor and a class-specific object locator whose convolutional parameters are generated from few-shot support samples.The generator synthesizes locator weights, converting conventional locator training into a feed-forward parameter-generation step.
- Training: Training proceeds in two stages: base-class supervised learning trains the feature extractor, then episodic meta-training learns the code generator with the extractor frozen.Base classes act as fake novel classes during meta-training, while the Stage I locator remains available for base-class testing.
- Meta-Testing: At test time, ONCE generates a new class code from labelled support samples and detects that class in query images through a feed-forward pass.Novel-class enrollment requires no model adaptation or update, and the resulting process supports independent introduction of novel classes.
4. Experiments
Experiments evaluate ONCE for incremental few-shot object detection on COCO and cross-dataset transfer to VOC, alongside fashion landmark detection on DeepFashion2. Across these settings, ONCE maintains performance as classes are added, outperforms alternatives, and supports accurate one-shot landmark detection.
- Object detection on COCO: COCO experiments compare ONCE with Fine-Tuning, MAML, and Feature-Reweight under incremental few-shot settings.The evaluation uses CentreNet/ResNet50 implementations and includes same-dataset and cross-dataset protocols.
- Object detection on COCO: ONCE achieves the best performance on most COCO experiments for novel and base classes simultaneously, with larger improvements when fewer shots are available.The method keeps base-class performance unchanged through class-specific detector learning and avoids test-time optimisation.
- Experimental considerations: The reported base-class AP is lower than in normal supervised training because of early stopping, which helps prevent overfitting to known classes and supports novel-class generalisation.The authors state that early stopping is important for iFSD despite lowering base-class AP.
- Incremental evaluation: ONCE changes little in all-classes accuracy as novel classes are added incrementally, while competitors drop quickly because of increased forgetting.The continuous incremental evaluation uses 10 shots per novel class and excludes MAML because it cannot detect base classes in this setting.
- Object detection transfer from COCO to VOC: COCO-to-VOC transfer preserves the same relative performance pattern, with higher novel-class performance on VOC for all methods.The transfer evaluation trains on COCO and evaluates novel classes on the VOC2007 test set; no base-class images are available in VOC.
- Fashion landmark detection: On DeepFashion2, ONCE consistently and significantly outperforms Fine-Tuning, including when only one shot is available for a novel landmark class.The benchmark contains 294 landmark classes, split category-disjointly into training, validation, and testing categories; the authors attribute higher accuracy to transferable clothing-landmark knowledge.
5. Conclusion
ONCE provides an initial solution to incremental few-shot object detection by registering novel classes from few examples without revisiting base-class data. It outperforms alternatives across object and landmark detection tasks, while motivating further work on iFSD.
- ONCE registers novel classes incrementally from few examples without revisiting base-class training data.
- ONCE achieves superior performance over several alternatives on both object and landmark detection tasks in the incremental few-shot setting.
- The study provides evidence that more effective solutions to incremental few-shot detection remain necessary.