PaperlayerView exact revision on arXiv

Source-linked AI summary

Few-Shot Incremental Learning with Continually Evolved Classifiers

Chi Zhang, Nan Song, Guosheng Lin, Yun Zheng, Pan Pan, Yinghui Xu

arXiv:2104.03047v1cs.CVcs.AIcs.LG

TL;DR

FSCIL must learn new classes from few examples while retaining old-class knowledge and supporting evaluation over all classes. The paper decouples representation and classifier learning, and introduces a graph-based Continually Evolved Classifier trained with pseudo-incremental tasks. Across three benchmark datasets, the method significantly outperforms baselines and state-of-the-art approaches.

  • Problem

    FSCIL requires learning novel classes from scarce data without forgetting old classes, while classifiers learned in individual sessions must support evaluation over all classes.

  • Method

    The method freezes a base-trained backbone in incremental sessions, adapts classifiers with a graph-based Continually Evolved Classifier, and trains its graph through pseudo-incremental tasks sampled from base data.

  • Results

    The method significantly outperforms baselines and state-of-the-art approaches across CIFAR100, miniImageNet, and CUB200; its PD exceeds state-of-the-art results by 10.80% on CIFAR100 and 12.52% on miniImageNet.

  • Takeaways & Limitations

    A pre-trained backbone with decoupled classifier learning and continual classifier adaptation provides a strong FSCIL approach across the evaluated benchmarks.

  • Takeaways & Limitations

    The setting assumes that previous-session training data are unavailable during later sessions, while evaluation includes all classes seen so far.

Abstract

from arXiv · show

Few-shot class-incremental learning (FSCIL) aims to design machine learning algorithms that can continually learn new concepts from a few data points, without forgetting knowledge of old classes. The difficulty lies in that limited data from new classes not only lead to significant overfitting issues but also exacerbate the notorious catastrophic forgetting problems. Moreover, as training data come in sequence in FSCIL, the learned classifier can only provide discriminative information in individual sessions, while FSCIL requires all classes to be involved for evaluation. In this paper, we address the FSCIL problem from two aspects. First, we adopt a simple but effective decoupled learning strategy of representations and classifiers that only the classifiers are updated in each incremental session, which avoids knowledge forgetting in the representations. By doing so, we demonstrate that a pre-trained backbone plus a non-parametric class mean classifier can beat state-of-the-art methods. Second, to make the classifiers learned on individual sessions applicable to all classes, we propose a Continually Evolved Classifier (CEC) that employs a graph model to propagate context information between classifiers for adaptation. To enable the learning of CEC, we design a pseudo incremental learning paradigm that episodically constructs a pseudo incremental learning task to optimize the graph parameters by sampling data from the base dataset. Experiments on three popular benchmark datasets, including CIFAR100, miniImageNet, and Caltech-USCD Birds-200-2011 (CUB200), show that our method significantly outperforms the baselines and sets new state-of-the-art results with remarkable advantages.

1. Introduction

FSCIL must learn novel classes from few examples while retaining old-class knowledge and evaluating all classes together. The paper addresses these challenges by decoupling representation and classifier learning, then adapting session-specific classifiers with a graph-based Continually Evolved Classifier trained through pseudo-incremental tasks.

  • Motivation: FSCIL combines scarce new-class data, severe class imbalance, and catastrophic forgetting when models are updated incrementally.New classes provide few labeled examples, while old-session data may no longer be available for fine-tuning.
  • Decoupled Learning: The decoupled strategy learns representations only in the base session, then fixes the backbone and adapts classifiers in later sessions.This is intended to avoid overfitting and knowledge forgetting in the representations.
  • Decoupled Learning: A pre-trained backbone with a class mean classifier can beat state-of-the-art approaches.
  • Continually Evolved Classifier: CEC uses a graph model to adapt classifiers learned in individual sessions so predictions can involve all classes.The adaptation accounts for current and historical task context.
  • Training Paradigm: Pseudo incremental learning samples episodic tasks from base-session data to train the graph model under an incremental-learning scenario.Large-angle rotations synthesize unfamiliar classes and encourage context propagation in the graph model.
  • Experiments: Experiments on CIFAR100, CUB200, and miniImagenet report performance that significantly outperforms baselines and sets new state-of-the-art results.

2. Related Work

The related work situates FSCIL among few-shot, incremental, and few-shot class-incremental learning. It contrasts optimization- and metric-based few-shot methods with incremental-learning approaches such as distillation and nearest-neighbor classifiers.

  • Few-Shot Learning: Few-shot learning studies classification of unseen images from scarce labeled training examples.The literature includes optimization-based and metric-based approaches.
  • Few-Shot Learning: Optimization-based methods target fast network adaptation with limited data, whereas metric-based methods rely on learned representations and comparison mechanisms.
  • Incremental Learning: Incremental learning aims to acquire new knowledge continually without forgetting existing knowledge.
  • Incremental Learning: Early incremental-learning approaches use knowledge distillation, while iCaRL combines exemplar-based nearest-neighbor classification with distillation loss.
  • Few-Shot Class-Incremental Learning: FSCIL combines few-shot learning with incremental learning by classifying novel and old classes together under limited data.

3. Problem Set-up

FSCIL presents sequential learning sessions in which only current-session training data are available, while evaluation covers all classes seen so far. The setting combines a data-rich base session with severely data-limited incremental sessions.

  • Task Definition: FSCIL learns novel classes from few examples without forgetting old-class knowledge across sequential sessions.
  • Task Definition: Training classes are disjoint across sessions, with each session using only its own training set.The label spaces satisfy Ci∩Cj = ∅ for different sessions.
  • Task Definition: At each session, previous training datasets are unavailable, but evaluation includes test data from all previous and current classes.
  • Data Regime: The base session provides relatively abundant training data, whereas subsequent sessions contain limited data.
  • Data Regime: In CIFAR100, the base session has 60 classes with 500 images per class, while each incremental session has 5 classes with 5 images per class.

4. Method

The method decouples representation and classifier learning, then uses a graph-based Continually Evolved Classifier to adapt session-specific classifiers with global context. A pseudo-incremental training procedure learns this adaptation module from base-session data.

  • Framework overview: The framework has three stages: feature pre-training, pseudo-incremental learning, and classifier learning in each new session.The first two stages use base-session data; later sessions update only the classifier on a fixed backbone.
  • Decoupled representation and classifier learning: Decoupled training fixes the backbone after base-session representation learning and adapts only classifiers in new sessions.This is intended to avoid representation overfitting and catastrophic forgetting under severely unbalanced base and novel-class data.
  • Continually Evolved Classifier: CEC updates session-specific classifier weights by propagating global context among class prototype vectors with a Graph Attention Network.The graph treats classifier weight vectors as nodes, and its attention-based updates are permutation invariant and reusable for any number of classes.
  • Graph attention adaptation: The graph adaptation computes pairwise relation coefficients, normalizes them with softmax attention, aggregates node information, and fuses it with each original node representation.Linear transformations project prototype vectors into a new metric space before similarity-based attention is computed.
  • Pseudo-incremental learning: Pseudo-incremental learning episodically samples support and query sets from the base dataset to mimic incremental tasks and optimize the GAT.Sampled classes serve as pseudo-base or pseudo-incremental classes, with rotated pseudo-incremental classes used to enforce context propagation.

5. Experiments

Experiments evaluate the method on three FSCIL benchmarks, analyze its components and pseudo-incremental configurations, and compare it with state-of-the-art methods. The adaptation module improves predictions, while decoupled training and pseudo-incremental learning provide measurable benefits.

  • Experiment Setup: Experiments evaluate the method on CIFAR100, miniImageNet, and CUB200 using session-wise Top 1 accuracy and performance dropping rate.PD is defined as the accuracy difference between the base and last sessions.
  • Ablation Study: Decoupling representation and classifier learning reduces performance dropping rate by 28.81% for cosine classifiers and 3.39% for linear classifiers.The ablation attributes this benefit to avoiding catastrophic forgetting during incremental learning.
  • Ablation Study: Pseudo incremental learning improves performance across sessions by up to 3.13% and decreases performance dropping rate by 2.87%.Meta-learning the adaptation module without pseudo incremental learning fails to improve performance.
  • Adaptation Analysis: The adaptation module produces less scattered CIFAR100 confusion matrices, with predictions concentrated more closely on the diagonal.The t-SNE analysis likewise reports classifier weights moving away from confusion areas to form better decision boundaries.
  • Pseudo-Incremental Learning: Pseudo-incremental learning performs best with 15 ways and 1 shot, while randomly selecting 180°, 90°, and −90° rotations yields 61.33% average accuracy and 23.57% performance dropping.Larger ways, fewer shots, and larger rotation angles are preferred in the reported CUB200 analyses.
  • State-of-the-Art Comparison: On CIFAR100, miniImageNet, and CUB200, the method achieves the highest average accuracy and lowest performance dropping rate among the compared methods.The reported performance-dropping advantages over state-of-the-art results are 10.80% on CIFAR100 and 12.52% on miniImageNet; the CUB200 value is truncated in the supplied passage.

6. Conclusion

The paper addresses few-shot incremental learning by separating representation and classifier learning, then adapting session classifiers using global context. Experiments on three datasets show that the method significantly outperforms baselines and state-of-the-art approaches.

  • The decoupled strategy separates representation learning from classifier learning to avoid knowledge forgetting in the backbone.
  • The continually evolved classifier updates classifier weights using global context from all sessions.
  • A pseudo incremental learning paradigm enables training of the classifier adaptation module.
  • Experiments on three datasets show significant improvements over baselines and state-of-the-art approaches.

A. Introduction

The supplementary material provides additional experimental details.

  • The supplementary material presents more details about the paper’s experiments.

B. Detailed Result

The detailed results complement line-chart comparisons with tabulated state-of-the-art results across CIFAR100, miniImagenet, and CUB200. The method achieves new state-of-the-art performance on all three datasets.

  • The detailed numbers are presented in Table 3 alongside the line-chart comparison in Fig. 7.
  • The method achieves new state-of-the-art performance on CIFAR100, miniImagenet, and CUB200.
  • The CUB200 results use a 10-way 5-shot FSCIL setting.
Loading 2104.03047v1…