PaperlayerView exact revision on arXiv

Source-linked AI summary

MgSvF: Multi-Grained Slow vs. Fast Framework for Few-Shot Class-Incremental Learning

Hanbin Zhao, Yongjian Fu, Mintong Kang, Qi Tian, Fei Wu, Xi Li

arXiv:2006.15524v4cs.CVcs.LG

TL;DR

Few-shot class-incremental learning must preserve old knowledge while adapting to new classes from limited data. MgSvF tackles this dilemma with intra-space frequency-aware regularization and inter-space feature composition, and reports substantially better performance than state-of-the-art methods. The method is evaluated under protocols where new sessions contain few samples and classes do not overlap across tasks.

  • Problem

    FSCIL must balance slow forgetting of old knowledge with fast adaptation to new knowledge when new tasks provide only a few samples.

  • Method

    MgSvF uses frequency-aware regularization within a feature space and feature-space composition between a slowly updated space and another space.

  • Results

    The proposed approach significantly outperforms other state-of-the-art approaches by a large margin across the evaluated datasets and protocols.

  • Takeaways & Limitations

    Analyzing slow-versus-fast learning at multiple feature-space grains provides an effective FSCIL strategy for balancing old-knowledge preservation and new-knowledge adaptation.

  • Takeaways & Limitations

    The evaluation assumes disjoint class sets across tasks and access only to the current task's data during each session.

Abstract

from arXiv · show

As a challenging problem, few-shot class-incremental learning (FSCIL) continually learns a sequence of tasks, confronting the dilemma between slow forgetting of old knowledge and fast adaptation to new knowledge. In this paper, we concentrate on this "slow vs. fast" (SvF) dilemma to determine which knowledge components to be updated in a slow fashion or a fast fashion, and thereby balance old-knowledge preservation and new-knowledge adaptation. We propose a multi-grained SvF learning strategy to cope with the SvF dilemma from two different grains: intra-space (within the same feature space) and inter-space (between two different feature spaces). The proposed strategy designs a novel frequency-aware regularization to boost the intra-space SvF capability, and meanwhile develops a new feature space composition operation to enhance the inter-space SvF learning performance. With the multi-grained SvF learning strategy, our method outperforms the state-of-the-art approaches by a large margin.

1 INTRODUCTION

FSCIL must balance preserving old knowledge with adapting quickly to new few-shot classes. MgSvF addresses this slow-versus-fast dilemma through frequency-aware intra-space learning and inter-space feature composition.

  • 1 INTRODUCTION: FSCIL is difficult because slow forgetting can underfit new tasks, whereas fast adaptation can mix old-task representations and cause forgetting.The target is a trade-off that keeps both old and new task samples discriminative.
  • 1 INTRODUCTION: Lower-frequency components produce less forgetting than higher-frequency components across both two-session and ten-session CIFAR100 settings.The figure measures forgetting with average forgetting and shows forgetting increasing with frequency.
  • 1 INTRODUCTION: DCT creates an orthogonal frequency space whose components can be separately evaluated for their old-knowledge transfer behavior.The resulting frequency-aware learning strategy improves performance.
  • 1 INTRODUCTION: A separate slowly updated feature space is composed with the other space, and even uniform concatenation yields a dramatic performance improvement.This inter-space design complements the intra-space frequency strategy.
  • 1 INTRODUCTION: The proposed multi-grained SvF framework balances old-knowledge preservation and new-knowledge adaptation through intra-space and inter-space strategies.It uses frequency-aware regularization within one feature space and feature-space composition between separate spaces.
  • 1 INTRODUCTION: Across datasets, extensive experiments report that the method outperforms state-of-the-art approaches by a large margin.The introduction presents this as the overall empirical effectiveness claim.

2 RELATED WORK

Prior incremental-learning methods use architectural, rehearsal, or regularization strategies, while FSCIL focuses on learning new classes from few samples. MgSvF extends unified-space approaches by analyzing slow-versus-fast learning at intra-space and inter-space grains.

  • 2 RELATED WORK: Incremental-learning research is organized into architectural, rehearsal, and regularization strategies for retaining prior knowledge while learning new tasks.Architectural methods manipulate network structure, rehearsal methods retain examples, and regularization methods constrain updates.
  • 2 RELATED WORK: FSCIL is a challenging class-incremental setting because only a few samples are available for new tasks and task identity is unavailable at inference.The paper considers FSCIL within class-incremental learning rather than task-incremental learning.
  • 2 RELATED WORK: Existing approaches commonly use one unified feature space to balance old-task preservation and new-task adaptation, but MgSvF studies separate intra-space and inter-space grains.Its proposed strategies are frequency-aware regularization and feature-space composition.
  • 2 RELATED WORK: Frequency transformations have also been used for computational efficiency, augmentation, robustness, and domain adaptation, distinct from MgSvF's slow-versus-fast analysis.The paper positions its frequency-aware strategy alongside these broader frequency-aware applications.

3 METHODS

MgSvF addresses FSCIL by separating slow old-knowledge preservation from fast new-task adaptation at intra-space and inter-space levels. It uses frequency-aware regularization and composition of separately updated feature spaces for classification.

  • 3 METHODS: The method trains slow and fast feature spaces and composes them for classification, combining intra-space frequency regularization with inter-space feature composition.At the base session, an embedding model is trained on large-scale data; later sessions update two models differently before composing their spaces.
  • 3.1 Few-Shot Class-Incremental Learning: The embedding framework maps samples into a feature space where semantic discrepancy is represented by distance and uses a nearest class mean classifier.The embedding is denoted z_j, while each class prototype u_c is the mean embedding of that class.
  • 3.1 Few-Shot Class-Incremental Learning: FSCIL requires balancing old-knowledge slow-forgetting with fast adaptation to few-shot new classes while avoiding degraded discriminability and catastrophic forgetting.The setup provides many samples for the base task but only a few samples for later tasks, with non-overlapping class sets and access only to the current task’s data.
  • 3.2 Intra-Space Level SvF learning: The objective combines metric learning with regularization, using triplet-style distances to bring similar instances closer and separate dissimilar ones while retaining past knowledge.The regularization term is weighted against the metric-learning term by λ; d+ and d− measure distances to positive and negative instances, respectively.
  • 3.2 Intra-Space Level SvF learning: Intra-space SvF applies DCT to transform entangled embeddings into an orthogonal frequency space, where frequency components can receive different preservation strengths.The transformed feature retains the original length, and low-frequency components receive higher regularization weights because they contribute more to preserving old knowledge.
  • 3.3 Inter-Space Level SvF Learning: Inter-space SvF updates one feature space slowly for old-knowledge preservation and another quickly for new-knowledge adaptation, then composes them into a discriminative space.The composite representation can use simple concatenation, and a metric matrix can control the relative importance of slow- and fast-updated features.

4.1 Datasets

The experiments use CIFAR100, CUB200-2011, and MiniImageNet, covering object recognition, fine-grained bird classification, and few-shot image classification.

  • CIFAR100 contains 60,000 32 × 32 RGB images across 100 object-recognition classes.Each class has 500 training images and 100 test images.
  • CUB200-2011 contains 12,000 images across 200 fine-grained bird categories.The dataset has 6,000 training and 6,000 testing images at 256 × 256 resolution.
  • MiniImageNet contains 100 classes with 500 84 × 84 training images and 100 testing images per class.It is a subset of ImageNet-1k used for few-shot learning.

4.2 Implementation Details

The evaluation follows FSCIL protocols across three datasets, using few-shot new tasks, ResNet18 embeddings, and separate slow- and fast-updated models.

  • Evaluation Protocol: Experiments use 5-way 5-shot sessions for CIFAR100 and MiniImageNet and 10-way 5-shot sessions for CUB200.CIFAR100 and MiniImageNet have nine sessions, while CUB200 has ten new-learning sessions after its base task.
  • Evaluation Protocol: Each new session samples five training examples per class, while evaluation uses the original test set across all classes seen so far.Models are evaluated after training on each new batch of classes.
  • Implementation: The implementation uses a ResNet18 backbone, a 512-dimensional embedding network, and Adam optimization under the comparison strategy of prior work.The base model follows the same training strategy as the reference method for fair comparison.
  • Implementation: Slow-updated models use γ_q = 1 only for q = 1, whereas fast-updated models use γ_q = 1 for q ≠ 1.Old-class centers are computed after each task and fixed in subsequent tasks.

4.3 Comparison to State-of-the-Art Methods

The proposed method is compared with established FSCIL baselines across CUB200, CIFAR100, and MiniImageNet, and outperforms the alternatives throughout the reported learning sessions.

  • The comparison includes iCaRL, EEIL, LUCIR, TOPIC, SDC, and POD, with iCaRL and LUCIR also evaluated using NCM classifiers.The paper distinguishes CNN- and NCM-based variants for those two methods.
  • The method outperforms all other methods at every encountered learning session on CIFAR100 and MiniImageNet.Figure 5 reports comparisons under the 5-way 5-shot FSCIL setting with ResNet18.

4.4 Ablation Study

Ablations show that both intra-space and inter-space SvF strategies contribute to performance, while their complementary slow- and fast-updated spaces address different knowledge-transfer needs.

  • Effect of Intra-space and Inter-space SvF Strategies: Both inter-space and intra-space SvF strategies improve the baseline, with inter-space exceeding intra-space by around 1% on CUB200.On CIFAR100 and MiniImageNet, intra-space performance is comparable to inter-space performance.
  • Effect of Intra-space and Inter-space SvF Strategies: The fast-updated space is important for fitting new knowledge because it substantially outperforms the slow-updated space on current-task data.Figure 6 reports a large current-task performance gap between the two spaces.
  • Composition Operation: The sophisticated PCA-based composition operation outperforms the simple composition version by around 1%.The comparison covers CUB200, CIFAR100, and MiniImageNet.
  • Motivation: A unified feature space makes it difficult to balance slow forgetting of old knowledge with fast adaptation to new knowledge.The paper identifies these updating requirements as difficult to satisfy through one shared space.
  • Intra-space SvF Analysis: Low-frequency components are more conducive to preserving old knowledge, so the regularization assigns them higher weights.Across the analyzed groups, forgetting increases with frequency; results are shown on CUB200 and MiniImageNet.

4.4.4 Inter-Space SvF analysis

The inter-space analysis shows that combining fast- and slow-updated feature spaces improves over using either space alone, while PCA-based composition yields a further accuracy gain.

  • Inter-space complementarity: The composite feature space outperforms either the slow-updated or fast-updated space alone, demonstrating their complementarity.Slow-only features contain limited new-task knowledge, whereas fast-only features suffer from forgetting old knowledge.
  • Inter-space complementarity: Performance varies with the composition coefficient a: intermediate values outperform using only the slow-updated or only the fast-updated space.Here, a = 0 uses only the slow-updated space and a = 1 uses only the fast-updated space.
  • Inter-space composition operations: The simple composition concatenates separate fast-updated and slow-updated spaces, while PCA composition reduces each space before concatenation.The two strategies are denoted “inter-space-simple” and “inter-space-pca,” respectively.
  • Inter-space composition operations: Baseline+inter-space-pca achieves 1% higher accuracy than the baseline, using PCA reduction before concatenating the two feature spaces.The method sets P1 = P2 because few new-task samples cannot estimate a reasonable P2.

4.4.6 The Effect of The Number of Training Samples

Across 5-, 10-, 20-, and 50-shot settings, performance improves with more training samples, with sample count mattering more in later sessions. The method is stable across frequency-group counts and performs best with eight groups.

  • Average forgetting: The method outperforms the baseline by more than 5% in average forgetting at the last learning session on CUB200, CIFAR100, and MiniImageNet.Average forgetting is evaluated across 11 sessions on CUB200 and 9 sessions on CIFAR100 and MiniImageNet.
  • Hyperparameter choices: The experiments use higher slow-model weights on low-frequency groups and a fast-model learning rate typically 10 times larger than the slow-model rate.The reported learning rates are 1e-5 for the fast-updated model and 1e-6 for the slow-updated model.
  • Frequency-group analysis: Average accuracy across all learning sessions is 52.76%, 53.03%, 53.24%, and 52.78% for NQ = 2, 4, 8, and 16, respectively.The method is stable as the number of frequency groups changes and achieves its best result at NQ = 8.

5 CONCLUSION

The paper concludes with a multi-grained SvF pipeline for FSCIL that combines intra-space frequency-aware regularization with inter-space feature composition. Comprehensive experiments show significant improvement over state-of-the-art approaches.

  • 5 CONCLUSION: The proposed pipeline combines intra-space frequency-aware regularization with inter-space composition to balance slow forgetting and fast adaptation.The two mechanisms enforce SvF behavior across frequency components and between separate feature spaces.
  • 5 CONCLUSION: Comprehensive experiments demonstrate that the approach significantly outperforms other state-of-the-art approaches by a large margin.
Loading 2006.15524v4…