PaperlayerView exact revision on arXiv

Source-linked AI summary

Few-Shot Lifelong Learning

Pratik Mazumder, Pravendra Singh, Piyush Rai

arXiv:2103.00991v1cs.CV

TL;DR

Few-shot class-incremental learning requires models to learn classes with very few labels as new class sets arrive, while limiting overfitting and catastrophic forgetting. FSLL trains only a few currently unimportant parameters for each new class set and reports significant gains over existing methods, with further improvement from self-supervision.

  • Problem

    Few-shot class-incremental learning must handle very few examples per class and classes that become available incrementally, while addressing overfitting and catastrophic forgetting.

  • Method

    FSLL selects very few currently unimportant parameters as session-trainable parameters for each new few-shot class set instead of training the full model.

  • Results

    FSLL significantly outperforms state-of-the-art methods and baselines across the compared datasets, including a 19.27% absolute margin over TOPIC on CUB-200.

  • Takeaways & Limitations

    Using self-supervision as an auxiliary task can further improve performance in the few-shot class-incremental setting.

  • Takeaways & Limitations

    The setting assumes no examples from previously seen classes are stored and provides no task information at test time.

Abstract

from arXiv · show

Many real-world classification problems often have classes with very few labeled training samples. Moreover, all possible classes may not be initially available for training, and may be given incrementally. Deep learning models need to deal with this two-fold problem in order to perform well in real-life situations. In this paper, we propose a novel Few-Shot Lifelong Learning (FSLL) method that enables deep learning models to perform lifelong/continual learning on few-shot data. Our method selects very few parameters from the model for training every new set of classes instead of training the full model. This helps in preventing overfitting. We choose the few parameters from the model in such a way that only the currently unimportant parameters get selected. By keeping the important parameters in the model intact, our approach minimizes catastrophic forgetting. Furthermore, we minimize the cosine similarity between the new and the old class prototypes in order to maximize their separation, thereby improving the classification performance. We also show that integrating our method with self-supervision improves the model performance significantly. We experimentally show that our method significantly outperforms existing methods on the miniImageNet, CIFAR-100, and CUB-200 datasets. Specifically, we outperform the state-of-the-art method by an absolute margin of 19.27% for the CUB dataset.

1 Introduction

Few-shot class-incremental learning must handle scarce labels and classes arriving over time, while avoiding overfitting and catastrophic forgetting. FSLL addresses these challenges by training only a few unimportant parameters, with self-supervision providing an additional performance gain.

  • Problem: Few-shot class-incremental learning combines scarce per-class labels with incrementally arriving, disjoint class sets.The model must ultimately classify jointly across all encountered classes.
  • Problem: Training the entire network on few-shot classes can cause overfitting, while unavailable old-class data can cause catastrophic forgetting.The two problems must be addressed simultaneously in the FSCIL setting.
  • Method: FSLL trains very few session parameters selected from currently unimportant model parameters instead of updating the entire network.Keeping important parameters intact is intended to preserve old knowledge while reducing overfitting.
  • Enhancement: Self-supervision used as an auxiliary task can further improve model performance in the few-shot class-incremental setting.The paper reports this as an empirical contribution alongside the parameter-selection method.
  • Results: FSLL significantly outperforms baselines and state-of-the-art methods across the compared datasets.Experiments cover miniImageNet, CIFAR-100, and CUB-200, with additional ablation experiments validating method components.

2 Proposed Method

FSLL incrementally learns few-shot classes by training only selected unimportant parameters while preserving important parameters, combining metric, regularization, and prototype-separation objectives. The method also evaluates rotation-based self-supervision as an auxiliary task.

  • Problem Setting: The model is first trained on base classes, then incrementally evaluated on all encountered classes as new few-shot class sets become available.Base training uses many examples per class; subsequent sessions provide only the current few-shot set, with prior data inaccessible.
  • Parameter Selection: Training only selected parameters addresses few-shot overfitting while preserving important parameters to reduce catastrophic forgetting.The method targets both challenges caused by incremental few-shot training: overfitting to scarce examples and loss of knowledge from older classes.
  • Parameter Selection: FSLL selects very few low-magnitude feature-extractor parameters for each new session and freezes the remaining knowledge-retention parameters.Layer-specific thresholds identify parameters with low absolute weights as session-trainable, while the remaining parameters stay frozen.
  • Session Training: The session-trainable parameters are optimized with triplet loss, which pulls same-class examples together and pushes different-class examples apart.The loss uses examples xi, xj, xk with yi = yj and yi ≠ yk, and d denotes Euclidean distance.
  • Session Training: An ℓ1 regularization loss keeps current session-trainable weights near their previous values, while cosine distance separates new and previous class prototypes.The total loss combines triplet, cosine, and regularization terms; λ controls the regularization contribution.
  • Prototype Classification: After each session, class prototypes are computed from feature means and nearest-prototype classification uses prototypes from all encountered classes.The classifier is discarded after base training, and current-session features are used to update the prototype-based prediction procedure.
  • Self-Supervised Auxiliary Task: Rotation-prediction self-supervision improves FSLL performance and outperforms SimCLR and patch-location prediction as an auxiliary task in the reported ablations.The auxiliary network predicts one of four rotation angles from features extracted by the feature extractor.

3 Related Work

Related work distinguishes class-incremental learning, which requires joint classification across encountered classes, from task-incremental learning, which maintains separate classifiers. The paper focuses on the more challenging few-shot class-incremental setting and contrasts it with prior replay, regularization, dynamic-network, and few-shot approaches.

  • Class-Incremental Learning: Class-incremental learning trains on disjoint class sets sequentially and tests jointly across all encountered classes.The paper focuses on this setting in a few-shot regime, where classes have very few training examples.
  • Task-Incremental Learning: Task-incremental learning maintains a separate classifier for each disjoint task, reducing the test-time search space relative to class-incremental learning.Task-incremental methods include regularization-based, replay-based, and dynamic-network-based approaches.
  • Continual-Learning Methods: Regularization-based methods preserve prior knowledge by limiting changes to outputs or important parameters, while EWC reduces learning rates for parameters important to older tasks.Knowledge distillation is another regularization-based strategy described in the related work.
  • Continual-Learning Methods: Replay-based methods store old-task exemplars or generate substitute data and include them during training on new tasks to reduce catastrophic forgetting.These approaches depend on replayed or generated information from earlier tasks.
  • Continual-Learning Methods: Dynamic-network methods modify network structure through expansion, pruning, or parameter masking to prevent catastrophic forgetting.PackNet is cited as using pruning to free parameters for new tasks.
  • Scope of Comparison: Task-incremental methods are excluded from the paper’s comparisons because the paper addresses the class-incremental setting.The two settings differ in whether task identity and separate task classifiers are available at test time.
  • Few-Shot Learning: Standard few-shot learning commonly evaluates episodes restricted to a few classes, whereas few-shot class-incremental learning evaluates across all encountered classes.The paper characterizes the latter as more realistic and challenging and compares against TOPIC in that setting.
  • Self-Supervised Learning: Self-supervised methods learn from labels extracted from the data itself, with examples including rotation prediction, contrastive multiview coding, MoCo, and SimCLR.The cited methods differ in their auxiliary task or architectural requirements, such as specialized encoders or a memory bank.

4 Experiment

Experiments evaluate FSLL with ResNet-18 across miniImageNet, CIFAR-100, and CUB-200 FSCIL settings against established baselines. Results show strong performance, including large margins over TOPIC and best CUB-200 performance with 10% session-trainable parameters.

  • Experimental setup: Experiments use ResNet-18 on CIFAR-100, miniImageNet, and CUB-200 under FSCIL settings.CIFAR-100 and miniImageNet use 5-way 5-shot sessions, while CUB-200 uses 10-way 5-shot sessions.
  • Experimental setup: The evaluation reports average test accuracy over all encountered classes across repeated runs.Experiments run 10 times, use standard augmentation, and report low average standard deviations of around 0.5%.
  • CUB-200 results: 19.27% is FSLL’s absolute margin over TOPIC on CUB-200, while FSLL also outperforms Ft-CNN and Joint-CNN.The Joint-CNN comparison is attributed to overfitting when CUB-200 contains 100 few-shot classes.
  • miniImageNet results: 15.07% is FSLL’s approximate absolute margin over TOPIC on miniImageNet, and FSLL outperforms Ft-CNN and slightly exceeds Joint-CNN.The reported FSLL* variant further improves performance through weight-decay tuning.
  • CIFAR-100 results: 9.09% is FSLL’s absolute margin over TOPIC on CIFAR-100.An FSLL+SS experiment additionally trains the network on an auxiliary self-supervised rotation-prediction task during base training.

5 Ablation Experiments

Ablations show that FSLL is most effective when it balances knowledge retention with enough trainable parameters, uses regularization, and selects an appropriate auxiliary self-supervised task. The prototype cosine similarity loss and rotation-based self-supervision improve performance, while applying self-supervision to few-shot sessions does not help.

  • Proportion of Session Trainable Parameters: 10% of ΘF as session trainable parameters gives FSLL’s best CUB-200 performance; smaller proportions underfit, while larger proportions include important parameters and hurt performance.The ablation evaluates final accuracy at session 11.
  • Regularization: Regularization improves performance even when session trainable parameters comprise approximately 10% of ΘF and becomes especially important when that proportion is high.With more trainable parameters, fewer knowledge-retaining parameters remain.
  • Regularization: λ = 5 produces the best session 11 performance on CUB-200 and is used for all experiments.The result is reported for the 10-way 5-shot FSCIL setting with ResNet-18.
  • Prototype Cosine Similarity Loss: Removing the prototype cosine similarity loss reduces CUB-200 session 11 performance from 45.55% to 44.32%.This ablation supports the contribution of prototype separation to classification performance.
  • Choice of Self-Supervised Auxiliary Task: Rotation-based self-supervision performs significantly better than relative patch location prediction and SimCLR among the evaluated auxiliary tasks.The comparison uses FSLL* on CUB-200 in the 10-way 5-shot FSCIL setting.
  • Choice of Self-Supervised Auxiliary Task: Adding self-supervision to few-shot training sets produces 54.34% session 11 accuracy, below the 55.82% achieved by FSLL*.The experiments therefore report no benefit from applying the auxiliary task to D(t>1).

6 Conclusion

The paper proposes FSLL for few-shot class-incremental learning by training very few unimportant parameters on each new class set. It reports that FSLL outperforms the state-of-the-art method, and that self-supervision can further improve performance.

  • 6 Conclusion: FSLL selects very few unimportant parameters as session trainable parameters for each new few-shot class set.The method addresses overfitting and catastrophic forgetting in few-shot class-incremental learning.
  • 6 Conclusion: FSLL significantly outperforms the state-of-the-art method in the few-shot class-incremental learning setting.The conclusion also reports an additional benefit from self-supervision.
  • 6 Conclusion: Self-supervision as an auxiliary task can further improve model performance in few-shot class-incremental learning.
Loading 2103.00991v1…