Source-linked AI summary
Few-Shot Class-Incremental Learning via Training-Free Prototype Calibration
Qi-Wei Wang, Da-Wei Zhou, Yi-Kai Zhang, De-Chuan Zhan, Han-Jia Ye
TL;DR
FSCIL requires learning new classes from few labeled examples while retaining base-class knowledge, yet existing methods often misclassify new-class samples into base classes. TEEN calibrates new prototypes using weighted base prototypes and semantic similarity from a base-trained feature extractor. It improves new-class accuracy over the runner-up FSCIL method and reports competitive results in FSCIL and few-shot learning; its evaluation assumes base and new classes come from the same dataset.
Problem
Existing methods can perform well on average accuracy across all classes yet poorly on new classes, whose samples are prone to be predicted as base classes.
Source: §1 · p. 2Method
TEEN fuses biased new-class prototypes with class-specific, semantic-aware weighted base prototypes, using a feature extractor trained on base classes without extra optimization or parameters.
Source: §1 · p. 3, §5.1 · p. 7Results
New-class accuracy is 10.02% ∼18.40% better than the runner-up FSCIL method, with reported improvements on standard FSCIL benchmarks and competitive few-shot results.
Source: §1 · p. 3Takeaways & Limitations
TEEN is a plug-and-play calibration module with no additional optimization or model parameters once the feature extractor is trained.
Source: §1 · p. 3The evaluated FSCIL setting assumes base and new classes come from the same dataset, leaving cross-domain few-shot class-incremental learning for future work.
Source: §7 · p. 10
Abstract
from arXiv · showhide
Real-world scenarios are usually accompanied by continuously appearing classes with scare labeled samples, which require the machine learning model to incrementally learn new classes and maintain the knowledge of base classes. In this Few-Shot Class-Incremental Learning (FSCIL) scenario, existing methods either introduce extra learnable components or rely on a frozen feature extractor to mitigate catastrophic forgetting and overfitting problems. However, we find a tendency for existing methods to misclassify the samples of new classes into base classes, which leads to the poor performance of new classes. In other words, the strong discriminability of base classes distracts the classification of new classes. To figure out this intriguing phenomenon, we observe that although the feature extractor is only trained on base classes, it can surprisingly represent the semantic similarity between the base and unseen new classes. Building upon these analyses, we propose a simple yet effective Training-frEE calibratioN (TEEN) strategy to enhance the discriminability of new classes by fusing the new prototypes (i.e., mean features of a class) with weighted base prototypes. In addition to standard benchmarks in FSCIL, TEEN demonstrates remarkable performance and consistent improvements over baseline methods in the few-shot learning scenario. Code is available at: https://github.com/wangkiw/TEEN
1 Introduction
FSCIL must learn new classes from few examples while retaining base-class knowledge, but existing methods often misclassify new-class samples as similar base classes. TEEN uses semantic similarity encoded by a base-trained feature extractor to calibrate new-class prototypes without additional training.
1 Introduction
- CIL addresses continual data arrival, where models must learn new knowledge while maintaining old-task ability; FSCIL adds few labeled examples and risks overfitting.Prototype-based methods freeze the base-trained feature extractor and use new-class prototypes, but scarce examples can bias those prototypes.Source: §1 · pp. 1–2
- Existing methods can perform well on average accuracy across all classes while performing poorly on new classes, whose performance can be obscured by numerous base classes.The paper notes that more recent tasks may be especially important in some real-world applications.Source: §1 · p. 2
- Existing FSCIL methods often predict new-class instances as base classes, while a feature extractor trained only on base classes can represent semantic similarity between base and new classes.The paper identifies similar base classes as common destinations for misclassified new-class samples.Source: §1 · p. 2
- TEEN calibrates biased new-class prototypes by fusing them with class-specific, semantic-aware weighted base prototypes, without extra optimization or model parameters.It is designed as a plug-and-play module once the feature extractor has been trained on base classes.Source: §1 · p. 3
- New-class accuracy is 10.02% ∼18.40% better than the runner-up FSCIL method, and TEEN also achieves higher average accuracy.The paper reports validation on standard FSCIL benchmarks and competitive performance in few-shot learning.Source: §1 · p. 3
2 Related Works
The related-work review covers approaches to continual learning, few-shot learning, and their combination. It positions TEEN against FSCIL methods that overlook base-class semantic information and require optimization.
2.1 Class Incremental Learning
2.2 Few-Shot Learning
- Few-shot learning methods use optimization for fast adaptation or metric learning for semantic similarity, while [51] models class features with Gaussians and trains logistic regression on augmented features.The paper reports that TEEN outperforms [51] in most settings without training cost when recognizing new classes.Source: §2.2 · p. 3
2.3 Few-Shot Class-Incremental Learning
- FSCIL methods use trainable feature-extractor or representation-adaptation modules, but overlook base-class semantic information; unlike [1], TEEN uses that information without optimization.Other reviewed approaches include TOPIC [42], word embeddings [8], and quantized reference vectors [6].Source: §2.3 · pp. 3–4
3 Preliminaries
FSCIL proceeds through a base session and incremental sessions with limited labeled data, while retaining knowledge of previously seen classes. The preliminaries define prototypes as mean features used as classifier weights, with a frozen base-trained encoder for new classes.
3.1 Definition and notations
- FSCIL has a base session followed by incremental sessions, and each session’s test label space includes all classes seen so far.An incremental session is an N-way K-shot task; only the current session’s training data are available.Source: §3.1 · p. 4
- Unlike conventional CIL, FSCIL learns new classes from limited labels; unlike conventional few-shot learning, it must also retain previously seen knowledge.Source: §3.1 · p. 4
3.2 Prototypical Network
- A class prototype is its mean feature and serves as a classifier weight; for new classes, the feature extractor trained on base classes is frozen.The frozen encoder can alleviate catastrophic forgetting, while prototype-based classifier updates can relatively circumvent overfitting.Source: §3.2 · p. 4
4 A Closer Look at FSCIL
This section examines why existing FSCIL methods underperform on new classes and finds that their samples are often confused with similar base classes. It connects that error pattern to calibrating biased new-class prototypes with base prototypes.
4.1 Understanding the reason for poor performance in new classes
- On CIFAR100 session 1, FPR is exceedingly high and FNR relatively low, indicating new-class samples are readily misclassified as base classes.Table 1 reports the false positive and false negative rates for the session in which new classes first appear.Source: Table 1 · p. 5, §4.1 · p. 5
- Table 2 reports relatively lower TBR for TEEN than for baseline methods on CIFAR100 session 1.For new classes, the table's TBR analysis considers the 10 most similar base classes out of 60.Source: Table 2 · p. 5
- Base-class accuracy is much higher than new-class accuracy; the authors attribute the gap to a frozen extractor and biased new prototypes.They describe the extractor as forcing overfit to base classes and the biased prototypes as causing underfit to new classes.Source: §4.1 · p. 5
- Many misclassified new-class samples are closer to their nearest base prototype than to their target prototype, with TBR high and TNR relatively low.Table 2 analyzes TBR and TNR on CIFAR100 session 1, considering the 10 most similar base classes for new-class errors.Source: §4.1 · pp. 5–6, Table 2 · p. 5
- The analysis separates base classes as positive and new classes as negative, then evaluates false negative and false positive rates for each incremental session.The section uses this binary view to inspect misclassification patterns between the two class groups.Source: §4.1 · p. 5
- TEEN uses the frozen feature extractor as a bridge, calibrating biased new prototypes by fusing them with well-calibrated base prototypes.The proposed calibration draws on the extractor's ability to represent semantic similarity between base and new classes.Source: §4.1 · p. 6
5 Similarity-based Prototype Calibration
TEEN uses semantic similarity between well-calibrated base prototypes and biased new prototypes to calibrate new classes without training. The analysis examines the calibration mechanism and its effects on predictions.
5.1 Fusing the biased prototypes with calibration item
- The method targets biased prototypes from incremental sessions, which are considered unreliable because those sessions provide limited data.Base prototypes are treated as approximately consistent with expected class representations because base training has sufficient classes and samples.Source: §5.1 · p. 7
- TEEN calibrates new-class prototypes by blending each biased prototype with a similarity-weighted combination of well-calibrated base prototypes.Cosine similarity determines softmax weights, and the procedure introduces no learnable components or training parameters.Source: §5.1 · p. 7
5.2 Effect of calibrated prototypes
- TEEN aligns calibrated new prototypes toward their most similar base prototypes, with the similarity-weight sharpness controlled by τ.Larger τ concentrates the weight on the most similar base prototypes.Source: §5.2 · p. 7
- W→R prediction changes are mainly new-class samples, while R→W changes are mainly base-class samples; the reported negative effect is negligible.The comparison is shown for miniImageNet per-session accuracy and across benchmark results.Source: §5.2 · p. 7, Table 3 · p. 8, Figure 4 · p. 8
6 Experiments
The experiments evaluate TEEN on FSCIL benchmarks, few-shot learning, and ablations. Across several measures, the reported comparisons support its effectiveness, while the ablations examine calibration settings and similarity weighting.
6.1.1 Implementation Details
- The evaluation uses CIFAR100, CUB200-2011, and miniImageNet with dataset splits consistent with prior work and shared backbones for fair comparisons.The reported implementation uses ResNet20 for CIFAR100, pre-trained ResNet18 for CUB200, and randomly initialized ResNet18 for miniImageNet.Source: §6.1.1 · p. 8
6.1.2 Comparison results
- TEEN outperforms previous state-of-the-art methods across the reported performance measures, and its PD and ΔPD comparisons are unaffected by pretrained results.The paper reports lower FPR and TBR as further evidence of effective calibration.Source: §6.1.2 · p. 9
6.2 Comparison results of FSL
- TEEN outperforms the previous state-of-the-art method in several few-shot settings and adds no training cost when recognizing new classes.Once sample features are extracted, the paper considers TEEN’s inference cost negligible compared with methods requiring heavy training.Source: §6.2 · p. 9, Table 5 · p. 9
6.3 Ablation Study
- A proper α can improve new-class performance, while relatively smaller τ smooths the weight distribution and is associated with stronger performance.The study varies τ over {8, 16, 32, 64} and α over {0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}.Source: §6.3 · p. 10
- Removing similarity-based weights causes a significant performance drop, particularly on new classes.Figure 5 compares AvgAcc and NAcc for the similarity-weighted method and its simplified alternatives.Source: §6.3 · p. 10, Figure 5 · p. 10
7 Conclusion
Existing FSCIL methods often misclassify new-class samples into similar base classes, although a base-trained feature extractor can represent semantic similarity between base and new classes. TEEN calibrates new-class prototypes and obtains competitive results in FSCIL and FSL.
7 Conclusion
- TEEN is a training-free prototype-calibration strategy that combines the paper’s findings about new-class errors and base-to-new semantic similarity.The paper reports competitive results in FSCIL and FSL.Source: §7 · p. 10
- The paper identifies same-dataset selection of base and new classes as a limitation of current FSCIL methods.It leaves cross-domain FSCIL for future work.Source: §7 · p. 10
Supplementary materials
The supplementary analysis reports that off-the-shelf features represent semantic similarity without extra training or auxiliary side information. It also describes TEEN’s incremental cost and its compatibility with stronger pretrained representations.
A.1 Implicit Semantic Similarity
- The off-the-shelf feature extractor represents semantic similarity without extra training costs or auxiliary side information such as class-name text.The paper gives textual information of the class name [8] as an example of auxiliary side information.Source: §A.1 · p. 13
A.2 Efficiency
- TEEN’s incremental calibration adds no training cost or update procedure, making its incremental learning cost negligible compared with existing methods.The paper identifies base-class pretraining as TEEN’s only time cost.Source: §A.2 · p. 13
A.3 Combination with better representation
- TEEN can be combined with prototype-based methods and more powerful pretrained models, including self-supervised models [7] and vision-language models such as CLIP [34].The paper describes vision-language models as providing better representation.Source: §A.3 · p. 13
B A Closer Look at FSCIL
The additional benchmark analysis finds that prototype-based FSCIL methods often misclassify new-class samples into base classes, particularly into their most similar base classes. The supplied table captions identify per-session accuracy and HMean/NAcc comparisons.
B A Closer Look at FSCIL
- Additional benchmark observations confirm that prototype-based FSCIL methods are prone to misclassifying new-class samples as base-class samples.The paper says these observations are consistent with those in the main paper and further confirm their universality.Source: B A Closer Look at FSCIL · p. 13
- New classes are often incorrectly predicted as their corresponding most similar base classes.The paper says Table 11 further verifies this observation.Source: B A Closer Look at FSCIL · p. 13
- Tables 6 and 7 report per-session average accuracy for CIFAR100 and CUB200, while Table 8 reports HMean and NAcc for miniImageNet.Table 8 presents only six incremental sessions in the supplied caption.Source: Table 6 · p. 13, Table 7 · p. 14, Table 8 · p. 14
C.1 Details of experiments
The supplementary materials describe benchmark splits, baselines, and training settings, then report consistent performance improvements and further evidence that TEEN calibrates biased new-class prototypes. They also note that some base-class accuracy can be lost while overall accuracy remains competitive.
C.1 Details of experiments
- The evaluation uses CIFAR100, CUB200-2011, and miniImageNet, with each dataset split into base classes and few-shot incremental tasks.The specified splits include 60 base classes and eight 5-way 5-shot tasks for CIFAR100 and miniImageNet, and 100 base classes and ten 10-way 5-shot tasks for CUB200.Source: §C.1 · p. 14
- Comparisons include CIL, few-shot, and FSCIL baselines, with the compared methods using the same backbone and initialization.The listed FSCIL baselines are TOPIC [42], CEC [57], and FACT [58].Source: §C.1 · p. 14
- The reported training setup uses cosine similarity for instance-to-prototype comparisons and dataset-specific hyperparameters and training schedules.The supplied settings include α = 0.5 for miniImageNet and CUB200, α = 0.1 for CIFAR100, and τ = 16 for all three.Source: §C.1 · pp. 14–15
C.2 More comparison results
- TEEN shows consistent improvement across the reported performance measures, including performance drop rate, average accuracy, harmonic mean accuracy, and new-class accuracy.The supplementary comparison reports detailed results for CIFAR100 and CUB200.Source: §C.2 · p. 15
D.1 Effects of TEEN
- TEEN calibrates biased new-class prototypes, and its average accuracy across all classes remains competitive despite some base-class misclassification.Table 9 reports detailed ratios of base and new classes among UC, W→R, and R→W samples.Source: §D.1 · p. 15, Table 9 · p. 15
D.2 Combination of TEEN with existing methods
TEEN can be added to existing prototype-based methods as a plug-and-play calibration module. With FACT, it improves new-class and harmonic-mean accuracy while having little effect on average accuracy across all classes.
D.2 Combination of TEEN with existing methods
- TEEN is presented as a plug-and-play module for calibrating biased new-class prototypes in existing prototype-based methods.The authors suggest other methods, including, may also benefit from its flexibility.Source: §D.2 · p. 15
- When combined with FACT, TEEN has little effect on average accuracy across all classes, but improves average accuracy on new classes and harmonic-mean accuracy, balancing performance between base and new classes.Source: §D.2 · p. 15
- Combining TEEN with FACT has little effect on average accuracy across all classes.The authors characterize the resulting performance as better balanced between base and new classes.Source: §D.2 · p. 15
E Broader impact
The paper reports improved new-class accuracy in FSCIL and few-shot learning, and highlights the importance of evaluating new-class performance. Its FACT comparisons and miniImageNet prediction analyses also report better balance between base and new classes and lower error rates with TEEN.
E Broader impact
- TEEN significantly improves new-class accuracy in FSCIL and few-shot learning, while the analyses call for greater attention to new-class performance.The authors also suggest the method may have heuristic effects on expectation estimation of the distribution.Source: E Broader impact · p. 15
- With FACT, TEEN yields better-balanced performance between base and new classes on CUB200, miniImageNet, and CIFAR100.The comparison reports little effect on average accuracy across all classes, alongside improvements in new-class and harmonic-mean accuracy.Source: Figure 6 · p. 16, Figure 7 · p. 16, Figure 8 · p. 16
- On miniImageNet session 1, TEEN achieves relatively lower FPR and TBR than baseline methods.The TBR analysis considers the most similar classes, using cosine similarity between class prototypes.Source: Table 10 · p. 16, Table 11 · p. 17