Source-linked AI summary
FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learning
Grégoire Petit, Adrian Popescu, Hugo Schindler, David Picard, Bertrand Delezoide
TL;DR
Exemplar-free class-incremental learning must retain past knowledge without replaying unavailable or costly-to-store images. FeTrIL freezes the initial feature extractor, translates new-class features into pseudo-features for past classes, and incrementally trains a linear classifier; it outperforms ten existing methods in most tested cases. The method also supports frequent one-class updates with accuracy close to larger incremental steps.
Problem
Exemplar-free class-incremental learning must balance plasticity for new classes with stability for past classes despite unavailable past images and catastrophic forgetting.
Method
FeTrIL freezes the feature extractor, generates past-class pseudo-features by geometric translation of new-class features using stored centroids, and trains a linear classifier incrementally.
Results
FeTrIL outperforms ten existing methods in most cases and supports one-class increments with accuracy close to that observed for T = 20.
Takeaways & Limitations
FeTrIL offers a simple, fast, memory-efficient exemplar-free approach whose performance remains close to exemplar-based methods in many cases.
Takeaways & Limitations
FeTrIL freezes a feature extractor learned on the initial state and tends to favor past classes over new ones.
Abstract
from arXiv · showhide
Exemplar-free class-incremental learning is very challenging due to the negative effect of catastrophic forgetting. A balance between stability and plasticity of the incremental process is needed in order to obtain good accuracy for past as well as new classes. Existing exemplar-free class-incremental methods focus either on successive fine tuning of the model, thus favoring plasticity, or on using a feature extractor fixed after the initial incremental state, thus favoring stability. We introduce a method which combines a fixed feature extractor and a pseudo-features generator to improve the stability-plasticity balance. The generator uses a simple yet effective geometric translation of new class features to create representations of past classes, made of pseudo-features. The translation of features only requires the storage of the centroid representations of past classes to produce their pseudo-features. Actual features of new classes and pseudo-features of past classes are fed into a linear classifier which is trained incrementally to discriminate between all classes. The incremental process is much faster with the proposed method compared to mainstream ones which update the entire deep model. Experiments are performed with three challenging datasets, and different incremental settings. A comparison with ten existing methods shows that our method outperforms the others in most cases.
1. Introduction
Class-incremental learning must preserve past knowledge while integrating sequentially arriving classes, but exemplar-free settings cannot replay stored images. FeTrIL addresses this tension by combining a frozen feature extractor with geometrically translated pseudo-features and performs strongly against existing methods.
- Motivation: Sequentially arriving data and costly retraining motivate class-incremental learning, whose central challenge is catastrophic forgetting without stored past examples.Exemplar-free methods avoid replay because past data may be unavailable and image storage can burden constrained devices.
- Proposed approach: FeTrIL combines a frozen feature extractor with a pseudo-feature generator to improve the stability-plasticity balance in exemplar-free learning.New-class features are translated toward past-class regions using stored class centroids.
- Proposed approach: Geometric translation creates usable pseudo-features of past classes from new-class features and differences between target and new-class centroids.The procedure is illustrated in a 2D toy projection with an initial state and two incremental states.
- Results: Experiments in a standard setting show better behavior than ten existing methods, including recent approaches.The setting uses a larger initial state followed by smaller incremental states containing equal numbers of classes.
2. Related Work
Prior exemplar-free class-incremental methods commonly update the deep model with regularization or distillation, while transfer-learning approaches freeze the feature extractor. FeTrIL differs by using fixed representations with geometric feature translation rather than replay, covariance storage, or continual backbone fine-tuning.
- Exemplar-free CIL: Replay-based methods mitigate forgetting but require storing past samples, whereas exemplar-free learning imposes stronger memory and computational constraints.Exemplar-free methods commonly use distillation while updating the deep model at each incremental step.
- Distillation-based methods: Distillation-based methods preserve past knowledge during model updates but require storing the current and preceding models for incremental updates.Feature distillation additionally seeks to preserve the geometry of past classes and improve inter-class separation.
- Prototype and feature-generation methods: Semantic drift across incremental states motivates prototype-based and feature-generation methods such as PASS, IL2A, and SSRE.IL2A uses class distributions but requires storing a covariance matrix for each class, limiting scalability.
- FeTrIL’s distinction: FeTrIL shares prototype usage with recent methods but freezes the model after the initial state and uses geometric translation in the fixed representation space.Only the final linear layer is trained, unlike methods that fine-tune the backbone.
- Transfer-learning approaches: Transfer-learning approaches also freeze an initially trained feature extractor and update an external classification layer to integrate new classes.Their stated advantage is simplicity, although the supplied passage truncates the remaining comparison.
- Related prototype models: A few-shot prototype method predicts class means and variances with supplementary neural networks, but its extra parameters are disadvantageous in computationally constrained CIL settings.The paper distinguishes this setting from CIL, where past-class means are considered well placed in representation space.
3. Proposed Method
FeTrIL addresses exemplar-free class-incremental learning with a frozen feature extractor, centroid-guided geometric translation, and an incremental linear classifier. It stores class prototypes rather than past images and can generate past-class representations from new-class features.
- Problem setting: Exemplar-free CIL stores no past images, requires recognition of new and past classes together, and has unknown state boundaries at test time.The process consists of an initial state followed by incremental states with disjoint class sets.
- Overview: FeTrIL freezes feature extractor F after the initial state, preserving a stable representation space throughout incremental learning.The generator then uses new-class features and stored past-class prototypes to produce pseudo-features.
- Pseudo-feature generation: The pseudo-feature generator offers a simpler alternative to GANs, image inversion, and covariance-based past-class models.It produces effective past-class representations through geometric translation.
- Pseudo-feature generation: For target past class Cp and new class Cn, pseudo-feature ˆf t(cp) translates f(cn) by the centroid difference µ(Cp) − µ(Cn).The centroids are computed when classes first appear and stored because the frozen extractor keeps their features fixed.
- Feature selection: When a state contains multiple new classes, FeTrIL can transfer features from the kth most similar new class using cosine similarity between class centroids.Different k values test whether class similarity affects performance; other strategies are compared in the method.
- Linear classification layer training: The method trains a linear classifier for all P + N seen classes using pseudo-features for past classes and actual features for new classes.Weights are defined for every known class in the current incremental state.
- Linear classification layer training: Two implementations use LinearSVCs as external classifiers or a fully connected layer for end-to-end training.Both operate as alternatives for the final classification layer.
4. Evaluation
FeTrIL is evaluated across four datasets, varied incremental settings, and comparisons with exemplar-free and exemplar-based methods. It generally achieves strong accuracy while balancing stability and plasticity, with efficient incremental updates.
- Evaluation setup: Experiments cover four datasets and CIL configurations ranging from five to 100 incremental states.CIFAR-100 and ImageNet-Subset use multiple initial-state and incremental-state configurations; TinyImageNet includes one-class increments.
- Comparison to existing EFCIL methods: FeTrIL1 outperforms all compared methods in 11 of 12 tested EFCIL configurations.It is close to the best method in the remaining configuration.
- Comparison to existing EFCIL methods: FeTrIL gains over 4 and 3 top-1 accuracy points on TinyImageNet and ImageNet-Subset, respectively.On CIFAR-100, FeTrIL1 and SSRE are close, with relative differences between 0.4 and -0.2.
- Behavior for minimal incremental updates: FeTrIL supports one-class increments, achieving accuracy close to that observed with T = 20.Its pseudo-features can originate from a single new class, unlike compared discriminative methods requiring at least two classes per state.
- Detailed view of accuracy: FeTrIL is better throughout the process on TinyImageNet and better than SSRE during early ImageNet-Subset states.Its evolution is very similar to SSRE on CIFAR-100, while gains over other methods are larger across all incremental states.
- Comparison with exemplar-based methods: FeTrIL remains competitive with exemplar-based methods despite their storage of 20 images per class.It exceeds the three AANets base methods on CIFAR-100, while on ImageNet-Subset it trails Mnemonics slightly and PODNet by approximately 3.5 points.
- Pseudo-feature selection comparison: Pseudo-feature accuracy varies within a relatively small range across selection strategies, supporting FeTrIL's robustness to feature-selection policy.Translation toward the past-class centroid is more important than the sampling policy, motivating FeTrIL1 for the main experiments.
- Stability-plasticity balance: The accuracy gap between past and new classes is smaller for FeTrIL than SSRE except at state 4.The authors attribute the state-4 new-class performance to a probable strong domain shift; globally, FeTrIL improves the stability-plasticity balance.
5. Conclusion
FeTrIL combines a frozen feature extractor with pseudo-feature generation to improve exemplar-free class-incremental learning while reducing memory and speed demands. Its main limitations concern bias toward past classes, imperfect pseudo-features, and potentially improvable selection strategies.
- FeTrIL combines a frozen feature extractor with geometric pseudo-feature translation, improving results over recent exemplar-free class-incremental methods.The method generates pseudo-features for past classes and is advantageous from memory and speed perspectives.
- FeTrIL is particularly relevant for edge devices because their storage and computation capacities are limited.
- FeTrIL narrows, but does not eliminate, the performance gap between exemplar-free and exemplar-based class-incremental learning.
- FeTrIL favors past classes over new ones because its feature extractor is frozen after training on the initial state.Future work will combine pseudo-feature generation with fine-tuning to improve global performance and the stability-plasticity balance.
- Future work will refine pseudo-features toward original past-class features and improve selection by filtering outliers in representation space.