Source-linked AI summary
Re-Evaluating Continual Learning with Few-Shot Adaptation
Amogh Inamdar, Matthew So, Vici Milenia, Richard Zemel
TL;DR
Standard continual-learning measures can miss recovery of prior-task performance and rapid adaptation to new tasks. The paper evaluates continual image-classification models with few-shot adaptation, introduces per-shot plasticity, and develops foresight meta-learning. Few-shot evaluation reveals strong backward-task recovery and adaptation patterns that conventional measures miss.
Problem
Standard stability evaluation uses performance on prior tasks without adaptation, while plasticity is typically measured on the most recently learned task, leaving rapid reacquisition and within-task adaptation unmeasured.
Source: §1 · pp. 1–2Method
The paper evaluates all tasks with few-shot adaptation, introduces per-shot plasticity, and meta-learns from new future examples to optimize few-shot performance.
Source: §1 · p. 2, §2.3 · p. 4, §5 · p. 8Results
Few-shot evaluation finds strong recovery on backward tasks with very few examples and adaptation patterns missed by standard metrics, including gradual plasticity loss in EWC [29].
Source: §6 · p. 10Takeaways & Limitations
The findings support measuring plasticity as few-shot forward transfer over the learning sequence rather than only as aggregated post-training accuracy.
Source: §6 · p. 10The mechanism by which future-example foresight improves performance and learning-to-learn behavior is not investigated.
Source: §6 · p. 11
Abstract
from arXiv · showhide
Continual learning methods aim to maximize the stability and plasticity of machine learning models that are trained on a sequence of tasks. The standard measure of stability (i.e., forgetting) is the 0-shot performance of a model on previously learned tasks, and plasticity, the performance on the most recently learned task. However, 0-shot evaluation does not fully measure a model or method's ability to retain learned information or adapt quickly to new information, as it requires perfect recall across multiple tasks. In this paper, we propose few-shot evaluation as a more comprehensive assessment of the stability and plasticity of a continual learning system. We conduct a fine-grained assessment on task sequences for continual image classification and find that this paradigm produces novel insights into the performance of popular continual learning strategies. Through few-shot evaluation with a novel metric -- per-shot plasticity -- we show that adding `foresight' to continual learning methods via the meta-learning of a short sequence of future tasks induces learning-to-learn behavior over the task sequence.
1 Introduction
Standard 0-shot measures can miss both the recovery of prior-task performance and how quickly a model adapts to new tasks. The paper proposes continuous few-shot evaluation, a per-shot plasticity metric, and foresight meta-learning for continual learning.
1 Introduction
- Few-shot adaptation can reveal that low recall on prior tasks is not necessarily catastrophic forgetting, since a few examples may recover information that recent optimization made harder to access.The paper relates this to the relearning effect: rapid re-acquisition of seemingly forgotten concepts.Source: §1 · pp. 1–2
- Plasticity is often measured only by recent-task performance, while forward transfer can be trivially zero across tasks with different outputs and does not measure within-task adaptation.The paper argues that an ideal plasticity measure should capture adaptation with limited information or training.Source: §1 · p. 2
- The paper proposes continuous few-shot evaluation, per-shot plasticity, and foresight meta-learning that uses future samples to optimize learning-to-learn.The foresight method meta-initializes the learner on future samples before each task.Source: §1 · p. 2, Figure 1 · p. 2
2 Related Work
Continual learning spans different task-sequence settings and methods, while its standard evaluation emphasizes forgetting and immediate post-task performance. Prior work on evaluation and meta-learning leaves gaps in measuring few-shot adaptation and learning-to-learn over task sequences.
2.1 Continual Learning
- Replay-based strategies store and retrain on past examples, alongside approaches that distill previous states, regularize information, or modify model architecture.Continual image classification is the focus here, amid class-, task-, and domain-incremental settings.Source: §2.1 · pp. 2–3
2.2 Evaluation in Continual Learning
- Standard evaluation measures stability through forgetting and plasticity through performance immediately after optimizing on a task.The standard stability measure is Backward Transfer/Average Forgetting; stability is defined as its negative.Source: §2.2 · p. 3
- Existing evaluation treats plasticity reactively, while 0-shot forward transfer is trivially zero when new tasks share no outputs or structure.Learning Curve Area averages performance across 0–K shots, rather than measuring adaptation rapidity; SAUCE measures improvement with added few-shot examples and cumulative error in early adaptation.Source: §2.2 · p. 3
2.3 Meta-learning and continual learning
- Existing continual meta-learning either modifies the learner architecture or uses a meta-optimizer, but many methods evaluate short sequences, focus on forgetting, and do not optimize few-shot performance.The paper develops foresight meta-continual learning that meta-learns on new examples to optimize few-shot performance.Source: §2.3 · p. 4
3 Stability and Plasticity in the Few-Shot Paradigm
The section compares zero- and few-shot evaluation across visual continual-learning sequences, finding that some apparent forgetting is recoverable and that few-shot transfer reveals plasticity differences obscured by zero-shot measures.
3.1 Datasets and Methods
- Five image-task sequences compare replay, distillation, regularization, meta-continual-learning, and SGD methods, including rotated domain-incremental MNIST and superclass-based CIFAR-100 tasks.The experiments also include MNIST binary tasks and CIFAR-100 sequences with random class groupings.Source: §3.1 · p. 4
3.2 Few-Shot Stability Highlights Non-Catastrophic Forgetting
- Five-shot adaptation lets SGD recover substantial backward-task accuracy, sometimes approaching replay methods, despite its near-random zero-shot performance.The passage describes few-shot test-time replay as nearly as effective as training-time replay, with lower overhead because examples are replayed only at evaluation.Source: §3.2 · p. 5
3.3 Few-Shot Forward Transfer Differentiates Method Plasticity
- Ten-shot forward transfer reveals method differences obscured by similar zero-shot current-task accuracy: EWC loses plasticity on long sequences, while AGEM’s plasticity tracks task overlap.AGEM performs well on low-overlap STRUCT-CIFAR100-20 and poorly on higher-overlap SEQ-CIFAR100-10; replay methods also fare worse with lower task overlap, and MER does not scale well beyond MNIST.Source: §3.3 · pp. 5–6
3.2 Few-Shot Stability Highlights Non-Catastrophic Forgetting
- After 10-shot adaptation, most methods maintain backward stability over 20-task sequences, but none significantly improves stability over time.MER is the exception to the reported stable capacity.Source: §3.2 · p. 5, Figure 4 · p. 6
3.3 Few-Shot Forward Transfer Differentiates Method Plasticity
- Checkpoint trends show AGEM stable on SEQ-CIFAR100-20, SGD and replay improving slightly, and ROT-MNIST-20’s similar end tasks benefiting every method except EWC.EWC’s forward transfer trends downward beyond the first few tasks.Source: Figure 4 · p. 6, §3.3 · p. 6
4 Improving Plasticity with Foresight Meta-Learning
Continual-learning methods can stagnate or lose few-shot adaptation ability over a task sequence. The paper adds foresight meta-learning using examples from upcoming tasks; most methods then improve forward transfer beyond the look-ahead window, with some also improving backward transfer.
4 Improving Plasticity with Foresight Meta-Learning
- The method uses up to k unseen examples from future tasks to meta-learn the task sequence before full training on the upcoming task, adding a forward-looking inner loop to continual learning.It is distinguished from hindsight meta-learning, which uses examples already learned.Source: §4 · p. 6
- Foresight meta-learning improves forward transfer beyond the look-ahead window for most methods on each sequence, and sometimes improves backward transfer despite no direct optimization for it.STRUCT-CIFAR100-20 is the exception, where the lack of shared task structure appears to lead to worse performance with meta-learning.Source: Figure 5 · p. 7, §4 · p. 8
- An inner loop uses MAML or Reptile on look-ahead tasks; the structured variant trains each look-ahead model across a future-task sequence, which is a singleton in these experiments.The passage notes that additional sequences, such as permutations, may help when task ordering is not crucial.Source: §4 · p. 7
- On ROT-MNIST-20, second-order MAML generally beats alternatives, Reptile performs well with ER, EWC gains little, and sequential and parallel updates perform similarly.The authors hypothesize that longer task sequences may be needed to reveal differences between update strategies.Source: Figure 6 · p. 7, §4 · p. 8
5 Measuring the Rate of Adaptation with Per-shot Plasticity
The section introduces SAUCE to isolate rapid adaptation from absolute performance and applies forward and backward per-shot plasticity across task sequences. Foresight meta-learning improves rapid adaptation and, on all but ROT-MNIST-20, shows increasing plasticity over sequences.
5 Measuring the Rate of Adaptation with Per-shot Plasticity
- Choosing a shot count depends on the task, method, adaptation needed to distinguish methods, and examples available in deployment; aggregating across shot counts can miss adaptation speed.The caption identifies this multi-shot aggregation with LCA [13].Source: Figure 7 · p. 8
- Adding foresight meta-learning modestly improves average rapid adaptation across settings, with the largest improvement for AGEM, especially on low-similarity tasks.Figure 8 reports forward and backward per-shot plasticity measured by averaged SAUCE; lower is better.Source: Figure 8 · p. 9, §5 · p. 9
- SAUCE isolates adaptation rate by scaling worst and best performance to 1 and 0, respectively, and normalizing by the adaptation range to yield a value in [0,1].It builds on AUAC, which aggregates error across adaptations and approximates early training regret for gradient descent.Source: §5 · pp. 8–9
- Forward per-shot plasticity measures learning-to-learn on future tasks, while backward per-shot plasticity aggregates SAUCE over previously learned tasks.SAUCE measures relative performance improvement as the amount of adaptation increases.Source: §5 · p. 9
- Replay methods ER and DER++ perform well in few-shot evaluation but adapt less rapidly than SGD and regularization-based methods, a trade-off when examples are scarce.Their high performance ceiling may not suit practitioners when adaptation examples are very limited.Source: §5 · p. 9
- Without foresight meta-learning, adaptation rates remain relatively flat across long sequences, with task-similarity-dependent differences between methods.On overlapping SEQ-CIFAR100-20, EWC and SGD have the highest forward plasticity and MER the lowest; on STRUCT-CIFAR100-20, AGEM improves most rapidly and has the best final plasticity, while methods are equivalent and relatively flat on ROT-MNIST-20.Source: §5 · p. 9
- On every sequence except ROT-MNIST-20, foresight meta-learning steadily increases plasticity in both forward and backward tasks, indicating learning-to-learn behavior.The reported trend cannot be attributed simply to the additional forward information from look-ahead examples.Source: §5 · pp. 9–10, Figure 9 · p. 10
6 Discussion
Few-shot evaluation reveals recovery of performance on earlier tasks and changes how continual-learning plasticity is assessed. The discussion also reviews continual-learning and meta-learning approaches and identifies limits of the study.
6 Discussion
- Few-shot adaptation found strong recovery on backward tasks, sometimes with only 1 or 2 examples per class, suggesting some apparent forgetting reflects retrieval rather than catastrophic knowledge loss.The authors suggest practitioners may rely on few-shot test-time adaptation to the task at hand rather than storing and replaying samples for every task; replay may potentially hurt stability [38].Source: §6 · p. 10
- Per-shot plasticity measures few-shot forward transfer across the sequence, and foresight meta-learning optimizes plasticity over that sequence.The analysis surfaces gradual plasticity loss in EWC [29] and AGEM [13] benefiting from task independence; per-shot plasticity highlights learning-to-learn behavior when present.Source: §6 · p. 10
- The study is limited by the parameter settings assessable at scale, an unexplored mechanism for foresight improvements, and its focus on visual classification.The authors identify in-context learning as a direction for studying adaptation in continually learning foundation models.Source: §6 · p. 11
A.1 Continual Learning Strategies
B Meta-Learning and Meta-CL
- MAML and Reptile learn an initialization using few examples from pretraining tasks, while first-order approximations reduce MAML’s second-order computation.Reptile’s finite-difference approximation supports a model-merging update; with a small meta-learning rate, the update remains near the base parameters.Source: B Meta-Learning and Meta-CL · p. 14
- Meta-CL methods either use meta-information to modify learner architecture or use a meta-optimizer to leverage cross-task information during updates.Examples include ANML [4] and OML [27] for architectural strategies, and MER, Online Meta-Learning, La-MAML, and VR-MCL for meta-optimizer strategies.Source: B Meta-Learning and Meta-CL · p. 15
C.1 Datasets
The experiments use five continual-learning task sequences spanning image classification.
C.1 Datasets
- Five continual-learning task sequences spanning image classification are evaluated, and Table 1 summarizes their key properties.The supplied passage identifies Table 1 as dataset statistics; it does not state the table’s values.Source: §C.1 · p. 15, Table 1 · p. 15
C.2 Code and Computer Resources
Experiments use the Mammoth visual continual-learning library and shared GPU resources.
C.2 Code and Computer Resources
- Experiments use Mammoth implementations of six continual-learning methods and shared NVIDIA A6000 and A100 GPUs.Each experiment involves up to 1000 cumulative training epochs and 2000 evaluation forward passes; each epoch or pass takes seconds or minutes on the available resources.Source: §C.2 · p. 15
C.2.1 CL Methods
The study trains models sequentially on tasks and applies foresight meta-learning as a wrapper around base continual-learning methods.
C.2.1 CL Methods
- Each method trains one model sequentially across tasks, using a randomly initialized ResNet-18 or 32-hidden-unit MLP; non-meta methods see no future-task data during training.A checkpoint is saved after each task for k-shot adaptation and evaluation.Source: §C.2.1 · p. 15
- Base methods use original-paper hyperparameters; listed learning rates are SGD 0.1, ER 0.1, DER++ 0.03, EWC 0.1, AGEM 0.03, and MER 0.1.Buffer sizes are ER 500, DER++ 500, AGEM 500, and MER 200; DER++ uses α=0.3, β=0.5, EWC uses λ=10, γ=1.0, and MER uses β=0.01, γ=0.03.Source: §C.2.1 · pp. 15–16
- MER uses batch size 1; other methods use batch size 32 on CIFAR100 variants and 64 on MNIST, with 50, 5, and 1 training epochs respectively.The 50-epoch schedule applies to CIFAR100 variants, 5 epochs to ROT-MNIST-20, and 1 epoch to SEQ-MNIST-5.Source: §C.2.1 · p. 16
C.2.2 Foresight Meta-Learning
- Foresight meta-learning wraps a base method with a meta-update at each task’s start, then runs the base method’s training logic unchanged.Meta-SGD uses SGD without weight decay in the inner loop; other wrapped methods use their own training logic, including replay or regularization.Source: §C.2.2 · p. 16
- Foresight uses k=5 examples per class per lookahead task, with L=1 for SEQ-MNIST-5 and L=3 for other datasets.Reptile uses β=0.2 on SEQ-CIFAR100-10, β=0.1 on SEQ-CIFAR100-20, and β=0.01 on remaining datasets; MAML uses β=0.001. Total foresight examples N are 10 for SEQ-MNIST-5, 25 for SEQ-CIFAR100-20 and STRUCT-CIFAR100-20, and 50 for the remaining datasets.Source: §C.2.2 · p. 16
C.3 Evaluation
Each saved checkpoint is tested on held-out test sets for all tasks, with k-shot adaptation before evaluation. For meta-CL methods, look-ahead training samples are excluded from forward-transfer evaluation.
C.3 Evaluation
- For each checkpoint and task, evaluation adapts on k randomly sampled training examples for 10 gradient steps, then tests on the task’s full held-out set.The adaptation learning rate is taken from Table 2.Source: §C.3 · p. 16, Table 2 · p. 16
- Each checkpoint is evaluated on tasks both past and future relative to that point in the sequence.Source: §C.3 · p. 16
- Meta-CL forward-transfer evaluation omits look-ahead samples used during training, avoiding evaluation on tasks explicitly optimized during training.Source: §C.3 · p. 16
- Table 2 lists evaluation adaptation learning rates by method and dataset.Source: Table 2 · p. 16
D Additional Results
The additional results show k-shot accuracy across checkpoints and task sequences, including comparisons with foresight meta-learning. The reported plots cover task- and domain-incremental settings and describe differences in per-shot plasticity.
D Additional Results
- Figure 10 plots mean k-shot accuracy for non-meta continual-learning baselines on backward, current, and forward SEQ-MNIST-5 tasks, averaged across checkpoints.Error bars show standard error over 10 seeds.Source: Figure 10 · p. 17
- Figure 11 plots SGD k-shot accuracy across 20 checkpoints and evaluation tasks on SEQ-CIFAR100-20, with adaptation budgets from 0 to 10 shots.Shaded bands show standard error over 5 seeds.Source: Figure 11 · p. 18
- Figure 12 presents the corresponding checkpoint-wise SGD k-shot evaluation on ROT-MNIST-20, with budgets from 0 to 10 shots.Shaded bands show standard error over 10 seeds.Source: Figure 12 · p. 18
- Figure 13 reports improved 5-shot forward transfer when foresight meta-learning is added to continual-learning methods.Methods use MAML with one look-ahead task on SEQ-MNIST-5 or three on other datasets; look-ahead tasks are excluded from the forward measurement.Source: Figure 13 · p. 19
- Figure 14 plots meta-SGD variants’ mean k-shot accuracy on backward, current, and forward ROT-MNIST-20 tasks, averaged across checkpoints.Error bars show standard error over 10 seeds.Source: Figure 14 · p. 20
- Figure 15 plots parallel MAML with 20 adaptation and 20 meta-adaptation steps across ROT-MNIST-20 checkpoints, tasks, and 0-to-10-shot budgets.Shaded bands show standard error over 10 seeds.Source: Figure 15 · p. 21
- On SEQ-MNIST-5 and ROT-MNIST-20, the adaptation rate (SAUCE) shows little variation; the authors hypothesize task-shift structure explains its behavior on ROT-MNIST-20.They suggest foresight methods benefit from fine-grained structure shared across tasks, such as a common superclass.Source: Figure 16 · p. 21