PaperlayerView exact revision on arXiv

Source-linked AI summary

Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence

Arslan Chaudhry, Puneet K. Dokania, Thalaiyasingam Ajanthan, Philip H. S. Torr

arXiv:1801.10112v3cs.CV

TL;DR

Incremental learning lacks precise definitions, evaluation settings, and dedicated metrics, while models must learn new tasks with limited access to old data. The paper introduces forgetting and intransigence measures plus RWalk, a KL-divergence-based generalization of EWC++ and Path Integral. RWalk outperforms the baselines in average accuracy and trade-off between forgetting and intransigence, while representative samples mitigate intransigence.

  • Problem

    Incremental learning lacks precise problem definitions, evaluation settings, and metrics despite requiring models to learn new tasks while preserving existing knowledge.

  • Method

    The paper introduces forgetting and intransigence metrics and presents RWalk as a KL-divergence-based generalization of EWC++ and Path Integral.

  • Results

    RWalk outperforms all baselines in average accuracy and provides a better trade-off between forgetting and intransigence.

  • Takeaways & Limitations

    Small representative subsets of previous-task data can substantially mitigate intransigence, including in single-head evaluation.

  • Takeaways & Limitations

    CIFAR-100 results for iCaRL were substantially worse than previously reported, possibly because the smaller CNN reduced feature-space expressivity.

Abstract

from arXiv · show

Incremental learning (IL) has received a lot of attention recently, however, the literature lacks a precise problem definition, proper evaluation settings, and metrics tailored specifically for the IL problem. One of the main objectives of this work is to fill these gaps so as to provide a common ground for better understanding of IL. The main challenge for an IL algorithm is to update the classifier whilst preserving existing knowledge. We observe that, in addition to forgetting, a known issue while preserving knowledge, IL also suffers from a problem we call intransigence, inability of a model to update its knowledge. We introduce two metrics to quantify forgetting and intransigence that allow us to understand, analyse, and gain better insights into the behaviour of IL algorithms. We present RWalk, a generalization of EWC++ (our efficient version of EWC [Kirkpatrick2016EWC]) and Path Integral [Zenke2017Continual] with a theoretically grounded KL-divergence based perspective. We provide a thorough analysis of various IL algorithms on MNIST and CIFAR-100 datasets. In these experiments, RWalk obtains superior results in terms of accuracy, and also provides a better trade-off between forgetting and intransigence.

1 Introduction

The paper frames incremental learning as continual task learning with limited access to prior data, requiring classifiers to preserve old knowledge while learning new tasks. It introduces metrics for forgetting and intransigence and proposes RWalk as a KL-divergence-based generalization of EWC++ and Path Integral.

  • Incremental learning expands the output space while restricting access to previous-task data, unlike classical learning with fixed outputs and complete datasets.
  • IL algorithms must preserve previously learned knowledge while updating the classifier for new tasks, creating a trade-off between forgetting and intransigence.Forgetting is catastrophic loss of prior knowledge, whereas intransigence is inability to learn new tasks.
  • The paper introduces forgetting and intransigence metrics alongside multi-class average accuracy to analyze incremental-learning behavior.
  • RWalk generalizes EWC++ and Path Integral using a theoretically grounded KL-divergence perspective.EWC++ is presented as an efficient, online version of EWC, while RWalk modifies Path Integral's sensitivity measure using approximate KL divergence.
  • The study examines sampling strategies that retain a small representative subset of previous-task data to support recall and current-versus-previous task discrimination.The retained subset is no larger than 5% of the dataset.
  • The paper analyzes incremental-learning methods in terms of accuracy, forgetting, and intransigence on MNIST and CIFAR-100.

2 Problem Set-up and Preliminaries

The paper distinguishes single-head and multi-head incremental-learning evaluations and connects KL divergence to a Fisher-induced Riemannian distance. The Fisher matrix is attractive theoretically but expensive to compute and store at neural-network scale.

  • Problem setup: Incremental learning assumes a stream of tasks, each associated with a set of labels, and previous data cannot be stored indefinitely in a scalable manner.
  • Evaluation settings: Single-head evaluation requires classification over all labels seen so far without a task identifier, whereas multi-head evaluation supplies the task identifier and restricts prediction to current-task labels.
  • Evaluation settings: Single-head evaluation is harder because the classifier must learn inter-task discrimination with limited or no access to previous-task data.
  • Probabilistic preliminaries: For a softmax network trained with cross-entropy, the output can be interpreted as a categorical likelihood distribution over classes.
  • Riemannian perspective: For small parameter changes, KL divergence is approximated quadratically using the empirical Fisher Information Matrix.
  • Riemannian perspective: The Fisher matrix induces a Riemannian distance, but storing the full matrix is infeasible for networks with millions of parameters, motivating a diagonal approximation.The diagonal approximation assumes parameter independence; computing its entries still requires a full forward-backward pass over the dataset.

3 Forgetting and Intransigence

The paper evaluates incremental learning through average accuracy, forgetting, and intransigence, separating preservation of prior knowledge from learning of new tasks. Its forgetting metric tracks the maximum earlier knowledge reached, while intransigence compares incremental learning with a reference model.

  • Interpretation: The proposed metrics jointly expose the trade-off between preserving prior knowledge and learning new tasks, which standard accuracy alone cannot characterize.
  • Evaluation metrics: Average accuracy summarizes performance across held-out tasks but does not reveal an algorithm's forgetting or intransigence profile.
  • Forgetting: Forgetting measures the difference between the maximum knowledge previously gained for a task and the model's current knowledge of it.
  • Forgetting: The forgetting measure uses the maximum prior accuracy rather than only accuracy immediately after learning the task, capturing changes throughout incremental training.
  • Intransigence: Intransigence is defined as the inability to learn new tasks and is quantified by comparing incremental accuracy with a reference model trained using all task data.
  • Intransigence: Lower intransigence is better, and negative intransigence indicates positive forward transfer while positive intransigence indicates negative forward transfer.

4 Riemannian Walk for Incremental Learning

RWalk combines KL/Fisher regularization, optimization-path parameter importance, and representative old-task samples to address forgetting and intransigence with task-independent memory growth.

  • RWalk combines KL-divergence regularization, optimization-path parameter importance, and representative samples from previous tasks.The first two components mitigate catastrophic forgetting, while sampling addresses intransigence.
  • EWC++ maintains a single diagonal Fisher matrix online, avoiding per-task storage and an additional end-of-task dataset pass.It stores only two Fisher sets at any instant, irrespective of the number of tasks.
  • RWalk augments Fisher importance with a trajectory score based on loss change relative to KL-divergence movement between successive model distributions.Importance is higher when a small distribution change produces a large loss improvement.
  • RWalk averages and normalizes importance scores across tasks, reducing the influence of distant tasks and making λ less sensitive to task count.The Fisher and trajectory-based terms are normalized to [0, 1] before combination.
  • RWalk uses at most O(P) space, independent of the number of tasks.This contrasts with EWC’s O(kP) Fisher storage requirement when each task is stored separately.
  • Representative old-task samples help single-head classifiers learn inter-task discrimination because training labels cover only the current task.The paper considers uniform random sampling and boundary-based sampling, where samples near the decision boundary are treated as more representative.

5 Related Work

Related approaches address forgetting through network expansion, activation or parameter regularization, moment matching, Bayesian weight distributions, gradient updates, or generative replay.

  • Network-expansion methods add capacity for each task but are not scalable as the number of tasks grows.
  • RWalk regularizes parameters by individual importance and generalizes EWC++ and Path Integral, while EWC++ is an efficient version of EWC.
  • Other approaches use moment matching, Bayesian parameter distributions, gradient constraints, or generated previous-task samples.

6 Experiments

Experiments on incremental MNIST and CIFAR-100 compare RWalk with regularization, sampling, and unregularized baselines under multi-head and single-head evaluation. RWalk achieves the strongest average-accuracy and forgetting–intransigence trade-offs, while representative samples substantially mitigate intransigence.

  • Results: RWalk outperforms all baselines in average accuracy and provides a better trade-off between forgetting and intransigence.The comparison covers MNIST and CIFAR-100 in both multi-head and single-head settings.
  • Evaluation settings: Multi-head evaluation makes incremental learning appear easier because all methods except Vanilla achieve state-of-the-art accuracy with almost zero forgetting and intransigence.The task identifier is available during multi-head evaluation, unlike the more difficult single-head setting.
  • Single-head evaluation: On MNIST single-head evaluation, Vanilla’s forgetting rose from 0.12 to 0.62 and intransigence from 6.6 × 10^-4 to 0.29, while average accuracy fell from 90.3% to 38.0%.PI’s accuracy likewise fell from 99.3% in multi-head evaluation to 57.6% in single-head evaluation.
  • Representative samples: With only 10 (≈0.2%) samples per previous MNIST class, PI’s intransigence dropped from 0.8 to 0.05 and average accuracy increased from 57.6% to 78.7%.On CIFAR-100, almost identical behaviour was observed with 5% representative samples.
  • Effect of increasing the number of samples: For smaller memories, regularized methods outperform Vanilla; Vanilla catches up with 20 samples on MNIST but requires 200 samples on CIFAR-100.These values correspond to 0.4% of total samples for MNIST and 40% for CIFAR-100.
  • Sampling strategies: Mean-of-Features subset selection performs best overall, although uniform sampling is as good as more complex strategies and regularized methods are insensitive to sampling choice.Vanilla’s performance varies substantially across sampling strategies.

7 Discussion

The paper frames incremental learning around catastrophic forgetting and intransigence, introduces metrics for both, and presents EWC++ and RWalk as KL-divergence-based regularization methods. Experiments show representative samples can alleviate intransigence in the single-head setting.

  • The paper introduces forgetting and intransigence as metrics for analyzing the competing challenges of preserving old knowledge and learning new tasks.
  • EWC++ is an efficient version of EWC, while RWalk generalizes EWC++ and Path Integral through a KL-divergence-based perspective.
  • Parameter-regularization methods suffer from high intransigence in practical single-head evaluation.
  • A small subset of representative samples alleviates the intransigence of parameter-regularization methods.
  • Because these methods are more memory efficient than knowledge-distillation methods such as iCaRL, the paper identifies incremental learning on segmentation tasks as a direction for future research.

Supplementary Material

The supplementary material extends the main analysis with KL-divergence details, comparisons against GEM, hyperparameter sensitivity experiments, and implementation information.

  • The supplement provides details on the KL-divergence approximation using the Fisher information matrix, including true versus empirical Fisher and why Fisher vanishes at a minimum.
  • A supplementary comparison reports that RWalk significantly outperforms GEM, although this comparison is not available in the main paper.
  • The supplement analyzes sensitivity to the regularization hyperparameter λ and gives architecture details in later sections.

A.1 Proof of Approximate KL divergence

The proof derives a second-order approximation of KL divergence and clarifies how Fisher matrices arise under model-based and empirical expectations. It also explains why the Fisher approaches zero at an optimum and why empirical Fisher is used computationally.

  • The proof begins from the KL-divergence definition and applies a second-order Taylor expansion around θ for a small parameter change Δθ.
  • Under model-based sampling, first-order terms cancel and the expected negative Hessian equals the true Fisher matrix by the Information Matrix Equality.
  • With dataset sampling, the resulting matrix is the empirical Fisher, whereas at an optimum the model distribution approaches the data distribution and the two behave similarly.
  • The paper uses empirical Fisher approximation because computing the true Fisher requires multiple backward passes and is expensive.
  • At any optimum, the loss gradient approaches zero, so the Fisher, as the expected loss-gradient covariance matrix, also approaches a zero matrix.

B.1 Comparison with GEM [15] on ResNets

The supplementary experiments compare RWalk with iCaRL and GEM on ResNet-based CIFAR-100 tasks and examine sensitivity to regularization strength. RWalk outperforms the compared methods and is less sensitive to λ than EWC and PI.

  • The ResNet experiments use CIFAR-100 split into 20 tasks of five consecutive classes and evaluate all methods in the multi-head setting.
  • Table 3 compares MNIST and CIFAR-100 methods as regularization strength λ varies, reporting forgetting, intransigence, and their changes relative to each method’s first row.
  • RWalk outperforms iCaRL and GEM by a significant margin in the reported ResNet comparison.
  • When λ varies by 1 × 10^5 on MNIST, EWC changes by −0.06 in forgetting and 0.14 in intransigence, while PI changes by −0.07 and 0.13, respectively.
  • RWalk is less sensitive to λ than EWC and PI, which the paper attributes to normalization of Fisher and path-based importance scores.

B.3 CIFAR Architecture and Task-Level Analysis

The CIFAR-100 experiments use a task-specific convolutional architecture and analyze accuracy across tasks under multi-head and single-head evaluations. Figure 6 tracks task performance as training proceeds and reports average accuracy across tasks.

  • CIFAR Architecture: Unlike PI, the network uses only one fully-connected layer, while each task dynamically adds weights in the last layer.This task-specific last-layer expansion is described as part of the CIFAR-100 experimental setup.
  • CIFAR Architecture: The CIFAR-100 experiments use a CNN architecture shared across Vanilla, EWC, PI, iCaRL, and RWalk.The architecture is detailed in Table 4, where n denotes the number of classes per task.
  • Task-Level Analysis: Figure 6 evaluates incremental CIFAR-100 accuracy with multi-head evaluation and with single-head evaluation both without and with samples.The figure presents these settings in top, middle, and bottom panels, respectively.
  • Task-Level Analysis: The first ten columns show how each task’s performance changes as the model is sequentially trained on ten tasks.The first plot, for example, tracks Task 1 while the network is trained on Tasks 1 through 10.
  • Task-Level Analysis: The final column reports average accuracy, denoted A_k, while varying k.This complements the task-level accuracy trajectories shown in the preceding columns.
Loading 1801.10112v3…