PaperlayerView exact revision on arXiv

Source-linked AI summary

TRGP: Trust Region Gradient Projection for Continual Learning

Sen Lin, Li Yang, Deliang Fan, Junshan Zhang

arXiv:2202.02931v1cs.LG

TL;DR

Continual learning seeks to avoid catastrophic forgetting, but constraints that protect old tasks can hinder learning a strongly correlated new task. TRGP selects correlated old tasks layer by layer and reuses their frozen weights through scaled projections. The authors report substantial gains over related state-of-the-art methods across benchmarks and tasks.

  • Problem

    Methods that constrain new-task optimization to reduce interference may compromise new-task performance, particularly when the new task is strongly correlated with old tasks.

    Source: §1 · p. 1

  • Method

    TRGP selects correlated old tasks layer by layer using projected-gradient norms, reuses their frozen weights through scaled projections, and jointly optimizes scaling matrices and the new-task model.

    Source: §1 · p. 2

  • Results

    TRGP substantially improves over related state-of-the-art approaches across standard continual-learning benchmarks and tasks.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    The reported experiments indicate that TRGP promotes forward knowledge transfer while alleviating forgetting.

    Source: §1 · p. 2

    The setting assumes sequential tasks and access only to the current task’s dataset during learning.

    Source: §3 · p. 3

Abstract

from arXiv · show

Catastrophic forgetting is one of the major challenges in continual learning. To address this issue, some existing methods put restrictive constraints on the optimization space of the new task for minimizing the interference to old tasks. However, this may lead to unsatisfactory performance for the new task, especially when the new task is strongly correlated with old tasks. To tackle this challenge, we propose Trust Region Gradient Projection (TRGP) for continual learning to facilitate the forward knowledge transfer based on an efficient characterization of task correlation. Particularly, we introduce a notion of `trust region' to select the most related old tasks for the new task in a layer-wise and single-shot manner, using the norm of gradient projection onto the subspace spanned by task inputs. Then, a scaled weight projection is proposed to cleverly reuse the frozen weights of the selected old tasks in the trust region through a layer-wise scaling matrix. By jointly optimizing the scaling matrices and the model, where the model is updated along the directions orthogonal to the subspaces of old tasks, TRGP can effectively prompt knowledge transfer without forgetting. Extensive experiments show that our approach achieves significant improvement over related state-of-the-art methods.

1 INTRODUCTION

Continual learning methods can still forget old-task knowledge, while non-expansion methods constrain new-task updates to limit interference. TRGP characterizes task correlation to support forward transfer while mitigating forgetting.

1 INTRODUCTION

  • Existing non-expansion methods constrain new-task updates to reduce interference, but those constraints may throttle new-task learning.The paper conjectures this is especially likely when the new task is strongly correlated with old tasks.Source: §1 · pp. 1–2
  • The paper’s central insight is that strongly correlated old tasks can offer better forward transfer even when they make the new task’s optimization space more restrictive.Weights important to both the new task and correlated old tasks may need updating, although existing methods often freeze them.Source: §1 · p. 2
  • TRGP’s layer-wise, single-shot trust region selects old tasks correlated with the new task using projected-gradient norms onto task-input subspaces.The selected tasks have similar input features at the corresponding layer.Source: §1 · p. 2
  • Scaled weight projection reuses frozen weights from selected old tasks through a learned scaling matrix in each layer.The projection is used without modifying the model weights being reused.Source: §1 · p. 2
  • TRGP jointly optimizes the scaling matrices and new-task model while updating the model orthogonally to old-task subspaces.The orthogonal update is intended to mitigate forgetting further.Source: §1 · p. 2
  • TRGP substantially improves over related state-of-the-art approaches across standard continual-learning benchmarks and tasks.The reported performance indicates effective forward transfer while alleviating forgetting.Source: §1 · p. 2

2 RELATED WORK

Related continual-learning work includes methods that expand network capacity and methods that retain a fixed architecture. Fixed-architecture approaches use regularization, memory, or orthogonal gradient projection to address forgetting.

Expansion-based methods

  • Expansion-based methods dynamically add network capacity to reduce interference between new and old tasks.Examples include PNN, which preserves old-task weights while expanding the architecture, and LWF, which grows task-specific layers.Source: §2 · p. 2

Regularization-based methods

  • Regularization-based methods penalize changes to weights considered important for old tasks.EWC uses Fisher information, HAT learns hard attention masks, and MAS estimates output sensitivity to inputs.Source: §2 · p. 2

Memory-based methods

  • Memory-based methods either replay old-task data or project updates orthogonally to old-task subspaces.GEM and A-GEM alter gradients using memory data; OWM, OGD, and GPM use orthogonal-projection approaches, with differing stored representations.Source: §2 · pp. 2–3

3 PROBLEM FORMULATION

The formulation considers sequential tasks and a fixed-capacity network, with access to the current task’s dataset during learning. Orthogonal projection can preserve old-task outputs, but its restrictions may impede correlated new tasks.

3 PROBLEM FORMULATION

  • The setting uses a sequence of tasks in a fixed-capacity network and makes only the current task’s dataset available during its learning.Each task has input-label pairs, and the model has layer-wise weights.Source: §3 · p. 3
  • Orthogonal updates preserve the old task’s outputs because the model change is orthogonal to its input subspace, introducing no interference in the stated analysis.The passage says this argument generalizes to a sequence of tasks.Source: §3 · p. 3
  • Restricting updates to directions orthogonal to old-task inputs can narrow the new task’s optimization space and compromise its performance.The formulation motivates this concern with examples involving two tasks.Source: §3 · p. 3
  • In toy example 1, sign-reversed inputs share the old task’s subspace, so orthogonal projection yields zero gradient update although the new task’s optimum uses negated weights.The example assumes both tasks share a classifier.Source: §3 · p. 3
  • In toy example 2, orthogonal task-input subspaces leave the new-task gradient unchanged under projection, allowing steepest descent without interference to the old task.Source: §3 · p. 3
  • Task correlation is characterized through input-subspace orientation: nearly orthogonal subspaces indicate weak correlation, while nearly parallel subspaces indicate strong correlation.Figure 1 illustrates the relationship for a two-dimensional plane.Source: §3 · p. 4, Figure 1 · p. 4

4 TRUST REGION GRADIENT PROJECTION FOR CONTINUAL LEARNING

TRGP aims to facilitate forward knowledge transfer without forgetting by selecting related old tasks and reusing their frozen weights through scaled weight projection. Its method combines layer-wise trust regions, task-subspace construction, and scaled projection.

4.1 TRUST REGION

  • TRGP selects related old tasks layer by layer using the norm of the new task’s gradient projected onto each old task’s input subspace; selecting top-K tasks with the largest projection norm can reduce sensitivity to the threshold.The authors report that K = 1 is sufficient for significant performance improvement.Source: §4.1 · pp. 4–5

4.2 SCALED WEIGHT PROJECTION

  • Scaled weight projection lets the new task reuse frozen weights from selected old tasks by learning a scaling matrix for the projection into their subspaces.This addresses the tension that projected weights may matter to the new task but are frozen to protect the old task.Source: §4.2 · pp. 5–6

4.3 TASK SUBSPACE CONSTRUCTION

  • Task subspaces are built from layer representations using SVD, with later tasks selecting important bases from both previously learned and newly constructed bases.For the first task, the method uses a low-rank approximation to select left singular vectors as subspace bases.Source: §4.3 · p. 6

4.4 CONTINUAL LEARNING WITH TRUST REGION GRADIENT PROJECTION

  • The TRGP procedure selects top-K correlated old tasks for each layer and updates the new-task gradient after removing its component in the subspace of old tasks.The approach builds on task-subspace construction, trust regions, and scaled weight projection.Source: §4.4 · p. 6

5 EXPERIMENTAL RESULTS

The experiments compare TRGP with continual-learning baselines across sequential benchmarks, then analyze its performance and design choices. The reported results favor TRGP on accuracy, while ablations examine threshold robustness, layer-wise trust regions, and task selection.

5.1 EXPERIMENTAL SETUP

  • Evaluation covers five sequential benchmarks and compares memory-, regularization-, and expansion-based methods using average final accuracy and backward transfer.The benchmarks are PMNIST, CIFAR-100 Split, CIFAR-100 Sup, 5-Datasets, and MiniImageNet Split.Source: §5.1 · p. 7

5.2 MAIN RESULTS

  • TRGP gains 2.43% over GPM on PMNIST, 1.98% on CIFAR-100 Split, and 1.37% on MiniImageNet in accuracy.It also gains 2.34% over HAT on 5-Datasets; on CIFAR-100 Split, its BWT is marginally worse than HAT's despite a 2.4% accuracy gain.Source: §5.2 · pp. 7–8
  • TRGP matches GPM on easier tasks and improves difficult-task accuracy across three datasets; on CIFAR-100 Sup, it outperforms other CL methods with fixed capacity.Figure 4 compares final accuracy on all tasks; the 5-Datasets discussion highlights gains on CIFAR-10, SVHN, and not-MNIST.Source: Figure 4 · p. 8, §5.2 · p. 8
  • TRGP learns scaled projections of related old-task weights in its trust region, while freezing unrelated projections and updating orthogonal directions.The authors attribute improved forward transfer to reusing strongly correlated task knowledge and reduced backward transfer to task-unique information in the scaled projection.Source: §5.2 · p. 8
  • During sequential training, TRGP outperforms GPM on the compared tasks across three datasets and reduces forgetting on PMNIST and 5-Datasets.Figure 5 shows accuracy evolution for selected tasks on PMNIST.Source: Figure 5 · p. 8, §5.2 · p. 8

5.3 ABLATION STUDY AND ANALYSIS

  • Across CIFAR-100 Split and 5-Datasets, accuracy is stable for trust-region thresholds 0.2, 0.5, and 0.7 because the selected top-two tasks are usually unchanged.Small fluctuations occur when increasing the threshold causes only one old task to satisfy the selection condition for some layers.Source: §5.3 · p. 9
  • Layer-wise trust regions gain 1.21% accuracy over task-wise regions on CIFAR-100 Split and perform better than or comparably to task-wise regions across tasks.Figure 6 compares final task accuracy for layer-wise TRGP, task-wise TRGP, and GPM.Source: Figure 6 · p. 9, §5.3 · p. 9
  • Selecting the top two most correlated tasks gives better accuracy than top one, while layer-wise selection varies by layer and can identify semantically related tasks.For Fashion MNIST, layer-wise selection often favors MNIST or not-MNIST over CIFAR-10 or SVHN; task-wise selection instead chooses adjacent previous tasks across layers.Source: §5.3 · p. 9, Figure 7 · p. 9

6 CONCLUSION

TRGP uses a layer-wise trust region to select old tasks correlated with a new task and scaled weight projection to reuse their frozen weights. The paper reports significant improvement over related state-of-the-art methods.

  • TRGP combines layer-wise, single-shot selection of correlated old tasks with scaled weight projection that reuses their frozen weights.The approach is based on an efficient characterization of task correlation.Source: §6 · p. 9
  • The paper reports significant improvement over related state-of-the-art methods.Source: §6 · p. 9

REPRODUCIBILITY STATEMENT

The supplemental material includes code for the main-text experiments. Training details and dataset explanations are identified in the paper.

A EXPERIMENT SETUPS

The experiments cover five continual-learning settings with task-specific network architectures and training schedules. Across experiments, the trust-region threshold is 0.5, top-2 qualifying tasks are selected, and models use plain stochastic gradient descent.

  • PMNIST uses a three-layer fully connected network with two 100-unit hidden layers, trained for 5 epochs per task with batch size 10.Source: A EXPERIMENT SETUPS · p. 12
  • CIFAR-100 Split uses a five-layer AlexNet for up to 200 epochs per task with early termination and batch size 64.Source: A EXPERIMENT SETUPS · p. 12
  • CIFAR-100 Sup uses modified LeNet-5 with 20-50-800-500 neurons, trained for 50 epochs per task with batch size 64.Source: A EXPERIMENT SETUPS · p. 12
  • 5-Datasets and MiniImageNet Split train for up to 200 and 100 epochs per task, respectively, with early termination and batch size 64.MiniImageNet Split uses reduced ResNet18 and an initial learning rate of 0.1.Source: A EXPERIMENT SETUPS · p. 12
  • All experiments set threshold ϵ_l to 0.5, select the top-2 tasks satisfying Eq. (3), initialize the scaling matrix as identity, and use plain stochastic gradient descent.The subspace-construction threshold matches GPM (Saha et al., 2021).Source: A EXPERIMENT SETUPS · p. 12

B MORE EXPERIMENTAL RESULTS

The supplementary experiments report accuracy evolution on CIFAR-100 Split and 5-Datasets, and standard deviations for averaged accuracy and backward transfer across five runs on all datasets.

B.1 ACCURACY EVOLUTION

  • Figure 8 presents accuracy evolution for different tasks on CIFAR-100 Split and 5-Datasets.Source: Figure 8 · p. 12

B.2 STANDARD DEVIATION

  • Table 4 summarizes standard deviations for averaged accuracy and backward transfer over five runs on all datasets.Source: §B.2 · p. 12

B.3 FORWARD TRANSFER

The section evaluates forward transfer using accuracy on each task immediately after sequential learning and reports gains for TRGP over GPM across four benchmarks.

B.3 FORWARD TRANSFER

  • Forward transfer is measured as accuracy Ai,i after learning the i-th task sequentially, using each task’s own-data, train-from-scratch accuracy as the shared baseline.Source: §B.3 · pp. 12–13
  • TRGP gains 0.17% on PMNIST, 2.01% on CIFAR-100 Split, 2.00% on 5-Datasets, and 2.36% on MiniImageNet over GPM in forward transfer.Source: §B.3 · p. 13, Table 5 · p. 13, Table 6 · p. 13
  • The per-task Ai,i comparisons cover 10-task PMNIST, 10-task CIFAR-100 Split, 5-task 5-Dataset, and 20-task MiniImageNet Split settings.Source: Table 5 · p. 13, Table 6 · p. 13, Table 7 · p. 13

B.4 COMPUTATIONAL COMPLEXITY

TRGP incurs additional training time relative to GPM, with the comparison varying by benchmark, while its additional memory for scaling matrices is described as marginal and controllable.

B.4 COMPUTATIONAL COMPLEXITY

  • TRGP stores additional per-task scaling matrices, whose dimensions match the number of extracted input-subspace bases; memory growth is marginal and controllable, and can be reduced by learning matrices only for convolutional layers.Source: §B.4 · p. 13
  • On CIFAR-100 Split, TRGP’s training time is comparable with HAT and ER Res and lower than OWM and EWC; Table 9 normalizes training time to GPM.Source: §B.4 · p. 14, Table 9 · p. 14
  • For 5-Datasets, TRGP takes around 21% more time than GPM but is much faster than EWC, HAT, A-GEM, and ER Res.Source: Table 9 · p. 14, §B.4 · p. 14
  • For MiniImageNet, TRGP takes around 34% more time than GPM, is comparable with EWC, and is much faster than A-GEM.Source: Table 9 · p. 14, §B.4 · p. 14

B.5 ACCURACY VS LEARNING EPOCHS

Across the learning dynamics shown for CIFAR-100 Split and 5-Dataset, TRGP performs significantly better than GPM on some tasks, especially late in the task sequence. The reported curves use validation accuracy, which varies with the relatively small validation split.

B.5 ACCURACY VS LEARNING EPOCHS

  • The plotted measure is validation accuracy on a split validation dataset used during training; testing accuracy in the tables is evaluated on the test dataset after training.Source: §B.5 · p. 14
  • TRGP performs significantly better than GPM on some tasks, especially late in the sequence, where GPM’s increasingly restrictive optimization space limits new-task performance.Source: §B.5 · p. 14, Figure 9 · p. 15, Figure 10 · p. 15
  • The curves show accuracy versus learning epochs for tasks on CIFAR-100 Split and 5-Dataset.Source: Figure 9 · p. 15, Figure 10 · p. 15

Related papers

Loading 2202.02931v1…