Continual Learning with Node-Importance based Adaptive Group Sparse Regularization

Sangwon Jung; Hongjoon Ahn; Sungmin Cha; Taesup Moon

Source-linked AI summary

Continual Learning with Node-Importance based Adaptive Group Sparse Regularization

Sangwon Jung, Hongjoon Ahn, Sungmin Cha, Taesup Moon

arXiv:2003.13726v4Mar 30, 2020cs.LGstat.ML

TL;DR

Continual learning must balance retaining prior-task knowledge with adapting to sequentially arriving tasks. AGS-CL uses adaptive node-wise group-sparsity penalties, proximal optimization, and reinitialization of unimportant nodes to control this trade-off. It reportedly outperforms strong baselines across supervised and reinforcement-learning benchmarks while using far less additional memory.

Problem
Continual learning must balance stability against plasticity as training data arrives sequentially in task groups.
Method
AGS-CL selectively applies two node-wise group-sparsity penalties using adaptive importance parameters, with proximal gradient descent enforcing sparsity and freezing.
Results
AGS-CL significantly outperforms strong state-of-the-art baselines across tested supervised and reinforcement-learning benchmarks and uses orders of magnitude less additional memory.
Takeaways & Limitations
AGS-CL provides a fixed-capacity continual-learning approach that controls plasticity and stability while reducing regularization-parameter storage.
Takeaways & Limitations
The authors identify depleted network capacity as a future-work issue.

Abstract

from arXiv · show

We propose a novel regularization-based continual learning method, dubbed as Adaptive Group Sparsity based Continual Learning (AGS-CL), using two group sparsity-based penalties. Our method selectively employs the two penalties when learning each node based its the importance, which is adaptively updated after learning each new task. By utilizing the proximal gradient descent method for learning, the exact sparsity and freezing of the model is guaranteed, and thus, the learner can explicitly control the model capacity as the learning continues. Furthermore, as a critical detail, we re-initialize the weights associated with unimportant nodes after learning each task in order to prevent the negative transfer that causes the catastrophic forgetting and facilitate efficient learning of new tasks. Throughout the extensive experimental results, we show that our AGS-CL uses much less additional memory space for storing the regularization parameters, and it significantly outperforms several state-of-the-art baselines on representative continual learning benchmarks for both supervised and reinforcement learning tasks.

1 Introduction

Continual learning must balance stability against plasticity as tasks arrive sequentially, while regularization-based methods seek to use fixed-capacity networks efficiently. AGS-CL addresses this trade-off with node-wise adaptive group sparsity and reports strong performance, low memory overhead, and broad benchmark coverage.

Continual learning presents a stability-plasticity dilemma: excessive stability harms forward transfer, whereas excessive plasticity causes catastrophic forgetting.
AGS-CL uses two node-wise group sparsity penalties to adaptively control plasticity and stability during sequential task learning.One penalty learns important nodes while maintaining structured sparsity; the other freezes incoming weights to important nodes against drift.
Proximal gradient descent provides exact sparsity and node freezing without an additional threshold to tune.
AGS-CL significantly outperforms strong state-of-the-art baselines across all tested supervised and reinforcement-learning benchmarks.The experiments also analyze stability-plasticity using metrics beyond average accuracy.
AGS-CL uses orders of magnitude less additional memory than baselines by maintaining only node-wise regularization parameters.The authors connect this compact memory usage to applicability on larger networks.
Unlike prior group-sparsity continual learning work, AGS-CL targets fixed capacity and avoids many hyperparameters and repeated retraining steps.The cited prior method also lacked group-level model freezing.

2 Motivation

The paper attributes catastrophic forgetting to model drift and negative transfer affecting information flowing through important nodes. AGS-CL addresses both by freezing important nodes, pruning unimportant-node outputs, and reinitializing their incoming weights.

Model drift changes the incoming weights of an important node, altering its representation and potentially degrading performance on an earlier task.
Negative transfer occurs when a previously unimportant lower-layer node changes and becomes important for a new task, interfering with earlier-task representations.
The proposed remedy freezes incoming weights of important nodes to prevent model drift and fixes outgoing weights of unimportant nodes at zero to eliminate transfer.
AGS-CL automatically identifies important and unimportant nodes, applies two selective group-sparsity penalties, and reinitializes unimportant-node incoming weights to preserve plasticity.

3 Adaptive Group Sparsity based Continual Learning (AGS-CL)

AGS-CL adaptively balances plasticity and stability with node-wise group-sparsity penalties selected by learned node importance. Proximal gradient updates provide exact sparsification and freezing, while re-initialization limits negative transfer and preserves capacity for future tasks.

3.1 Notations: AGS-CL represents each convolutional filter or channel as a node with an incoming weight vector and tracks node importance across tasks.The importance parameter is nonnegative; larger values indicate nodes learned as important for previous tasks, while zero denotes nodes not previously important.
3.2 Loss function: The loss combines task-specific training with two selectively applied group-sparsity penalties governed by node importance.Group Lasso activates previously unimportant nodes for new tasks, while the deviation penalty stabilizes sufficiently important nodes around their previous parameters.
3.3 Learning with proximal gradient descent: Proximal gradient descent yields exact sparsification for unimportant nodes and exact freezing for important nodes without additional thresholds or heuristics.The proximal operator acts independently on each node parameter group, and the resulting updates are designed to converge to a local minimum with an appropriate learning rate.
3.4 Updating Ωt−1: Average ReLU activation is used to update node importance, and AOPC comparisons show that pruning nodes in descending importance differs substantially from random or ascending-order pruning.The reported AOPC gaps support average ReLU activation as the importance measure used by AGS-CL.
3.4 Updating Ωt−1: After each task, AGS-CL zeros outgoing weights from unimportant nodes and randomly re-initializes their incoming weights with probability ρ.The first operation prevents future representation changes from affecting previously important upper-layer nodes, while the second restores learnability; ρ ≤ 0.5 typically gives a good sparsity-capacity trade-off.
3.4 Updating Ωt−1: The method’s importance-driven operations are also presented as a mechanism for mitigating negative transfer while retaining future-task plasticity.The order of re-initialization matters: outgoing connections are nullified before selected incoming weights are reinitialized.

4 Experimental Results

Across supervised vision and Atari continual-learning benchmarks, AGS-CL consistently outperforms strong baselines while using compact regularization state and adaptively managing model capacity. Its re-initialization steps improve the plasticity–stability trade-off and preserve capacity for future tasks.

Supervised learning: AGS-CL consistently dominates other baselines across the evaluated vision datasets throughout most tasks.Experiments cover CIFAR-100, CIFAR-10/100, Omniglot, CUB200, and a sequence of eight visual datasets.
Memory efficiency: More than four orders of magnitude separate AGS-CL from other methods in additional memory on CUB200.The comparison concerns memory required to store regularization parameters for a large-scale AlexNet model.
Capacity analysis: Sparsity decreases while used capacity increases as AGS-CL learns successive CIFAR-100 tasks.The trends indicate that active capacity is consumed while more nodes become frozen to protect previous-task information.
Capacity analysis: Without re-initialization, sparsity stops decreasing beyond a threshold, preventing AGS-CL from using the network’s full capacity.The analysis evaluates the re-initialization steps alongside sparsity and used-capacity trajectories.
Plasticity and stability: AGS-CL shows better plasticity–stability trade-offs than MAS and EWC, with stability near 1 and higher plasticity in most evaluated cases.The P–S analysis defines plasticity as forward transfer and stability as not forgetting; re-initialization components separately improve these dimensions.
Reinforcement learning: AGS-CL achieves approximately 3× the final accumulated reward of EWC and MAS, and approximately 5× that of fine-tuning, across eight Atari tasks.Rewards are normalized by the maximum reward obtained by fine-tuning for each task.

5 Concluding Remark

AGS-CL combines node-wise importance regularization, group-sparsity-based optimization, and re-initialization to support continual learning. It performed strongly across benchmarks while using far fewer regularization parameters, but capacity depletion remains a future-work boundary.

AGS-CL combines node-wise importance regularization, group-sparsity-based loss design, proximal gradient descent, and weight re-initialization.These components are presented as the method's central design elements.
AGS-CL dominated other state-of-the-art methods across various benchmark datasets with orders of magnitude fewer regularization parameters.
The method also produced promising results on pure continual reinforcement learning with Atari tasks.
When network capacity depletes, extending AGS-CL by adding free nodes is identified as future work.

6 Broader Impact

The paper addresses a general continual learning problem without a particular application focus. Its potential societal impact is reduced computing-resource use during repeated neural-network retraining.

The work tackles a general continual learning problem and does not target a particular application.
The potential societal impact is saving computing resources used for repetitive retraining of computationally intensive neural networks.

Supplementary Materials for Continual Learning with Node-Importance based

The supplementary materials identify the paper, its authors, institutional affiliations, and publication metadata.

The supplementary materials are titled “Adaptive Group Sparse Regularization.”
The paper is authored by Sangwon Jung, Hongjoon Ahn, Sungmin Cha, and Taesup Moon.
The manuscript is identified as arXiv:2003.13726v4, dated 29 May 2021.
The authors are affiliated with the Departments of Electrical and Computer Engineering and Artificial Intelligence at Sungkyunkwan University.

1 Proof of Lemma 1

The proof characterizes the proximal-gradient minimizer through convex optimality conditions and combines intermediate relations to establish the lemma.

The proximal operator prox_αf(v) is defined through minimization of a convex function, with θ* denoting its minimizer.
Because the objective is convex, the minimizer satisfies a zero-subgradient optimality condition.
When θ* ≠ θ0, the proof derives an additional relation from equations (2) and (3).
The lemma follows by combining the relations established in equations (4) and (6).

2 Additional ablation studies

The ablations examine how ρ and proximal gradient descent affect AGS-CL’s accuracy, sparsity, capacity use, and stability-plasticity trade-off. They also compare AGS-CL’s plasticity-stability trade-offs with MAS and EWC.

2.1 Ablation study of ρ: ρ = 0.1 limits early capacity growth, whereas ρ = 0.5 consumes capacity early and makes the network too stable later.
2.2 Ablation study of PGD: The proximal gradient descent update yields higher average accuracy than training without PGD.
2.2 Ablation study of PGD: Without PGD, sparsity decreases faster because weights associated with unimportant nodes are not exactly zero and continue learning across tasks.
2.3 Stability-plasticity analysis: AGS-CL has better plasticity-stability trade-offs than EWC and MAS in the CIFAR-100 comparison.

3 Implementation details

The supervised experiments use dataset-specific architectures, optimizers, schedules, and batch sizes, with hyperparameters extensively searched for fair comparison. Additional figures and tables document the CIFAR-100 stability-plasticity analysis and implementation settings.

Evaluation reporting: Figure 3 reports plasticity and stability for CIFAR-100.
Training settings: CIFAR-100, CIFAR-10/100, and Omniglot use Adam with batch size 256 for 100 epochs, while CUB200 uses SGD with momentum and batch size 64 for 40 epochs.
Hyperparameters: The supervised experiments extensively search each method’s best hyperparameters to make comparisons as fair as possible.
Network architectures: Tables 2 and 3 specify network architectures for CIFAR-100, CIFAR-10/100, and Omniglot, while CUB200 uses the official PyTorch AlexNet architecture.
Evaluation reporting: Table 4 reports average accuracy and standard deviation from five independent runs with different random seeds.

3.2 Reinforcement learning

The reinforcement-learning experiments evaluate Atari task sequences using enlarged convolutional architectures and PPO under shared baseline settings. Across detailed reward curves and an alternative μ setting, AGS-CL shows strong stability and plasticity.

Implementation: Atari training uses one architecture with four times more filters than the original to provide capacity for eight tasks.
Implementation: All Atari methods use PPO, are evaluated every 40 updates, and are trained with Adam at an initial learning rate of 0.0003.
Hyperparameters: The reinforcement-learning figures use λ settings of {1, 2.5, 10} × 10^4 for EWC, {1, 10} for MAS, and {1, 10} × 10^2 for AGS-CL.
Results with μ = 0.1: AGS-CL significantly outperforms EWC on Tasks 1, 2, and 7, and exceeds Fine-tuning on Task 7, indicating higher plasticity there.
Results with μ = 0.1: AGS-CL has higher stability than the other baselines for all tested λ values.
Results with μ = 0.125: With μ = 0.125, AGS-CL achieves the highest reward when λ is appropriately selected and retains similar advantages across tasks.