PaperlayerView exact revision on arXiv

Source-linked AI summary

Continual learning with hypernetworks

Johannes von Oswald, Christian Henning, Benjamin F. Grewe, João Sacramento

arXiv:1906.00695v4cs.LGcs.AIstat.ML

TL;DR

Continual learning must learn sequential tasks without catastrophically forgetting earlier ones. The paper uses task-conditioned hypernetworks to generate task-specific target weights and preserve them through output regularization. It reports state-of-the-art benchmark performance, long memory lifetimes in a compressive regime, and transfer to future tasks.

  • Problem

    Continual learning requires sequentially learning tasks while retaining prior performance and supporting backward and forward transfer.

  • Method

    Task-conditioned hypernetworks map task embeddings to target-model weights, while output regularization preserves prior task-specific configurations without explicitly storing datasets.

  • Results

    The approach achieves state-of-the-art results on standard benchmarks and suggests long memory lifetimes and forward information transfer, including in a compressive regime.

  • Takeaways & Limitations

    Task-conditioned hypernetworks provide a scalable, generic continual-learning framework usable alone or with generative replay.

  • Takeaways & Limitations

    The output regularizer protects previous solutions, allowing only weak backward transfer and leaving refinement of past memories for future work.

Abstract

from arXiv · show

Artificial neural networks suffer from catastrophic forgetting when they are sequentially trained on multiple tasks. To overcome this problem, we present a novel approach based on task-conditioned hypernetworks, i.e., networks that generate the weights of a target model based on task identity. Continual learning (CL) is less difficult for this class of models thanks to a simple key feature: instead of recalling the input-output relations of all previously seen data, task-conditioned hypernetworks only require rehearsing task-specific weight realizations, which can be maintained in memory using a simple regularizer. Besides achieving state-of-the-art performance on standard CL benchmarks, additional experiments on long task sequences reveal that task-conditioned hypernetworks display a very large capacity to retain previous memories. Notably, such long memory lifetimes are achieved in a compressive regime, when the number of trainable hypernetwork weights is comparable or smaller than target network size. We provide insight into the structure of low-dimensional task embedding spaces (the input space of the hypernetwork) and show that task-conditioned hypernetworks demonstrate transfer learning. Finally, forward information transfer is further supported by empirical results on a challenging CL benchmark based on the CIFAR-10/100 image datasets.

1 INTRODUCTION

Continual learning presents tasks sequentially, requiring models to retain prior performance while learning new tasks and potentially transferring knowledge. The paper addresses this challenge by preserving task-specific parameter configurations instead of replaying all prior input-output data.

  • Continual learning presents tasks sequentially and seeks retention, positive backward transfer, and positive forward transfer across tasks.
  • The proposed task-conditioned hypernetwork maps task embeddings to target-model weights, preserving prior solutions at the metamodel level.
  • Replacing rehearsal of prior datasets with stored task-specific parameter configurations yields a low-dimensional memory representation and substantial memory savings.
  • The approach reports no catastrophic forgetting on standard benchmarks and minimal performance loss over very long task sequences.
  • Task-conditioned hypernetworks exploit task similarities for forward information transfer and also extend to generative replay methods.

2 MODEL

The model uses task-conditioned hypernetworks to generate target weights from task and chunk embeddings, while output regularization protects previously learned configurations. Chunking supports compression, and the framework also accommodates unknown task identities and hypernetwork-protected replay.

  • Task-conditioned hypernetworks: Hypernetworks generate target-model weights from learned task embeddings rather than directly learning the target parameters.The metamodel output is the target network's parameter set.
  • Memory preservation: Rehearsal-based output regularization requires storing and iterating over previous data and may fail to constrain function behavior in relevant regions.
  • Memory preservation: Output regularization preserves prior task solutions by constraining hypernetwork outputs, avoiding explicit storage of previous task data.A two-step optimization first proposes a task-loss update, then computes the actual change under a total loss with a memory-preserving term.
  • Chunked hypernetworks: Chunked hypernetworks generate target weights iteratively, enabling reusable smaller hypernetworks and a compressive regime with fewer learned than target parameters.
  • Chunked hypernetworks: Chunk embeddings distinguish target-network partitions when a shared hypernetwork is reused across chunks.The full target parameter set is formed by iterating over chunk embeddings while keeping the task embedding fixed.
  • Unknown task identity and replay: The framework supports task inference from predictive uncertainty when task identity is unavailable and can protect replay generators with the same output-regularization principle.The replay extension combines synthetic replay with task-conditional metamodelling to reduce forgetting.

3 RESULTS

Across standard and extended continual-learning benchmarks, task-conditioned hypernetworks retain previous tasks while supporting transfer and model compression. Results span toy regression, MNIST variants, and CIFAR-10/100 experiments.

  • Nonlinear regression: On nonlinear regression, output regularization protects earlier polynomial solutions, matching an offline learner, whereas fine-tuning forgets past tasks.The task sequence contains noisy samples from polynomials of increasing degree.
  • Compression: Hypernetworks maintain good accuracy even when target-model parameters exceed their own, with performance remaining approximately constant across a range of compression ratios below one.The PermutedMNIST-10 compression experiment reports final and during-task task-averaged test accuracy across five random seeds.
  • Permuted MNIST: For Permuted MNIST-10, hypernetwork-based models are the best performers across all three continual-learning scenarios.HNET+ENT infers task identity from predictive-distribution entropy, while HNET+TIR and HNET+R use hypernetwork-protected replay variants.
  • Permuted MNIST: At T = 100 Permuted MNIST tasks, task-conditioned hypernetworks show minimal memory decay and high-performance solutions in a compressive regime.SI and DGR+distill degrade gracefully, while online EWC remains rigid and cannot reach high accuracy; HNET+TIR strongly outperforms related work in CL2/3.
  • Split MNIST: On Split MNIST, HNET+R improves DGR+distill on CL2 and CL3, while HNET+ENT outperforms EWC and SI without generative modeling.The benchmark uses five sequential binary classification tasks formed by pairing digits, creating task overlap.
  • Task embeddings: Two-dimensional task embeddings support high performance with virtually no forgetting, transfer across overlapping tasks, and moderate above-chance performance for earlier embeddings.The first task is solved across a volume containing later task embeddings, while the last task occupies a finite high-performance region.

4 DISCUSSION

The discussion contrasts Bayesian and task-conditioned approaches to continual learning, highlighting hypernetworks’ retention and transfer properties alongside limitations and biological interpretation.

  • On split CIFAR-10/100, hypernetwork-protected ResNet-32 shows virtually no forgetting, with final averaged performance matching immediate performance.
  • Bayesian continual learning can produce stiff models because approximate inference seeks a compromise within the first task’s mode.
  • Task-conditioning methods commonly use masks over modules, neurons, or weights, but work best with large networks and add storage overhead.
  • The output regularizer protects previously learned solutions, so only weak backward information transfer can occur; improving backward transfer remains future work.
  • A literal cortical implementation would require dynamically changing an entire network’s connectivity, although lower-dimensional modulatory signals offer a relaxed interpretation.

5 CONCLUSION

The paper concludes that task-conditioned hypernetworks provide a scalable, generic continual-learning model that retains long-lived memories and transfers information to future tasks.

  • Task-conditioned hypernetworks parameterize target functions with compressed task embeddings and preserve prior configurations through an output regularizer.
  • The approach applies standalone or with generative replay, achieving state-of-the-art benchmark results, long memory lifetimes, and forward information transfer.

A TASK-CONDITIONED HYPERNETWORKS: MODEL SUMMARY

The model uses a task-conditioned hypernetwork to generate target-network weights from task embeddings, while regularization preserves earlier task-specific outputs.

  • The hypernetwork produces target parameters Θtrgt from a task embedding and its own parameters.
  • Learning updates hypernetwork parameters and, when needed, a set of task embeddings for compression or oversized hypernetworks.
  • An output regularizer penalizes changes in target parameters generated for previously learned tasks.
  • Continual retention stores one embedding per task, so memory grows linearly with the number of tasks.
  • Each task embedding is a single low-dimensional vector, making the linear storage requirement essentially negligible in practice.
  • A frozen pre-task hypernetwork snapshot is retained to evaluate the output regularizer.

B ADDITIONAL DETAILS ON HYPERNETWORK-PROTECTED REPLAY MODELS

The replay extensions protect generative models with hypernetworks and use synthetic data for task inference and continual protection across VAE- and GAN-based setups.

  • Variational autoencoders: VAE replay generates past-task samples by varying task indicators and drawing latent samples, then mixes them with current-task data.
  • Variational autoencoders: In HNET+TIR and HNET+R, a task-conditioned hypernetwork parameterizes the VAE decoder and combines with output regularization to retain memories.
  • Variational autoencoders: The replay subsystem is optimized independently with a regularized VAE loss balancing reconstruction and prior-matching penalties.
  • Task inference network: The task-inference classifier uses a growing softmax over tasks and replay data from the protected VAE to avoid catastrophic forgetting.
  • Task inference network: Synthetic data is necessary for task inference because its cross-entropy objective requires at least two ground-truth classes, and replay can be generated online.
  • Replay model setups: Figure A1 depicts protected VAE, GAN, and task-inference-classifier configurations used across the replay experiments.
  • Hypernetwork-protected GANs: For GAN replay, the hypernetwork maps a condition to the generator’s full parameters, enabling one-condition-at-a-time conditional GAN training.
  • Hypernetwork-protected GANs: The class-incremental MNIST GAN used a fully connected chunked hypernetwork with two hidden layers of size 25 and an output size of 75,000.

C ADDITIONAL EXPERIMENTAL DETAILS

The appendix details experimental configurations for nonlinear regression, MNIST-based continual learning, replay models, and hyperparameter searches. It also records architectural choices, parameter counts, and limitations of the search procedure.

  • Training procedure: Previous task embeddings are kept fixed while only the current task embedding is learned, although adapting previous embeddings could improve performance.
  • Nonlinear regression: The nonlinear regression experiment uses noisy samples from sequential polynomial tasks and a two-hidden-layer target network with 141 weights.The target and hypernetwork each use two fully connected hidden layers of 10 neurons; task embeddings have dimension 2.
  • Nonlinear regression: Each nonlinear regression task is trained for 4000 iterations with Adam, a 0.01 learning rate, and batch size 32.
  • MNIST experiments: The MNIST experiments use fully connected target networks and match the reviewed benchmark setup in training iterations, learning rates, optimizer, and comparable parameter counts.The appendix states that hypernetworks use the same or fewer weights than the target network where applicable.
  • MNIST experiments: For the VAE in PermutedMNIST-10, the hypernetwork has 2,211,907 weights versus 2,227,024 in the corresponding target network.The configuration uses task embeddings of size 24, chunk embeddings of size 8, and βoutput = 0.05.
  • Hyperparameter selection: The appendix varies hypernetwork depth, output size, transfer functions, and learning rates, but explores only a random subset of possible hyperparameter combinations.Promising configurations were manually adjusted to approximately match target-network parameter counts; architecture strongly influences performance.

D ADDITIONAL EXPERIMENTS AND NOTES

Additional experiments examine benchmark replication, replay and task inference, compression, regularization, and long task sequences. They report competitive accuracy, robust regularization behavior, efficient stochastic regularization, and unresolved task-boundary requirements.

  • Split CIFAR-10/100: 71.29% ± 0.32% task-averaged accuracy is comparable to synaptic intelligence at 73.85% on split CIFAR-10/100.This result uses the shallower model reproduced from Zenke et al. (2017).
  • Regularization: Performance remains comparable across a wide range of βoutput values, while overly weak regularization permits forgetting and overly strong regularization lowers baseline accuracy.
  • PermutedMNIST: 93.91 ± 0.04 average accuracy is achieved on PermutedMNIST-10, increasing to 94.24 ± 0.03 with soft targets.
  • Efficient regularization: 94.19 ± 0.16 final average accuracy is obtained on PermutedMNIST-250 while estimating the regularizer from at most 32 random previous tasks per iteration.This stochastic subset reduces the regularizer’s runtime dependence on the total number of tasks.
  • Open limitation: The method does not address learning without task boundaries, requiring a future switching mechanism to detect task changes and save task embeddings.

E UNIVERSAL FUNCTION APPROXIMATION WITH CHUNKED NEURAL

The appendix argues that chunked hypernetworks retain universal approximation capability under a discrete chunk-index construction. It proves approximation through continuity, universal approximation, and the triangle inequality, while noting that useful compression bounds remain unavailable.

  • Proposition: A continuous function on a compact domain can be approximated by a chunked neural network with discrete chunk set C and output dimension r.The proposition specifies a compact input subset K and a target dimension n = r · NC.
  • Proof strategy: The proof first approximates the target function with a neural network and then constructs a chunked network that approximates it over every x in K and chunk c in C.
  • Proof strategy: Continuity on the product of the Euclidean domain and discrete chunk set allows the universal approximation theorem to be applied to the chunked construction.
  • Proof strategy: The proof concludes by combining approximation errors with the triangle inequality.
  • Limitations: The result leaves the number of chunks, chunk dimension, and embedding dimension unspecified, and provides no constructive bound on performance or compression rate.These aspects are evaluated empirically rather than derived theoretically.
Loading 1906.00695v4…