Source-linked AI summary
Continual Learning in Generative Adversarial Nets
Ari Seff, Alex Beatson, Daniel Suo, Han Liu
TL;DR
Standard generative-model training assumes concurrent i.i.d. data, limiting its fit to distributions encountered sequentially. The paper adapts protection of important parameters to GAN training, and reports that an EWC-augmented loss counteracts forgetting across sequentially observed conditional inputs.
Problem
Standard generative-model training assumes i.i.d. data, while sequentially training on new distributions can cause previously learned distributions to be forgotten.
Source: §1 · p. 1Method
The authors augment the generator's objective with a penalty protecting parameters important to previously observed distributions, identified using Fisher information.
Source: §4.1 · pp. 4–5Results
EWC-augmented loss counteracts catastrophic forgetting when GANs are trained sequentially on different sets of conditional inputs.
Source: §6 · p. 8Takeaways & Limitations
The approach applies where conditional-input distributions change over time or a time-of-capture condition is appended to data.
Source: §6 · p. 8Generated samples can retain peripheral noise because the discriminator's discriminative capability is local to the current real and generated distributions.
Source: §5.1.1 · p. 6
Abstract
from arXiv · showhide
Developments in deep generative models have allowed for tractable learning of high-dimensional data distributions. While the employed learning procedures typically assume that training data is drawn i.i.d. from the distribution of interest, it may be desirable to model distinct distributions which are observed sequentially, such as when different classes are encountered over time. Although conditional variations of deep generative models permit multiple distributions to be modeled by a single network in a disentangled fashion, they are susceptible to catastrophic forgetting when the distributions are encountered sequentially. In this paper, we adapt recent work in reducing catastrophic forgetting to the task of training generative adversarial networks on a sequence of distinct distributions, enabling continual generative modeling.
1 Introduction
Deep generative models can model high-dimensional distributions, but standard training assumes data arrives i.i.d. and concurrently. This work adapts methods for reducing catastrophic forgetting to sequential GAN training when earlier data is inaccessible.
1 Introduction
- Standard training assumes data is drawn i.i.d. from the distribution of interest, although real-world data may arrive sequentially or be available only briefly.The paper gives multi-class datasets as an example of concurrent training.Source: §1 · p. 1
- Unlabeled data applications such as camera-fed agents and text modeling may not be able to retain all data, making batched online or streaming approaches preferable.The paper also discusses sequential updates from private data sources without revisiting earlier sources.Source: §1 · pp. 1–2
- Sequentially training only on a newly observed distribution can make a generative model forget previously learned distributions.Training on old and new data together avoids this, but requires storing prior data or regenerating representative samples.Source: §1 · p. 2
- The authors propose scalable sequential GAN training without access to previous datasets, using reduced plasticity for parameters critical to earlier tasks.Their experiments demonstrate continual learning of novel generative tasks as the observed distribution changes.Source: §1 · p. 2
2 Deep generative models
Deep generative models learn distributions through networks that generate or score data. GANs use a generator and discriminator, while conditional GANs incorporate an additional input to guide generation and discrimination.
2 Deep generative models
- Some deep generative models estimate likelihood directly, while others map a fixed latent prior to samples through a trainable one-directional generator.GANs are among the latter models.Source: §2 · p. 2
2.1 Generative adversarial nets
- A GAN trains a generator to fool a discriminator that distinguishes generated samples from real data, using a two-player minimax objective.The generator maps a random noise input to a sample.Source: §2.1 · p. 2
2.2 Conditional generative adversarial nets
- Conditional GANs condition both generator and discriminator on an input y, which may represent a class label or a partially complete sample.The input's form and where it enters the networks are flexible architectural choices that can vary by domain.Source: §2.2 · pp. 2–3
3 Catastrophic forgetting and remedies
Sequential training can degrade performance on earlier tasks, a problem addressed by approaches that preserve prior mappings or protect important parameters. Elastic weight consolidation (EWC) constrains changes to parameters important for previous tasks.
3 Catastrophic forgetting and remedies
- Catastrophic forgetting occurs when training on later tasks decreases performance on earlier tasks.The cited classic example trains a network on a second addition task without access to the first task's data or loss.Source: §3 · p. 3
- Unrestricted weight changes to meet a current task's objective can severely disrupt performance on previous tasks.The phenomenon is reported across computer vision, speech recognition, and natural language processing.Source: §3 · p. 3
- Prior approaches preserve previous input/output mappings or add task-specific network columns, but the latter's parameter count grows quadratically with tasks.The authors assume even linear parameter growth is undesirable.Source: §3 · p. 3
3.1 Protecting critical parameters
- EWC protects parameters critical to previous-task performance with an additional loss restricting their movement; posterior variance is given by the inverse diagonal Fisher, while higher Fisher diagonals indicate greater certainty and less flexibility.Experiments in on permuted MNIST tasks and Atari games validated the approach for supervised classification and reinforcement learning.Source: §3.1 · p. 3
- An alternative estimates parameter importance online across a training trajectory and penalizes movement on later tasks.This is related to saliency-based pruning, but permits low-saliency parameters to remain flexible rather than removing them.Source: §3.1 · pp. 3–4
4 Continual learning with GANs
The paper adapts protection of important parameters to continual GAN training, where naive updates on new data can erase old distributions. Its approach uses conditional inputs and an augmented generator objective to avoid replaying earlier training data.
4 Continual learning with GANs
- Continual GAN training seeks to add new distributions without disrupting old ones, avoiding separate models that cannot share structure and do not scale in limited-capacity settings.Training only on the new distribution causes forgetting because the discriminator focuses on the new data and generator.Source: §4 · p. 4
- Regenerating and retraining on samples from all previous distributions reduces forgetting but costs O(t^2), whereas the proposed approach requires O(t) time and no training-data regeneration.The replay strategy regenerates samples for the previous t−1 distributions at each step.Source: §4 · p. 4
4.1 Augmented generator objective
- Constraining generator weights alone in an unconditional GAN can fail because the discriminator can overcome fixed constraints and still drive the generator toward the new distribution.The proposed objective replaces the generator's portion of the standard minimax objective with an augmented loss.Source: §4.1 · p. 4
- Conditional inputs provide a defined mapping from joint noise and condition to data, allowing constraints when p_data(x|y) stays fixed as p_y(y) changes or distinct conditions are assigned over time.This covers sequential class datasets as well as settings without available conditions or with arbitrary distribution changes.Source: §4.1 · p. 4
- In practice, the authors use −log(D(G(z, y), y)) as the generator cost to provide better gradients early in training.An online parameter-importance method from is also identified as an alternative.Source: §4.1 · pp. 4–5
- The generator's objective penalizes movement of parameters critical to fooling the discriminator, estimated through Fisher information for D(G(z, y), y) with respect to generator parameters.Parameters whose perturbation strongly affects the discriminator's output have higher diagonal Fisher values and are considered salient.Source: §4.1 · pp. 4–5
5 Experimental Results
Across MNIST and SVHN class-conditional experiments, EWC-augmented training retained previously learned classes during sequential updates. The experiments also report limited sensitivity to λ magnitude and a discriminator-related sample-noise limitation.
5 Experimental Results
- The evaluation uses sequential class-conditional GAN training on MNIST and SVHN, with previously observed data unavailable and one generator shared across distributions.The framework does not require the same discriminator across tasks, though these experiments also assume previously trained discriminators are inaccessible.Source: §5 · p. 5
5.1 MLP + MNIST
- For MNIST, both generator and discriminator are two-layer MLPs with 128-unit ReLU hidden layers; the generator outputs 784 sigmoid units and the discriminator a sigmoid probability.The conditional input represents the digit class.Source: §5.1 · p. 5, §5 · p. 5
5.1.1 Overcoming forgetting; 5.2.1 Overcoming forgetting
- EWC preserved earlier MNIST digit samples when adding digit 3 and through digit 6, and retained SVHN digits 0–4 quality while learning digits 5–9.For SVHN, the earlier digits’ visual quality with EWC was similar to the initial model, while standard continued training showed forgetting.Source: §5.1.1 · pp. 5–6, Figure 2 · p. 6, §5.2.1 · p. 7
5.1.1 Overcoming forgetting
- EWC reduced forgetting but left some peripheral noise; the discriminator does not meaningfully classify images unlike real examples or the current generated distribution.The authors report that higher-capacity networks are less susceptible to this limitation.Source: §5.1.1 · p. 6
- With fixed z, training on digit 3 turned a previously sampled digit 1 into a skinny 3 under the standard objective, while EWC showed no apparent deformation.The comparison follows the same latent input during the first epoch of new-class training.Source: §5.1.1 · p. 6, Figure 5 · p. 7
5.2 DCGAN + SVHN
- For SVHN, the conditional DCGAN uses three convolutional layers and one fully connected layer in each network, appending y to z in G and adding a learned linear transform of y in D.The evaluated SVHN classes are digit categories, with y as a one-hot class label.Source: §5 · p. 5, §5.2 · p. 7
5.3 Invariance to λ
- Across MNIST + MLP and SVHN + DCGAN, EWC results were largely invariant to λ; low λ caused early parameter drift and poor samples, while high λ preserved fidelity and converged rapidly.Reported λ values were 1000 for MNIST and 100 for SVHN; SVHN results at λ = 1 and λ = 5000 differed very little in visual quality and not in diversity.Source: §5.3 · pp. 7–8
6 Conclusion
The paper introduces EWC-augmented sequential GAN training to counteract forgetting, and frames learning from data at its point of capture as a step toward modeling the world without storing all data.
6 Conclusion
- Sequential training on different conditional inputs with an EWC-augmented loss counteracts forgetting of previously learned distributions.The conclusion presents this as the paper's experimental finding.Source: §6 · p. 8
- The approach applies where conditional-input distributions change over time or where capture time can be appended as a conditional input.Examples include class labels and partially complete samples.Source: §6 · p. 8
- Learning from data at its point of capture, rather than storing it for future use, is identified as necessary to fully realize deep generative modeling's promise.The work is described as one step toward that goal.Source: §6 · p. 8