Source-linked AI summary
Boundary-Seeking Generative Adversarial Networks
R Devon Hjelm, Athul Paul Jacob, Tong Che, Adam Trischler, Kyunghyun Cho, Yoshua Bengio
TL;DR
GANs require differentiability and therefore do not directly support discrete data, motivating a method that supplies generator learning signals without that requirement. BGAN derives importance weights from the discriminator and applies the resulting policy gradients across discrete and continuous settings. The paper reports effective discrete generation and improved stability in continuous image-generation experiments, while noting that its image-quality advantage is not established over gradient-norm-regularized alternatives.
Problem
GANs normally require the generator-discriminator composition to be fully differentiable, preventing direct training on discrete data.
Method
BGAN uses a discriminator estimating an f-divergence to derive importance weights that provide policy gradients for the generator.
Results
BGAN was quantitatively evaluated on discrete image and language benchmarks and continuous image benchmarks, with stable character generation and improved stability in a difficult continuous setting.
Takeaways & Limitations
The boundary-seeking objective provides a unified training framework for discrete and continuous GANs and demonstrates stability properties across both settings.
Takeaways & Limitations
On CelebA, similar-quality images could be obtained without the boundary-seeking loss when gradient-norm regularization was used, so a necessary image-quality advantage was not established.
Abstract
from arXiv · showhide
Generative adversarial networks (GANs) are a learning framework that rely on training a discriminator to estimate a measure of difference between a target and generated distributions. GANs, as normally formulated, rely on the generated samples being completely differentiable w.r.t. the generative parameters, and thus do not work for discrete data. We introduce a method for training GANs with discrete data that uses the estimated difference measure from the discriminator to compute importance weights for generated samples, thus providing a policy gradient for training the generator. The importance weights have a strong connection to the decision boundary of the discriminator, and we call our method boundary-seeking GANs (BGANs). We demonstrate the effectiveness of the proposed algorithm with discrete image and character-based natural language generation. In addition, the boundary-seeking objective extends to continuous data, which can be used to improve stability of training, and we demonstrate this on Celeba, Large-scale Scene Understanding (LSUN) bedrooms, and Imagenet without conditioning.
1 INTRODUCTION
GANs can generate realistic samples from high-dimensional continuous data, but their fully differentiable formulation cannot directly handle discrete variables. BGAN addresses this gap by using discriminator-derived importance weights to provide generator training signals for discrete data.
- GANs use a generator and discriminator with opposing objectives, training the generator from a learned discriminator signal without explicitly modeling probability density.
- GANs have generated diverse, realistic samples from high-dimensional continuous data, but require the generator-discriminator composition to be fully differentiable.
- With discrete outputs such as step-function values, back-propagation fails because the derivative is zero almost everywhere.
- Credit assignment for computational graphs containing discrete operations remains difficult, and prior approximate solutions had not been demonstrated with GANs.
- BGANs use a discriminator optimized for an f-divergence to construct importance weights that provide policy gradients for training discrete-data generators.
- The proposed approach was evaluated across f-divergences, image and language benchmarks, and discrete settings against WGAN-GP.
2 BOUNDARY-SEEKING GANS
BGANs address the failure of ordinary GAN back-propagation on discrete data by converting discriminator-based divergence estimates into importance weights and policy gradients. The same framework motivates a continuous generator objective intended to improve GAN stability.
- Target-distribution estimation: The discriminator’s variational lower-bound defines an importance-weight estimator for the target density, with lower bias when the bound is tighter.The estimator uses a positive activation and a partition function, while practical discriminators may be sub-optimal.
- Problem statement: GANs require differentiable generator-discriminator composition, but discrete operations such as step functions yield zero gradients almost everywhere.This makes direct back-propagation unsuitable for discrete image, word-based, and character-based representations.
- Boundary-seeking GANs: BGANs train generators on discrete data by using a discriminator optimized for an f-divergence to construct importance weights and policy gradients.This extends adversarial training beyond settings where generated samples are fully differentiable with respect to generator parameters.
- Boundary-seeking GANs: The resulting policy gradient connects BGAN training to importance sampling and REINFORCE, with normalized importance weights serving as the reward in a special case.The reward approaches the likelihood ratio when the discriminator is optimal in the non-parametric limit.
- Continuous variables and stability: For continuous variables, BGANs replace direct variational-bound minimization with a squared log-weight objective designed to improve generator-training stability.The objective is based on matching the generated density to the estimated target when the importance weight equals 1.
3 RELATED WORK AND DISCUSSION
BGAN relates to prior likelihood-ratio, importance-sampling, and REINFORCE approaches while directly addressing discrete GAN training. The discussion also contrasts its f-divergence basis with IPM-based stabilization methods and notes gradient penalties used to improve training stability.
- Likelihood ratios and discrete GANs: BGAN is presented as the first approach to successfully formulate and apply discriminator-based likelihood-ratio estimation to discrete GAN training.Its theoretical foundation draws from f-GAN, while prior work established related likelihood-ratio and policy-gradient connections.
- Importance sampling: BGAN’s importance sampling and REINFORCE formulations are analogous to re-weighted wake-sleep and neural variational inference methods.The comparison links these methods through their use of policy gradients involving likelihood ratios.
- Discrete GANs for language: Prior language-model GAN methods use REINFORCE and focus primarily on improving credit assignment, while remaining compatible with BGAN’s policy gradients.These methods effectively use the sigmoid of an estimated log-likelihood ratio.
- Discrete GANs for language: Several language-data GAN approaches avoid direct discrete optimization by reformulating the problem in continuous latent spaces or ignoring discrete sampling.The passage distinguishes these approaches from methods that directly optimize a discrete generator.
- Stability and regularization: IPM-based GANs such as WGANs improve training stability but require restricting discriminator functions and enforcing constraints through regularization.The cited regularizers include weight clipping, gradient penalties over interpolations, and a gradient norm penalty formulated using f-divergences, which the authors report improves stability in nearly all results.
4 DISCRETE VARIABLES: EXPERIMENTS AND RESULTS
The experiments evaluate BGAN on discrete classification, image, and character-level language generation. BGAN is stable across these settings, while comparisons show weaker WGAN performance and lower-quality text than recurrent baselines.
- Adversarial classification: BGAN importance sampling and REINFORCE were evaluated on CIFAR-10 adversarial classification alongside cross-entropy and WGAN baselines.The study trained the models for 200 epochs and reported test accuracy.
- Adversarial classification: 72.3% error was achieved by WGAN on CIFAR-10 after 600 epochs, while gradient-penalty training failed completely.BGAN performed similarly to the cross-entropy baseline, and REINFORCE was only slightly worse.
- Discrete image generation: BGAN training on discrete MNIST was stable and produced realistic, highly variable handwritten digits.The figure shows Bernoulli centers of the generator conditional distribution.
- Discrete image generation: After 50 epochs, BGAN generated reasonably realistic and diverse 16-color CelebA images resembling the quantized dataset.CelebA images were downsampled to 32 × 32 and quantized using a 16-color palette.
- Natural language generation: BGAN yielded stable, reliably good character-level generation, but its text quality was poor compared with recurrent neural network methods.The result was obtained without continuous relaxation, pretraining, or an auxiliary supervised loss.
5 CONTINUOUS VARIABLES: EXPERIMENTS AND RESULTS
Continuous BGAN produces high-quality samples on CelebA, LSUN, and ImageNet, while showing resilience to generator over-training and limited disruption of adversarial learning.
- 5.1 GENERATION BENCHMARKS: BGAN produces very good samples on CelebA, LSUN, and the full 1000-label ImageNet dataset without conditioning.The models use deep ResNet architectures with gradient norm regularization.
- 5.1 GENERATION BENCHMARKS: ImageNet samples remain high quality despite training on the full dataset without label conditioning.
- 5.1 GENERATION BENCHMARKS: On CelebA, similar image quality can be achieved without the boundary-seeking loss when gradient norm regularization is used.The experiment therefore confirms high-dimensional applicability rather than superiority over the variational lower-bound in every setting.
- 5.2 STABILITY OF CONTINUOUS BGAN: BGAN remains resilient when the generator receives five updates per discriminator update, learning reasonable samples in fewer epochs than alternative objectives.Over-optimizing the generator otherwise significantly degrades sample quality.
- 5.2 STABILITY OF CONTINUOUS BGAN: Following the BGAN objective at the pixel level causes the least image-quality degradation, suggesting reduced disruption of adversarial learning.
6 CONCLUSION
The conclusion presents boundary-seeking as a unified GAN framework for discrete and continuous variables, with quantitative validation and demonstrated stability properties.
- Boundary-seeking reformulates the generator objective into a unified learning algorithm for discrete and continuous GAN variables.
- Experiments quantitatively verify BGAN across discrete and continuous variables and demonstrate properties of training stability.
7 APPENDIX
The appendix compares BGAN with WGAN-GP on discrete MNIST, validates its theoretical policy gradient, and evaluates a variance-reduction method for estimating the normalization term.
- 7.1 COMPARISON OF DISCRETE METHODS: The discrete-MNIST evaluation uses 60,000 training examples and 60,000 generated samples averaged over 12 batches.Test-set estimates were generally close to training-set estimates and did not diverge.
- 7.1 COMPARISON OF DISCRETE METHODS: BGAN achieves consistently lower estimated distribution differences than WGAN-GP, including when evaluated with Wasserstein distance.Visual inspection found the samples indistinguishable in quality, while softmax-based WGAN-GP evaluation produced a much lower Wasserstein distance.
- Theorem 2 states that zero expected conditional KL-divergence implies zero marginal KL-divergence between the proposal target and generator distributions.
- 7.2 THEORETICAL AND EMPIRICAL VALIDATION OF THE VARIANCE REDUCTION METHOD: The variance-reducing method from Equation 10 clearly improves over Monte-Carlo estimation of β, with nearly identical wall-clock times at equal total sample counts.Both methods improve as the number of conditional samples increases.