Source-linked AI summary
Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learning
Dilin Wang, Qiang Liu
TL;DR
The paper addresses inference when target distributions are complex and neural samplers may have intractable output densities. It trains samplers by amortizing SVGD and applies the approach to adversarial MLE for deep energy models, reporting realistic-looking images competitive with GAN results.
Problem
Complex targets and black-box samplers make inference challenging when the sampler density is unavailable, while standard variational inference requires that density.
Source: §1 · pp. 1–2Method
The method iteratively adjusts neural-network parameters so sampler outputs follow an SVGD direction that decreases KL divergence, and applies this approach to energy-model MLE.
Source: §1 · p. 2Results
SteinGAN produces realistic-looking images competitive with the state-of-the-art results produced by GANs.
Source: §1 · p. 2Takeaways & Limitations
The method provides a neural-sampler approach to inference for targets specified by unnormalized densities and to amortized MLE for deep energy models.
Source: §1 · p. 2
Abstract
from arXiv · showhide
We propose a simple algorithm to train stochastic neural networks to draw samples from given target distributions for probabilistic inference. Our method is based on iteratively adjusting the neural network parameters so that the output changes along a Stein variational gradient that maximumly decreases the KL divergence with the target distribution. Our method works for any target distribution specified by their unnormalized density function, and can train any black-box architectures that are differentiable in terms of the parameters we want to adapt. As an application of our method, we propose an amortized MLE algorithm for training deep energy model, where a neural sampler is adaptively trained to approximate the likelihood function. Our method mimics an adversarial game between the deep energy model and the neural sampler, and obtains realistic-looking images competitive with the state-of-the-art results.
1 INTRODUCTION
The introduction frames inference as a need to adapt sampling methods for complex distributions, especially when inference must be repeated across related tasks. It proposes amortized SVGD, including SteinGAN for energy-model MLE, without requiring the sampler’s density to be tractable.
1 INTRODUCTION
- Inference methods are often hand-designed, limiting automatic optimization and making repeated inference across many similarly structured distributions inefficient.The paper motivates adaptive systems that can improve computational efficiency and generalize to related tasks.Source: §1 · p. 1
- The target may be specified only up to its normalization constant, while a black-box sampler exposes outputs and derivatives but not its output density.The goal is to tune the sampler so its output distribution matches the target.Source: §1 · p. 1
- Traditional variational inference requires evaluating the proposal density or its derivative, forcing tractable proposal families and substantial design work.The paper calls methods that avoid requiring a tractable qη(x) “wild variational inference.”Source: §1 · p. 2
- The proposed method adjusts neural-network parameters along an SVGD direction that decreases KL divergence without explicitly calculating the sampler density.The paper presents this as amortizing SVGD for adaptable inference.Source: §1 · p. 2
- SteinGAN applies the sampler to energy-model MLE as an adversarial game and produces realistic-looking images competitive with GAN results.The neural sampler approximates the likelihood function for the deep energy model.Source: §1 · p. 2
- Prior neural-sampler MLE work used a batch-normalization diversity heuristic, whereas this method uses SVGD’s repulsive term; other cited energy-model approaches use MCMC-MLE or contrastive divergence.The paper describes these alternatives as having difficulty generating realistic-looking images from scratch.Source: §1 · p. 3
2 STEIN VARIATIONAL GRADIENT DESCENT (SVGD)
SVGD iteratively moves particles toward a target distribution using a direction chosen to decrease KL divergence. Its update combines target-density gradients with a kernel-based repulsion that promotes diversity.
2 STEIN VARIATIONAL GRADIENT DESCENT (SVGD)
- SVGD updates particles by a direction chosen to maximally decrease KL divergence between their distribution and the target.The perturbation directions are optimized within the unit ball of a vector-valued RKHS associated with a positive-definite kernel.Source: §2 · p. 3
- Stein’s identity gives zero expectation for the Stein operator when the distributions match, assuming p(x)φ(x) vanishes at the integration boundary.This boundary condition underlies the integration-by-parts step.Source: §2 · p. 4
- The optimal direction arises from a closed-form solution to the KL-divergence gradient optimization, expressed through kernelized Stein discrepancy.The discrepancy is zero if and only if the target and approximating distributions agree under mild regularity conditions.Source: §2 · pp. 3–4
- Each particle update combines the target log-density gradient, which moves particles toward high-probability regions, with kernel repulsion, which encourages diversity.Together the terms produce diverse points for distributional approximation and uncertainty assessment.Source: §2 · p. 4
- With one particle, the SVGD update reduces to gradient ascent on log p(x), equivalent to maximizing the target density at that point.This is the MAP-style special case rather than a diverse particle approximation.Source: §2 · pp. 3–4
3 AMORTIZED SVGD: TOWARDS AN AUTOMATIC NEURAL SAMPLER
Amortized SVGD trains a neural network to mimic SVGD updates, addressing the repeated computation and memory demands of particle methods. It incrementally adjusts network parameters so sampled outputs follow Stein directions without requiring the sampler density.
3 AMORTIZED SVGD: TOWARDS AN AUTOMATIC NEURAL SAMPLER
- Amortizing SVGD trains a neural network to mimic particle dynamics, addressing repeated-inference inefficiency and particle-storage demands across multiple tasks.Unlike particle methods, the network can improve from experience on prior tasks.Source: §3 · p. 4
- Rather than first running SVGD to convergence, the method incrementally changes network parameters so outputs follow Stein directions that decrease KL divergence.The paper notes that fitting fully converged SVGD results would require many epochs and can be slow.Source: §3 · pp. 4–5
- For each mini-batch, it computes network outputs and Stein gradients, forms target outputs by stepping along those gradients, then adjusts parameters to match them.The described example mini-batch size is m = 100.Source: §3 · p. 5
- A small-step least-squares approximation can be further approximated by one gradient-descent step, avoiding the matrix inversion required by the preceding update.The authors report this faster approximation works effectively in practice when the step size is small.Source: §3 · p. 5
- Unlike the reparameterized KL gradient, the Stein-gradient parameter update does not require explicitly computing qη, allowing complex samplers with unknown input distributions.The paper characterizes the Stein gradient as a kernel-smoothed version of the alternative gradient.Source: §3 · pp. 5–6
4 AMORTIZED MLE FOR GENERATIVE ADVERSARIAL TRAINING
The paper uses a neural sampler trained alongside an energy model to approximate the model expectation needed for maximum-likelihood training. This amortized approach produces realistic-looking images competitive with or better than state-of-the-art GANs.
4 AMORTIZED MLE FOR GENERATIVE ADVERSARIAL TRAINING
- The paper presents adaptive neural sampling as a way to design approximate samplers automatically and applies it to amortized MLE for deep generative models.The training procedure alternates sampler updates with updates to the energy-model parameters.Source: §4 · pp. 6–7
- The method produces realistic-looking images competitive with, or better than, state-of-the-art generative adversarial networks.The authors report this outcome for their amortized MLE method for deep generative models.Source: §4 · p. 6
- A neural network trained to sample from the energy model approximates the model expectation needed for its likelihood gradient update.The sampler is successively updated to follow p(x|θ) as θ is updated.Source: §4 · p. 6
- SteinGAN frames energy-model MLE and neural sampling as an adversarial game, with the energy model serving as discriminator.The section applies the sampling method to maximum-likelihood training of energy-based models with energy function φ(x; θ) and log-normalization constant Φ(θ).Source: §4 · p. 6
- The MLE gradient update lowers training-data energy and raises simulated-data energy, while the sampler’s SVGD update lowers simulated-data energy to fit the model.The authors contrast this with MCMC-MLE and contrastive divergence, describing the sampler as amortized during training.Source: §4 · p. 7
5 EMPIRICAL RESULTS
The experiments evaluate SteinGAN on MNIST, CIFAR-10, CelebA, and LSUN, with CIFAR-10 classification accuracy favoring SteinGAN over DCGAN. The section also describes the energy-model, kernel, and label-conditioning choices used to generate images.
5 EMPIRICAL RESULTS
- The energy model uses an autoencoder, and its RBF kernel operates on hidden representations with a bandwidth set to half the median pairwise distance of simulated images.The kernel’s repulsive force promotes variability, and its bandwidth adapts through the energy-model parameters and the simulated samples.Source: §5 · p. 7
- For datasets with labels, the joint energy model captures image structure and label relations, while conditioning the sampler on a chosen label enables category-specific image generation.The sampler draws labels according to empirical dataset counts before generating images from the label and a 100-dimensional random input.Source: §5 · p. 7
- On MNIST, both DCGAN and SteinGAN produce high-quality images described as visually indistinguishable from real images.Figure 1 shows generated images for each digit using the joint model, with m = 0.2.Source: §5 · p. 8, Figure 1 · p. 8
- On CIFAR-10, SteinGAN achieves higher testing accuracy than DCGAN, while its inception score is comparable to or slightly lower than DCGAN’s.The authors interpret inception scores as reflecting individual realism and label uniformity, and classification accuracy as reflecting information captured in the simulated image set.Source: §5 · p. 8, Figure 2 · p. 9
- The section presents generated-image results on CelebA and LSUN, using datasets of more than 200,000 faces and nearly 3 million bedroom images, respectively.Both datasets were cropped and resized to 64 × 64.Source: §5 · p. 8
6 CONCLUSION
The paper concludes by proposing a method for training neural samplers and SteinGAN for generative adversarial training. It identifies further applications and theoretical understanding of neural-sampler training as future directions.
6 CONCLUSION
- The conclusion presents neural-sampler training and SteinGAN generative adversarial training as the paper’s contributions.It names broader applications and theoretical understanding of neural-sampler training as future directions.Source: §6 · p. 9
- The accompanying CelebA and LSUN figures show generated images from SteinGAN and DCGAN.The CelebA figure also illustrates a random walk in the input changing generated facial attributes.Source: Figure 3 · p. 10, Figure 4 · p. 10