PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarially Learned Inference

Vincent Dumoulin, Ishmael Belghazi, Ben Poole, Olivier Mastropietro, Alex Lamb, Martin Arjovsky, Aaron Courville

arXiv:1606.00704v3stat.MLcs.LG

TL;DR

Deep generative models face a tension between efficient inference, sample fidelity, and useful representations. ALI jointly learns generation and inference through adversarial matching of data–latent joint distributions, producing coherent reconstructions and competitive semi-supervised results, while remaining limited for discrete data or discrete latent variables.

  • Problem

    Existing generative approaches trade off efficient inference, sharp samples, sampling speed, and abstract representations.

  • Method

    ALI jointly trains an encoder and decoder adversarially so their joint data–latent distributions become indistinguishable.

  • Results

    ALI learns mutually coherent inference and generation networks and achieves competitive state-of-the-art results on semi-supervised SVHN and CIFAR-10.

  • Takeaways & Limitations

    Learning inference jointly with generation benefits adversarial training and makes the learned latent mapping useful for auxiliary tasks.

  • Takeaways & Limitations

    ALI is not directly applicable to applications with discrete data or models with discrete latent variables, and reconstructions can be unfaithful because of underfitting.

Abstract

from arXiv · show

We introduce the adversarially learned inference (ALI) model, which jointly learns a generation network and an inference network using an adversarial process. The generation network maps samples from stochastic latent variables to the data space while the inference network maps training examples in data space to the space of latent variables. An adversarial game is cast between these two networks and a discriminative network is trained to distinguish between joint latent/data-space samples from the generative network and joint samples from the inference network. We illustrate the ability of the model to learn mutually coherent inference and generation networks through the inspections of model samples and reconstructions and confirm the usefulness of the learned representations by obtaining a performance competitive with state-of-the-art on the semi-supervised SVHN and CIFAR10 tasks.

1 INTRODUCTION

ALI integrates efficient inference into the GAN framework by jointly learning an inference network and a generative model through an adversarial game. Experiments report high-fidelity samples, efficient inference, and competitive semi-supervised performance.

  • Deep directed generative models enable fast ancestral sampling but are difficult to learn because inference is complex.
  • VAE methods provide reusable inference mechanisms but often produce blurry images, while autoregressive models sample slowly and lack abstract representations.
  • ALI jointly trains an encoder and decoder adversarially, requiring a discriminator to distinguish their joint data–latent distributions.
  • Experiments on SVHN, CIFAR-10, CelebA, and downsampled ImageNet report GAN-like sample fidelity together with efficient inference.
  • The learned representation achieves results competitive with the state of the art on semi-supervised SVHN and CIFAR-10.

2 ADVERSARIALLY LEARNED INFERENCE

ALI jointly learns an encoder and decoder by adversarially matching encoder and decoder joint distributions over data and latent variables. Under an optimal discriminator, this objective minimizes their Jensen-Shannon divergence, while practical training uses differentiable sampling and gradient-based updates.

  • ALI defines encoder and decoder joint distributions as q(x,z)=q(x)q(z|x) and p(x,z)=p(z)p(x|z).
  • Matching q(x,z) and p(x,z) matches their marginals and conditionals, including the encoder conditional q(z|x) and posterior p(z|x).
  • The reparametrization trick enables gradients to propagate through stochastic encoder sampling, while the training algorithm alternates discriminator and generator-network updates.
  • A discriminator distinguishes encoder pairs (x,ẑ) from decoder pairs (x̃,z), while both networks are trained to make the joint samples indistinguishable.
  • Under an optimal discriminator, the generator minimizes the Jensen-Shannon divergence between q(x,z) and p(x,z), reaching its minimum exactly when the joint distributions match.
  • ALI differs from GANs by using two generator components—encoder and decoder—and discriminating joint data-latent pairs rather than marginal data samples.

3 RELATED WORK

ALI is related to hybrid generative approaches that modify VAE objectives, adversarial autoencoders, and the independently proposed Bidirectional GAN. Its distinguishing choices include joint-pair discrimination, no explicit reconstruction loss, and stochastic inference networks.

  • Hybrid approaches modify VAE objectives by replacing the KL-divergence or reconstruction term with variants intended to improve their properties.
  • Unlike adversarial autoencoders, ALI optimizes no explicit reconstruction loss and discriminates joint (x, z) pairs rather than marginal latent samples.
  • Bidirectional GAN independently proposes the same model under a different name, emphasizing learned features for auxiliary supervised and semi-supervised tasks.
  • ALI uses a stochastic q(z | x) network, whereas the independently proposed Bidirectional GAN uses a deterministic one.

4 EXPERIMENTAL RESULTS

Experiments evaluate ALI across image generation, reconstruction, latent interpolation, semi-supervised learning, and joint inference-generation learning. ALI preserves crisp adversarial samples, supports useful representations, and generally improves mode coverage, while reconstructions can remain imperfect.

  • Experimental setup: ALI was applied to CIFAR10, SVHN, CelebA, and a 64 × 64 center-cropped ImageNet dataset.The experiments use four image datasets.
  • Samples and reconstructions: ALI samples exhibit the same image fidelity as samples from other adversarially-trained models.This qualitative result is reported across the evaluated datasets.
  • Samples and reconstructions: Reconstructions retain adversarial crispness but may misrepresent object placement, color, style, or identity, especially on CIFAR10.The authors interpret poor reconstructions as a sign of underfitting and note that ALI does not use an explicit reconstruction loss.
  • Latent space interpolations: Latent interpolations between validation examples produce smooth transitions and believable intermediate images, indicating latent features that generalize beyond memorized examples.Pairs are encoded, linearly interpolated in latent space, and decoded.
  • Semi-supervised learning: 3.00 ± 0.50% lower misclassification rate was achieved on SVHN than reported for GAN features when ALI’s inference network supplied representations.The result supports the usefulness of ALI’s inference mechanism for semi-supervised learning.
  • Semi-supervised learning: ALI achieved results competitive with the state of the art on semi-supervised tasks and modestly improved over Salimans et al. for 1000 and 2000 labeled examples without feature matching.The comparison concerns the semi-supervised benchmarks reported in Tables 1 and 2.
  • Importance of joint learning: 13.4 ± 5.8 modes were covered by ALI on average versus 10.4 ± 9.2 for GAN models in the 25-component toy mixture.Across runs, ALI covered 8–25 modes and GAN covered 1–22 modes.
  • Importance of joint learning: Jointly learning the encoder and decoder improves mode coverage and latent organization relative to GANs or post-hoc inverse mappings.The experiment concludes that adversarial training benefits from learning inference jointly with the decoder.

5 CONCLUSION

ALI jointly learns generation and inference networks through an adversarial process, producing mutually coherent mappings and competitive semi-supervised results.

  • ALI jointly learns a generation network and an inference network using an adversarial process.
  • The model learns mutually coherent inference and generation networks, as exhibited by its reconstructions.
  • ALI's learned latent-variable mapping achieves results competitive with the state-of-the-art on semi-supervised SVHN and CIFAR10.

A HYPERPARAMETERS

The appendix lists separate hyperparameter configurations for unsupervised CIFAR10, SVHN, CelebA, and Tiny ImageNet models.

  • Table 3 lists hyperparameters for the unsupervised CIFAR10 model.
  • The CIFAR10 discriminator uses Maxout layers.
  • Table 4 lists hyperparameters for the unsupervised SVHN model.
  • Table 5 lists hyperparameters for the unsupervised CelebA model.
  • Table 6 lists hyperparameters for the unsupervised Tiny ImageNet model.

B A GENERATIVE STORY FOR ALI

The Circle of Infinite Painters allegory depicts paired creative and technical roles whose coordinated work creates a unified output while evading a discriminator's scrutiny.

  • The Circle is secretly composed of two artistic duos rather than one artistic duo.
  • Xavier paints from intuition while Zach writes descriptions, forming the group's creative component.
  • Zelda describes paintings and Xena recreates descriptions technically, but neither Conditional sister is independently creative.
  • The four members work in pairs: Xavier's paintings are described by Zelda, while Zach's descriptions are painted by Xena.
  • The group monitors Joseph Discriminator and attenuates differences to preserve the illusion of a single artistic duo.
Loading 1606.00704v3…