PaperlayerView exact revision on arXiv

Source-linked AI summary

Semi-Supervised Learning with Generative Adversarial Networks

Augustus Odena

arXiv:1606.01583v2stat.MLcs.LG

TL;DR

The paper addresses semi-supervised classification while simultaneously learning a generative model. It introduces SGAN, which makes the discriminator predict N real classes plus FAKE, and reports more data-efficient classification and higher-quality generated samples than regular GANs. Systematic evaluation of sample quality across hyperparameters remains difficult.

  • Problem

    The paper seeks to jointly solve semi-supervised classification and generative modeling rather than use learned representations only after training.

    Source: §1 · p. 1

  • Method

    SGAN makes the discriminator output N class labels plus a FAKE label, allowing it to serve as the classifier while training alongside the generator.

    Source: §1 · p. 1, §2 · pp. 1–2

  • Results

    SGAN improves restricted-data classification and generated-sample quality while reducing generator training times relative to the stated baselines.

    Source: §1 · p. 1

  • Takeaways & Limitations

    The method provides a single jointly trained model for semi-supervised classification and generation, with reported data-efficiency and sample-quality benefits.

    Source: §1 · p. 1

    Systematic evaluation of sample quality across varying hyperparameters was difficult.

    Source: §3.1 · p. 2

Abstract

from arXiv · show

We extend Generative Adversarial Networks (GANs) to the semi-supervised context by forcing the discriminator network to output class labels. We train a generative model G and a discriminator D on a dataset with inputs belonging to one of N classes. At training time, D is made to predict which of N+1 classes the input belongs to, where an extra class is added to correspond to the outputs of G. We show that this method can be used to create a more data-efficient classifier and that it allows for generating higher quality samples than a regular GAN.

1. Introduction

The paper proposes SGAN, a semi-supervised GAN that jointly learns a generative model and classifier, motivated by feedback among the generator, discriminator, and classifier. It reports improved restricted-data classification and higher-quality samples with shorter generator training.

  • The proposed feedback loop lets improvements to D, C, and G iteratively support one another.Source: §1 · p. 1
  • SGAN extends GANs to learn a generative model and classifier simultaneously.Source: §1 · p. 1
  • SGAN improves classification performance on restricted datasets over a baseline classifier without a generative component.Source: §1 · p. 1
  • SGAN significantly improves generated-sample quality and reduces generator training times.Source: §1 · p. 1

2. The SGAN Model

SGAN modifies the discriminator to output N class labels plus a FAKE label, allowing it to serve as the classifier. Training uses labeled real examples and generated examples with opposing likelihood objectives.

  • The discriminator uses N+1 output units for CLASS-1 through CLASS-N and FAKE, and acts simultaneously as classifier C.Source: §2 · pp. 1–2
  • Real minibatch examples receive class-specific labels, while generated examples use the FAKE label.Source: §2 · p. 2
  • D/C minimizes negative log likelihood on the combined minibatch, whereas G maximizes it on generated samples.Source: §2 · p. 2
  • The paper identifies the discriminator augmentation as the same method proposed in concurrent work by Salimans et al. (2016).Source: §2 · p. 2

3. Results

MNIST experiments compare SGAN with regular GANs and an isolated classifier. The reported results indicate clearer generated samples and better data efficiency, while systematic sample-quality evaluation remains difficult.

3.1. Generative Results

  • SGAN outputs were significantly clearer than regular GAN outputs after MNIST training, across different initializations and architectures.Source: §3.1 · p. 2
  • The regular-GAN comparison used SGAN with actual MNIST labels and a REAL/FAKE-only configuration semantically equivalent to a normal GAN.Source: §3.1 · p. 2
  • Systematic evaluation of sample quality across varying hyperparameters was difficult.Source: §3.1 · p. 2

3.2. Classifier Results

  • SGAN outperformed an isolated-classifier baseline increasingly as the training set was reduced.Source: §3.2 · p. 2
  • The authors attribute the data-efficiency result to forcing D and C to share weights.Source: §3.2 · p. 2

4. Conclusion and Future Work

The paper proposes future extensions that vary parameter sharing, condition generation on class labels, or replace D/C with a ladder network using generated samples as unlabeled data.

  • Future work could share only some D and C weights so others specialize in discrimination or classification.Source: §4 · p. 2
  • A conditional GAN extension would generate class-labeled examples and have D/C predict 2N REAL/FAKE-by-class labels.Source: §4 · p. 2
  • Another direction would replace D/C with a ladder network and use G's samples as unlabeled training data.Source: §4 · p. 3

Related papers

Loading 1606.01583v2…