Source-linked AI summary
Semi-Supervised Learning with Generative Adversarial Networks
Augustus Odena
TL;DR
The paper addresses semi-supervised classification while simultaneously learning a generative model. It introduces SGAN, which makes the discriminator predict N real classes plus FAKE, and reports more data-efficient classification and higher-quality generated samples than regular GANs. Systematic evaluation of sample quality across hyperparameters remains difficult.
Problem
The paper seeks to jointly solve semi-supervised classification and generative modeling rather than use learned representations only after training.
Method
SGAN makes the discriminator output N class labels plus a FAKE label, allowing it to serve as the classifier while training alongside the generator.
Results
SGAN improves restricted-data classification and generated-sample quality while reducing generator training times relative to the stated baselines.
Takeaways & Limitations
The method provides a single jointly trained model for semi-supervised classification and generation, with reported data-efficiency and sample-quality benefits.
Takeaways & Limitations
Systematic evaluation of sample quality across varying hyperparameters was difficult.
Abstract
from arXiv · showhide
We extend Generative Adversarial Networks (GANs) to the semi-supervised context by forcing the discriminator network to output class labels. We train a generative model G and a discriminator D on a dataset with inputs belonging to one of N classes. At training time, D is made to predict which of N+1 classes the input belongs to, where an extra class is added to correspond to the outputs of G. We show that this method can be used to create a more data-efficient classifier and that it allows for generating higher quality samples than a regular GAN.
1. Introduction
The paper proposes SGAN, a semi-supervised GAN that jointly learns a generative model and classifier, motivated by feedback among the generator, discriminator, and classifier. It reports improved restricted-data classification and higher-quality samples with shorter generator training.
- The proposed feedback loop lets improvements to D, C, and G iteratively support one another.
- SGAN extends GANs to learn a generative model and classifier simultaneously.
- SGAN improves classification performance on restricted datasets over a baseline classifier without a generative component.
- SGAN significantly improves generated-sample quality and reduces generator training times.
2. The SGAN Model
SGAN modifies the discriminator to output N class labels plus a FAKE label, allowing it to serve as the classifier. Training uses labeled real examples and generated examples with opposing likelihood objectives.
- The discriminator uses N+1 output units for CLASS-1 through CLASS-N and FAKE, and acts simultaneously as classifier C.
- Real minibatch examples receive class-specific labels, while generated examples use the FAKE label.
- D/C minimizes negative log likelihood on the combined minibatch, whereas G maximizes it on generated samples.
- The paper identifies the discriminator augmentation as the same method proposed in concurrent work by Salimans et al. (2016).
3. Results
MNIST experiments compare SGAN with regular GANs and an isolated classifier. The reported results indicate clearer generated samples and better data efficiency, while systematic sample-quality evaluation remains difficult.
- 3.1. Generative Results: SGAN outputs were significantly clearer than regular GAN outputs after MNIST training, across different initializations and architectures.
- 3.1. Generative Results: The regular-GAN comparison used SGAN with actual MNIST labels and a REAL/FAKE-only configuration semantically equivalent to a normal GAN.
- 3.1. Generative Results: Systematic evaluation of sample quality across varying hyperparameters was difficult.
- 3.2. Classifier Results: SGAN outperformed an isolated-classifier baseline increasingly as the training set was reduced.
- 3.2. Classifier Results: The authors attribute the data-efficiency result to forcing D and C to share weights.
4. Conclusion and Future Work
The paper proposes future extensions that vary parameter sharing, condition generation on class labels, or replace D/C with a ladder network using generated samples as unlabeled data.
- Future work could share only some D and C weights so others specialize in discrimination or classification.
- A conditional GAN extension would generate class-labeled examples and have D/C predict 2N REAL/FAKE-by-class labels.
- Another direction would replace D/C with a ladder network and use G's samples as unlabeled training data.