Source-linked AI summary
Dual Discriminator Generative Adversarial Nets
Tu Dinh Nguyen, Trung Le, Hung Vu, Dinh Phung
TL;DR
GANs can collapse onto a few modes, motivating a method that preserves both coverage and sample quality. D2GAN uses two discriminators to combine KL and reverse KL objectives, and experiments report diverse, good-quality samples with scalability to ImageNet.
Problem
GAN training can suffer mode collapse, producing low-diversity samples because its JS-divergence objective behaves similarly to reverse KL.
Method
D2GAN combines KL and reverse KL divergences in a unified objective through a three-player minimax game with two discriminators and one generator.
Results
D2GAN improves diversity while maintaining good sample quality, scales to ImageNet, and achieves competitive variety with reasonably good images.
Takeaways & Limitations
The unified objective is intended to exploit complementary divergence properties and help avoid mode collapse while supporting large-scale generation.
Abstract
from arXiv · showhide
We propose in this paper a novel approach to tackle the problem of mode collapse encountered in generative adversarial network (GAN). Our idea is intuitive but proven to be very effective, especially in addressing some key limitations of GAN. In essence, it combines the Kullback-Leibler (KL) and reverse KL divergences into a unified objective function, thus it exploits the complementary statistical properties from these divergences to effectively diversify the estimated density in capturing multi-modes. We term our method dual discriminator generative adversarial nets (D2GAN) which, unlike GAN, has two discriminators; and together with a generator, it also has the analogy of a minimax game, wherein a discriminator rewards high scores for samples from data distribution whilst another discriminator, conversely, favoring data from the generator, and the generator produces data to fool both two discriminators. We develop theoretical analysis to show that, given the maximal discriminators, optimizing the generator of D2GAN reduces to minimizing both KL and reverse KL divergences between data distribution and the distribution induced from the data generated by the generator, hence effectively avoiding the mode collapsing problem. We conduct extensive experiments on synthetic and real-world large-scale datasets (MNIST, CIFAR-10, STL-10, ImageNet), where we have made our best effort to compare our D2GAN with the latest state-of-the-art GAN's variants in comprehensive qualitative and quantitative evaluations. The experimental results demonstrate the competitive and superior performance of our approach in generating good quality and diverse samples over baselines, and the capability of our method to scale up to ImageNet database.
1 Introduction
The introduction identifies mode collapse as a key GAN limitation and proposes D2GAN, which combines KL and reverse KL objectives through two discriminators. Theory and experiments support improved diversity, sample quality, and scalability to large datasets including ImageNet.
- Motivation: KL minimization tends to cover multiple data modes but may generate unseen samples, whereas reverse KL has complementary statistical behavior.The introduction frames objective selection as fundamental to generative-model learning and performance.
- Motivation: GAN’s JS-divergence objective behaves similarly to reverse KL, causing mode collapse and low-diversity generated samples.Mode collapse occurs when the generator produces similarly looking images from a low-entropy distribution.
- Proposed approach: D2GAN combines KL and reverse KL divergences in a unified objective using two discriminators and one generator in a three-player minimax game.One discriminator favors data samples, the other favors generated samples, and the generator fools both.
- Theory: Theoretical analysis shows that, with sufficiently capable discriminators, D2GAN minimizes both KL and reverse KL distances between data and model distributions.The analysis states that this encourages a fair distribution of probability mass across data modes.
- Evaluation: Experiments compare D2GAN with state-of-the-art baselines across synthetic data and MNIST, CIFAR10, STL-10, and ImageNet using multiple quantitative criteria.The evaluation is designed to assess both mode coverage and sample quality.
- Results: D2GAN improves diversity while retaining good sample quality and scales to ImageNet, where it achieves competitive variety and reasonably good images.The paper emphasizes both large-scale scalability and the balance between diversity and quality.
2 Generative Adversarial Nets
Standard GAN trains a generator and discriminator through a minimax game, with the optimal discriminator reducing generator learning to JS-divergence minimization. Because JS behaves like reverse KL, GAN can suffer mode collapse and produce low-diversity samples.
- GAN formulation: GAN uses a discriminator to distinguish real data from generated samples while the generator maps noise vectors into the data space.The two players are updated iteratively during training.
- GAN formulation: The standard GAN objective is a minimax game between the generator and discriminator.The displayed objective combines real-sample discrimination with generated-sample rejection.
- Optimization: With an optimal discriminator, generator optimization becomes minimization of the JS divergence between data and model distributions.At Nash equilibrium, the model distribution recovers the data distribution exactly.
- Limitation: Because JS has the same empirical nature as reverse KL, GAN suffers model collapse and generates samples with low diversity.The resulting samples may occupy only a narrow subset of the data distribution.
3 Dual Discriminator Generative Adversarial Nets
D2GAN uses two independent discriminators and one generator in a three-player game, combining KL and reverse KL objectives to address mode collapse. Its theoretical analysis links the generator’s optimized loss to both divergences and explains how α and β balance coverage against mode fidelity.
- Architecture: D2GAN consists of two independent discriminators, D1 and D2, plus a generator G that produces samples intended to fool both discriminators.D1 favors data-distribution samples, while D2 favors generator-distribution samples.
- Objective: The model is trained through a three-player minimax game whose objective contains separate terms weighted by α and β for the two discriminators.The hyperparameters also stabilize optimization because discriminator outputs are positive and unbounded.
- Optimization: D2GAN can be trained by alternately updating D1, D2, and G, similarly to GAN training.
- Theoretical analysis: Given sufficient discriminator and generator capacity, the theoretical analysis shows that D2GAN’s generator objective combines KL and reverse KL divergences between data and model distributions.The proof optimizes discriminators for a fixed generator before deriving the generator objective.
- Theoretical analysis: At the optimum, both divergences are minimized only when the generator distribution equals the data distribution, pG⋆ = pdata.At this point, both discriminators return the same score for real and generated samples.
- Hyperparameter effects: Increasing α emphasizes KL minimization and broader mode coverage, whereas increasing β emphasizes reverse KL minimization and better capture of a single mode.The paper notes that α may admit undesirable samples, while β may miss many modes; empirical adjustment balances these effects.
4 Experiments
Experiments evaluate D2GAN on synthetic multimodal data and increasingly diverse real-world datasets using visual and quantitative criteria. D2GAN covers modes more effectively than GAN baselines and scales to ImageNet while maintaining good sample quality.
- Synthetic data: On the eight-mode 2D mixture, standard GAN collapses near one mode, whereas UnrolledGAN and D2GAN distribute samples across all mixture components.The models use identical experimental settings for this comparison.
- Synthetic data: D2GAN achieves lower symmetric KL and Wasserstein distances than GAN and UnrolledGAN; its Wasserstein distance to the true distribution nearly reaches zero.Lower distances indicate closer agreement with the known true distribution, and the training curves also show D2GAN stability.
- Handwritten digit images: On MNIST, D2GAN's MODE scores are mostly near the maximum range of 8.0-9.0 and significantly exceed standard GAN and Reg-GAN.The scores are computed using a three-layer convolutional classifier with 0.65% MNIST test error.
- Handwritten digit images: With fixed α, increasing β initially improves D2GAN's MODE score, but further increases can significantly reduce it.This result demonstrates that the divergence-balancing hyperparameters require empirical adjustment.
- Natural scene images: On CIFAR-10, STL-10, and ImageNet, D2GAN trails DFM but outperforms other available unsupervised baselines by large margins; it also produces recognizable and varied images.The natural-image evaluation uses Inception scores and random, non-cherry-picked samples.
5 Conclusion
The paper introduces D2GAN, a three-player GAN framework that unifies KL and reverse KL divergences to improve generated-sample quality and diversity. Experiments show effectiveness and scalability across synthetic and large-scale real-world datasets, including ImageNet.
- D2GAN combines KL and reverse KL divergences in a unified density-estimation objective to improve sample quality and diversity.
- D2GAN formulates a minimax game with two discriminators and one generator.
- With fixed discriminators, generator learning simultaneously optimizes KL and reverse KL divergences, helping avoid GAN mode collapse.
- Extensive experiments evaluate the approach on synthetic and large-scale real-world datasets.
- D2GAN is reported as more scalable than state-of-the-art baselines, supports training on ImageNet, and achieves an Inception score below DFM but significantly above the other baselines.