Source-linked AI summary
Adversarial Generation of Natural Language
Sai Rajeswar, Sandeep Subramanian, Francis Dutil, Christopher Pal, Aaron Courville
TL;DR
Adversarial natural-language generation is hindered by discrete outputs that block discriminator gradients and by exposure bias in likelihood-trained sequence models. The paper trains GANs by feeding the discriminator continuous generator distributions and true-data one-hot vectors, evaluates grammar adherence, and reports realistic, competitive, and conditional generation results.
Problem
Discrete token sampling prevents discriminator gradients from reaching the generator, while likelihood training exposes models only to ground-truth contexts and can cause errors to accumulate.
Method
The method makes GAN training possible for text by having the discriminator compare generator probability distributions with one-hot data sequences, using WGAN and WGAN-GP objectives and grammar-based evaluation.
Results
Across grammar experiments and language-modeling datasets, WGAN and WGAN-GP generalize to longer grammar sequences and produce realistic sentences, while the strategy is competitive on Chinese poetry and supports conditional generation.
Takeaways & Limitations
Continuous-valued discriminator inputs provide a straightforward and effective baseline for training GANs to generate natural language without gradient estimators.
Takeaways & Limitations
A standard training and evaluation criterion for natural-language GANs does not exist, and GANs are notoriously hard to train in practice.
Abstract
from arXiv · showhide
Generative Adversarial Networks (GANs) have gathered a lot of attention from the computer vision community, yielding impressive results for image generation. Advances in the adversarial generation of natural language from noise however are not commensurate with the progress made in generating images, and still lag far behind likelihood based methods. In this paper, we take a step towards generating natural language with a GAN objective alone. We introduce a simple baseline that addresses the discrete output space problem without relying on gradient estimators and show that it is able to achieve state-of-the-art results on a Chinese poem generation dataset. We present quantitative results on generating sentences from context-free and probabilistic context-free grammars, and qualitative language modeling results. A conditional version is also described that can generate sequences conditioned on sentence characteristics.
1 Introduction
Likelihood-trained language models face exposure bias because training uses ground-truth contexts, while adversarial natural-language generation must handle discrete outputs. The paper addresses this gap with a continuous-distribution discriminator and evaluates GAN-based generation across several settings.
- Likelihood-trained recurrent and convolutional language models use teacher forcing to predict each next word or character from previous ground-truth observations.
- Exposure bias arises because training conditions models on ground-truth contexts rather than their own errors, allowing small mistakes to accumulate during generation.
- GAN-based natural-language generation has lagged behind GAN image generation and likelihood-based methods.
- Discrete token sampling is non-differentiable, preventing discriminator gradients from directly propagating to the generator.
- The proposed approach feeds the discriminator generator probability distributions and real-data 1-hot vectors, then evaluates language, grammar, poetry, and conditional generation tasks.
2 Generative Adversarial Networks
GANs train generators and discriminators through competing distribution-matching objectives, but practical optimization is difficult. Wasserstein-based objectives and gradient penalties are introduced as alternatives intended to improve training signals and regularization.
- A GAN formulates generation as a two-player minimax game between a generator producing samples and a discriminator distinguishing them from real data.
- The original minimax formulation is associated with minimizing Jensen-Shannon Divergence between generated and true data distributions.
- In practice, generators maximize log(D(G(z))) because it provides stronger gradients early in training.
- GANs are notoriously hard to train, motivating modified objectives and regularization techniques.
- WGAN minimizes Wasserstein distance, while WGAN-GP adds a gradient-penalty regularizer to the discriminator objective.
- Weight clamping enforces a Lipschitz constraint in WGAN, whereas gradient penalty provides an alternative because performance is sensitive to clamping hyperparameters.
3 Model architecture
The architecture maps noise to continuous token distributions so adversarial training can operate without discrete sampling in the discriminator. The paper examines recurrent and convolutional generators and discriminators, with sequence-level training signals and greedy decoding.
- The generator transforms Gaussian noise into a sequence of vocabulary-sized probability distributions, while real samples appear as sequences of 1-hot vectors.
- The discriminator distinguishes generated from real sequences, and the generator is trained to fool it.
- Recurrent models transform input sequences into hidden states that summarize preceding inputs and produce vocabulary-sized outputs at each step.
- During inference, the models use greedy decoding by selecting argmax y_t at every step.
- The recurrent experiments use LSTM-based sequence modeling, with standard update equations and learnable parameters.
- Convolutional models use 1-dimensional filters across time and residual connections in both generators and discriminators.
- The adversarial generator is encouraged to match entire sequences without explicit supervision at each generation step.
4 Experiments & Data
The experiments evaluate adversarial language generation with tractable CFG/PCFG likelihood proxies, Chinese poetry benchmarks, language modeling datasets, and conditional sentence attributes.
- Evaluation strategy: CFG and PCFG experiments provide tractable evaluation through grammar membership or sample likelihood under a constructed data-generating distribution.A constituency parser evaluates CFG membership or probability under a PCFG, while generated samples are assessed under the true distribution rather than the GAN model itself.
- CFG and PCFG tasks: The toy CFG uses 100,000 samples each of lengths 5 and 11, with vocabularies of 36 and 45 tokens, and measures grammatical accuracy and diversity.Diversity is approximated by counting unique generated samples.
- CFG and PCFG tasks: The induced PCFG is built from Penn Treebank sections 0–21 using production-count statistics, with a 2,000-word output vocabulary and Viterbi-parser likelihood evaluation.The parser-based measure primarily captures grammaticality but serves as a proxy for sample quality.
- Language modeling: Language-generation experiments use word- and character-level models across datasets of varying size and complexity, with curriculum learning for LSTM models.CMU−SE contains 44,016 sentences and 3,122 words; Penn Treebank contains 42,068 sentences and 10,000 words; the 1-billion-word subset uses the top 30,000 words.
- Conditional generation: Conditional GANs concatenate binary attribute feature maps to generator and discriminator layers to model sentiment and question-related sentence characteristics.The conditional experiments use positive/negative sentiment and question/statement labels.
5 Results and Discussion
Across grammar, poetry, language modeling, and conditional-generation experiments, the proposed adversarial approach produces grammatical, diverse, competitive, and attribute-consistent text.
- Grammar generation: WGAN and WGAN-GP generalize from length-5 to length-11 CFG sequences, whereas other tested objectives fit length 5 but do not generalize similarly.This result motivates restricting subsequent experiments to WGAN and WGAN-GP objectives.
- Grammar generation: WGAN-GP appears to encourage diverse samples while fitting the CFG data distribution well.Uniqueness among 1,280 generations serves as a rough indicator of sample diversity.
- PCFG evaluation: Negative log-likelihood is compared for WGAN-GP, GAN-GP, and MLE LSTM models on PCFG samples using 64 samples evaluated after each epoch.The sequence length in this comparison is 7.
- Chinese poetry: On Chinese poetry, the straightforward continuous-output strategy is competitive and outperforms more complicated methods.The comparison uses BLEU scores on poem-5 and poem-7 datasets.
- Conditional generation: Conditional generation captures consistent patterns associated with questions and sentiment while producing sentences.The reported attributes are positive/negative sentiment and questions/statements.
- Language generation: LSTM and CNN models produce realistic word- and character-level sentences, with the CNN WGAN-GP model appearing to maintain context over longer spans.These qualitative results are reported in Tables 3 and 4.
6 Conclusion and Future work
The paper presents a simple GAN method for natural-language generation and evaluates it across datasets and conditional text-generation settings. It also identifies non-goal-oriented dialogue as future work because a clear training and evaluation criterion does not exist.
- The method trains GANs for natural language by giving the discriminator generator probability distributions and true-data 1-hot vectors.
- WGAN and WGAN-GP objectives produce realistic sentences on CMU-SE, Penn Treebank, and the 1-billion-word dataset.
- The evaluation includes word- and character-level generations on the 1-billion-word dataset and word-level generations on Penn Treebank and CMU-SE.
- Conditional generation produces text controlled by high-level features including sentiment and questions.
- Non-goal-oriented dialogue remains future work because a clear training and evaluation criterion does not exist.
Appendix
The appendix demonstrates that the proposed discrete-output strategy can also generate reasonable binarized MNIST samples with a DCWGAN and gradient penalty.
- The experiment uses continuous generator outputs while the true-data samples are binarized.
- The appendix shows binarized MNIST samples generated by a DCWGAN with gradient penalty.