Source-linked AI summary
DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text
Jingjing Xu, Xuancheng Ren, Junyang Lin, Xu Sun
TL;DR
Existing text-generation methods can produce repeated and boring expressions, while classifier discriminators struggle to distinguish novelty because their rewards saturate. DP-GAN rewards novel, fluent text with a language-model discriminator and trains the generator accordingly; on review and dialogue generation, it substantially outperforms baselines in producing diverse and informative text.
Problem
Existing text-generation methods can repeatedly produce boring, high-frequency expressions, while classifier-based discriminators do not accurately reflect text novelty.
Method
DP-GAN uses a language-model discriminator with cross-entropy rewards and policy-gradient training to assign low reward to repeated text and high reward to novel, fluent text.
Results
DP-GAN substantially outperforms baseline methods on review and dialogue generation, producing more diverse and informative text in automatic and human evaluations.
Takeaways & Limitations
The proposed discriminator better distinguishes novel from repeated text without the saturation problem, and improved diversity brings the generated distribution closer to real-world data than MLE.
Takeaways & Limitations
The authors omit detailed BLEU comparisons because BLEU against a single reference is debated as a measure of generated-text quality.
Abstract
from arXiv · showhide
Existing text generation methods tend to produce repeated and "boring" expressions. To tackle this problem, we propose a new text generation model, called Diversity-Promoting Generative Adversarial Network (DP-GAN). The proposed model assigns low reward for repeatedly generated text and high reward for "novel" and fluent text, encouraging the generator to produce diverse and informative text. Moreover, we propose a novel language-model based discriminator, which can better distinguish novel text from repeated text without the saturation problem compared with existing classifier-based discriminators. The experimental results on review generation and dialogue generation tasks demonstrate that our model can generate substantially more diverse and informative text than existing baselines. The code is available at https://github.com/lancopku/DPGAN
1 Introduction
DP-GAN addresses repetitive, boring text caused by conventional MLE by rewarding novel, fluent text and using a language-model discriminator. Experiments on review and dialogue generation show substantially more diverse and informative outputs than existing methods.
- Conventional MLE overproduces high-frequency words, discouraging meaningful low-frequency words and causing generated text to become repetitive and boring.For example, the frequencies of “the”, “and”, and “was” rise from 4.2%, 3.2%, and 1.5% in real data to 7.1%, 4.6%, and 5.3% in MLE-generated review text.
- DP-GAN assigns low rewards to repeated text and high rewards to novel, fluent text, encouraging the generator to produce diverse outputs through policy gradient.
- Classifier-based discriminators fail to represent novelty accurately because rewards saturate for highly novel text and collapse near zero for most generated text.A classifier may assign nearly saturated rewards such as 0.997 and 0.998 to sentences with different novelty levels, while achieving almost 99% identification accuracy.
- The language-model discriminator uses cross-entropy rewards that remain high for novel text and small but discriminative for low-novelty text, avoiding classifier reward saturation.
- DP-GAN substantially outperforms baseline methods on review and dialogue generation, producing more diverse and informative text in automatic and human evaluations.
2 Related Work
Related work identifies repetitive or dull outputs in sequence-to-sequence generation and training difficulties for text GANs in discrete spaces. DP-GAN builds on adversarial training and policy-gradient approaches to target these limitations.
- Prior dialogue-generation methods include hand-crafted rules, templates, human or simulated-user interaction, information retrieval, reinforcement learning, mutual information, prototype editing, and self-attention.
- Sequence-to-sequence models tend to generate short, repetitive, and dull text, motivating methods focused on informative and diverse generation.
- Text-generation GANs are difficult to train because generation samples from a discrete space without the normal gradient-descent solution.
- DP-GAN uses adversarial training and follows policy-gradient approaches that treat generation as a stochastic policy in reinforcement learning.
3 Diversity-Promoting GAN
DP-GAN combines a sequence-to-sequence generator with a language-model discriminator that rewards novel, fluent text and penalizes repeated generated text. Sentence- and word-level rewards train the generator with reinforcement learning while avoiding classifier saturation.
- 3.1 Overview: The generator uses a sequence-to-sequence structure to produce multiple sentences of varying lengths from an input sentence.A hierarchical LSTM decoder separately models sentence representations and words, with attention used for word decoding.
- 3.4 Rewards and 3.5 Policy Gradient Training: The model combines sentence-level and word-level rewards with policy-gradient reinforcement learning, alongside teacher forcing during generator training.Policy-gradient training uses generated text, while teacher forcing supplies real data to the decoder at each time step.
- 3.3 Discriminator: DP-GAN uses a unidirectional LSTM language-model discriminator whose cross-entropy output serves as the reward for training the generator.The discriminator is trained to maximize rewards for real-world text and minimize rewards for generated text, discouraging repeated outputs while penalizing low-quality text.
- 3.3 Discriminator: The proposed discriminator better distinguishes high-novelty from low-novelty text without saturation, assigning high rewards to novel text and discriminative low rewards to low-novelty text.Its behavior is contrasted with classifier-based discriminators, whose rewards saturate for highly novel text and collapse toward zero for generated text.
- 3.4.2 Word-Level Reward: O(T) reward computation evaluates all word rewards more efficiently than classifier-based discriminators, which require Monte Carlo roll-outs with O(T^2) complexity.The classifier provides reward only for completed sequences, whereas DP-GAN computes word-level rewards directly.
4 Experiment
DP-GAN is evaluated on review and dialogue generation using automatic diversity metrics, human judgments, reward analysis, distribution similarity, and generated examples. Across these evaluations, it produces more diverse and informative text, while combining sentence- and word-level rewards and slightly reducing fluency.
- 4.4.1 Automatic Evaluation: Combining sentence-level and word-level rewards yields more diverse n-grams than sentence-level reward alone and longer text than word-level reward alone.Sentence-level reward better increases total generated words, whereas word-level reward more precisely encourages novel n-grams.
- 4.4.2 Human Evaluation: Human evaluation finds DP-GAN best for diversity and relevance, with slightly lower fluency attributed to generating substantially longer text.Annotators rank relevance, diversity, and fluency; DP-GAN generates almost three times as many words as baselines on Yelp.
- 4.4.3 Analysis: Why It Works: DP-GAN’s language-model reward distinguishes novelty more precisely than SeqGAN’s classifier reward, which saturates for highly novel text and approaches zero for generated text.DP-GAN assigns high, nonsaturating rewards to novel text and low but nonzero rewards to less-novel generated text.
- 4.4.3 Analysis: Why It Works: DP-GAN’s generated word-frequency distribution is closer to real-world data across frequency intervals, especially for low-frequency words.The analysis uses cosine similarity between real-world and generated frequency vectors.
- 4.4.3 Analysis: Why It Works: On Yelp examples, MLE, PG-BLEU, and SeqGAN produce more generic and repetitive text, whereas DP-GAN generates more specific details and higher diversity.The comparison is presented in Table 3 as qualitative examples.
5 Conclusions
DP-GAN substantially outperforms baseline methods on automatic and human evaluations, producing more diverse and informative text. Its discriminator better separates novel from repeated text without saturation, and its improved diversity brings the generated distribution closer to real-world data than MLE.
- DP-GAN substantially outperforms baseline methods in automatic and human evaluations, producing more diverse and informative text.
- The proposed discriminator better distinguishes novel text from repeated text without the saturation problem of traditional classifier-based discriminators.
- With improved diversity, DP-GAN’s generated data distribution is closer to the real-world distribution than MLE’s.