PaperlayerView exact revision on arXiv

Source-linked AI summary

A survey on text generation using generative adversarial networks

Gustavo Henrique de Rosa, João Paulo Papa

arXiv:2212.11119v1cs.CLcs.AIcs.LG

TL;DR

GANs were designed for continuous rather than discrete data, complicating their use for text generation. This survey critically analyzes recent GAN-based text-generation research and its main strategies for addressing that mismatch. It concludes that this young research area still has relatively few studies.

  • Problem

    GANs were not designed for discrete text data, so text-generation approaches need additional strategies to address the mismatch.

    Source: §5 · p. 33

  • Method

    The paper surveys and critically analyzes recent GAN-based text-generation research and its approaches.

    Source: §5 · p. 32

  • Results

    The survey identifies continuous-based outputs, Reinforcement Learning, and modified training objectives as strategies used to address GANs’ difficulty with discrete text.

    Source: §5 · p. 33

  • Takeaways & Limitations

    The surveyed approaches offer alternatives for generating text with adversarial models despite the discrete-data mismatch.

    Source: §5 · p. 33

    The surveyed GAN-based text-generation works are recent, mostly from 2016 to 2020, and the field still lacks research.

    Source: §5 · p. 32

Abstract

from arXiv · show

This work presents a thorough review concerning recent studies and text generation advancements using Generative Adversarial Networks. The usage of adversarial learning for text generation is promising as it provides alternatives to generate the so-called "natural" language. Nevertheless, adversarial text generation is not a simple task as its foremost architecture, the Generative Adversarial Networks, were designed to cope with continuous information (image) instead of discrete data (text). Thus, most works are based on three possible options, i.e., Gumbel-Softmax differentiation, Reinforcement Learning, and modified training objectives. All alternatives are reviewed in this survey as they present the most recent approaches for generating text using adversarial-based techniques. The selected works were taken from renowned databases, such as Science Direct, IEEEXplore, Springer, Association for Computing Machinery, and arXiv, whereas each selected work has been critically analyzed and assessed to present its objective, methodology, and experimental results.

1. Introduction

The introduction frames text generation as language modeling over sequential tokens, then explains why applying GANs to text is difficult: they were designed for continuous rather than discrete data. The paper surveys strategies for that mismatch and aims to critically analyze recent GAN-based text-generation work.

1. Introduction

  • Language modeling predicts later characters, words, or sentences from preceding timesteps, allowing a learned model to generate new text from past information.The introduction describes text as sequential data whose earlier tokens are carried forward during learning.Source: §1 · pp. 2–3
  • RNNs reuse prior knowledge to learn future information, but are susceptible to adversarial manipulation, which can cause mispredictions and inefficient performance.The introduction identifies potential security threats to real-world applications as a consequence of this vulnerability.Source: §1 · p. 3
  • GANs were designed for continuous information rather than discrete text, prompting approaches based on Gumbel-Softmax differentiation, Reinforcement Learning, or modified training objectives [13].The introduction identifies these as alternatives for applying GANs to text generation.Source: §1 · p. 3
  • The survey critically analyzes GAN-based text-generation approaches and aims to provide a comprehensive source on their architectures, datasets, objectives, evaluation metrics, and results.It presents these approaches as alternatives for generating natural language.Source: §1 · p. 3

2. Survey Structure

The survey positions itself alongside earlier reviews by addressing GAN-based text generation in greater breadth. It searches selected databases and analyzes eligible work by objective, methodology, experimental results, metrics, and differentiability strategy.

2.1. Related Surveys

  • Earlier surveys covered natural-language generation broadly, six adversarial architectures against Maximum Likelihood Estimation on two datasets, or NLP adversarial attacks; this survey targets GAN-based text generation.The cited review of six architectures established a reference, while the attack survey did not address recent GAN text-generation advances.Source: §2.1 · p. 4
  • The survey aims to fill literature gaps with a reference on adversarial text-generation models, critically analyzing objectives, methods, experimental results, metrics, and differentiability strategies.Its categories include Gumbel-Softmax differentiation, Reinforcement Learning, and modified training objectives.Source: §2.1 · pp. 4–5

2.2. Paper Selection

  • The search covered Science Direct, IEEEXplore, Springer, ACM, and arXiv, using “generative adversarial network” and “text generation” as keywords.Selected works had to be relevant to GAN-based text generation and cited at least once; the authors also checked researchers’ backgrounds.Source: §2.2 · pp. 5–6
  • The survey reports 11 works in 2019 and 6 in 2020, as illustrated by Figure 1; its authors found GAN-based text-generation references using Gumbel-Softmax only beyond 2016.The authors state that no work addressed GAN-based text generation with Reinforcement Learning and modified training objectives before 2017.Source: §2.2 · pp. 5–6

3. Theoretical Background

This section supplies theoretical background on NLP, language modeling, GANs, Reinforcement Learning, and Gumbel-Softmax differentiation, alongside text-generation datasets and evaluation metrics.

3. Theoretical Background

  • The section introduces background on Natural Language Processing and Language Modeling.Source: §3 · p. 6
  • It also covers Generative Adversarial Networks, Reinforcement Learning, and Gumbel-Softmax differentiation-based strategies.Source: §3 · p. 6
  • The section describes four commonly employed text-generation datasets and top evaluation metrics.Source: §3 · p. 6

3.1. Natural Language Processing

The background describes NLP as modeling human-machine interaction through natural language and presents language modeling as learning token probabilities from examples. It then outlines recurrent prediction and iterative sampling as routes to generating text.

3.1. Natural Language Processing

  • NLP models and comprehends human-machine interaction through natural language, but linguistic morphology, syntax, semantics, and sequence connections make the task difficult for machines.The text describes NLP as an open research problem.Source: §3.1 · p. 6

3.1.1. Language Modeling

  • Language models learn token-occurrence probabilities from examples and can generate new text according to learned structure, while also serving as root architectures for larger NLP tasks.The cited uses include speech recognition, optical character recognition, machine translation, and spelling correction.Source: §3.1.1 · p. 7
  • Because language models learn from examples, they perform poorly on unknown data such as tokens not encountered during learning.Source: §3.1.1 · p. 7
  • RNNs predict the next timestep from prior inputs by reusing hidden states, whereas a feed-forward network cannot use past “memory” in the same way.The passage illustrates the distinction with predicting a noun after a verb.Source: §3.1.1 · p. 7

3.1.2. How to Generate Text?

  • Language models estimate a token’s probability from preceding tokens, sample a token, and repeat the process for the next timestep until a stopping criterion is met.The resulting tokens are arranged sequentially to form generated text.Source: §3.1.2 · p. 8

3.2. Generative Adversarial Networks

GANs pair a generative network that produces fake data with a discriminator estimating whether samples are real. For text generation, discrete token sampling makes the generator’s output difficult to differentiate, motivating Gumbel-Softmax and reinforcement-learning approaches.

3.2. Generative Adversarial Networks

  • Goodfellow et al. introduced GANs as competing generative and discriminative networks: the generator produces fake data, while the discriminator estimates whether samples are real.The networks compete in a zero-sum approach.Source: §3.2 · p. 8
  • When the discriminator easily rejects early generated samples, GANs can become trapped in local optima; maximizing log(D(G(z))) improves initial generator gradients.Source: §3.2 · p. 9

3.2.1. Differentiating GANs with Gumbel-Softmax

  • RNN generators sample one-hot categorical outputs that cannot be differentiated, and Jang et al. propose Gumbel-Softmax, a continuous distribution approximating categorical samples.Source: §3.2.1 · pp. 9–10
  • The Gumbel-Softmax formulation combines Softmax with Gumbel noise and a temperature parameter to produce differentiable outputs from unnormalized log probabilities.The passages identify w as unnormalized log probabilities, g as a Gumbel-distributed sample, and τ as the temperature.Source: §3.2.1 · p. 10

3.2.2. Reinforcing GANs Learning

  • The discriminator trains on real and artificial samples and supplies a reward signal for updating the generator with policy-gradient methods such as REINFORCE [19].Source: §3.2.2 · pp. 10–11
  • In reinforcement learning, the generator is a stochastic policy that treats the generated prefix as its state and the next token as its action.Source: §3.2.2 · p. 10
  • For incomplete sequences, the action-value reward is estimated from discriminator scores over N Monte Carlo rollouts; completed sequences use D(y).Source: §3.2.2 · p. 11

3.3. Datasets

The survey describes datasets used for text generation, including product reviews, poems, image captions, and material adapted from machine-translation data.

3.3. Datasets

  • Amazon Customer Reviews contains more than 233,000,00 user reviews describing customers’ experiences with products bought through Amazon.Source: §3.3 · p. 11
  • Chinese Poems contains 284,299 poems collected from sources including Tang, Song, Ming, Qing, and Tai poems and Song Ci.Source: §3.3 · p. 11
  • COCO Image Captions comprises 1,026,459 human-created captions from the Microsoft Common Objects in Context dataset.Source: §3.3 · p. 11
  • The COCO data are split into subsets with five reference sentences per image and 40 references for 5,000 randomly selected images.Source: §3.3 · p. 11
  • The EMNLP2017 WMT News dataset pairs documents across two languages and can be adapted to generation using sentences in the same language.Source: §3.3 · p. 12

3.4. Evaluation Metrics

The section reviews metrics for assessing generated-language quality and comparing generated text with human-created text. It describes BLEU, Negative Log-Likelihood, and Perplexity and explains what their scores indicate.

3.4. Evaluation Metrics

  • Evaluation metrics estimate generated-language relevance and support quality comparisons, including comparisons with information created by humans.The section frames these measures as ways to assess generated information and benchmark language-generation quality.Source: §3.4 · p. 12

3.4.1. Bilingual Evaluation Understudy

  • BLEU compares reference and machine-generated text through n-gram co-occurrence precision, with a short-sentence penalty and weighted geometric-mean score in [0, 1].Values closer to 1 indicate greater sentence similarity.Source: §3.4.1 · pp. 12–13

3.4.2. Negative Log-Likelihood

  • Negative Log-Likelihood evaluates whether a language model predicts the next token from the preceding tokens.It is used as a loss function and tends toward 0 as the network learns training-set patterns.Source: §3.4.2 · p. 13

3.4.3. Perplexity

  • Perplexity is the exponentiation of Negative Log-Likelihood; lower values indicate a well-trained network, while higher values indicate poor learning.Its stated range is [1, +∞].Source: §3.4.3 · p. 14

4. Analysis and Discussion

The analysis groups surveyed GAN-based text-generation works by how they handle discrete text: Gumbel-Softmax differentiation, Reinforcement Learning, and modified training objectives. It compares architectures using BLEU-2 across four common datasets and describes a shared methodological pipeline.

4. Analysis and Discussion

  • The analysis organizes GAN-based text-generation works into Gumbel-Softmax differentiation, Reinforcement Learning, and modified training objectives.Figure 3 describes a pipeline commonly used to conduct adversarial-based text generation.Source: §4 · p. 14, Figure 3 · p. 14
  • Table 1 summarizes surveyed works by reference, architecture, datasets, objectives, evaluation metrics, and experimental results.The section describes the analysis as a critical discussion of the compiled works.Source: §4 · p. 14
  • The section evaluates architectures using BLEU-2 across Amazon Reviews, Chinese Poems, COCO Image Captions, and EMNLP2017 WMT News.These are identified as four of the most common datasets for the comparison.Source: §4 · p. 14
  • For Gumbel-Softmax differentiation and modified objectives, the discriminator directly rewards the generator without Reinforcement Learning.The passage notes that the functions differ, but does not specify the distinction further.Source: §4 · p. 14

4.1. Text Generation using GANs and Gumbel-Softmax Differentiation

This section surveys GAN-based text-generation approaches using Gumbel-Softmax and other continuous relaxations, reinforcement learning, latent representations, teacher–student embeddings, and modified objectives. It describes methods for improving diversity, feedback, stability, and controllability, along with reported findings and challenges such as mode collapse, computational cost, and diversity–quality tradeoffs.

4.1. Text Generation using GANs and Gumbel-Softmax Differentiation

  • Nevertheless, GSGAN could produce only comparable results to a pre-trained LSTM and was no match for the Relational Generative Adversarial Network (RelGAN), introduced by Nie et al..Source: §4.1 · p. 20
  • Meta-CoTGAN eased mode collapse but incurred higher computational cost by training an additional language model during adversarial training.The section also notes that pre-training both discriminator and generator adds computational burden.Source: §4.1 · pp. 20–21
  • Gumbel-Softmax adds a temperature parameter that controls generated-text diversity and quality, while replacing the output-layer Softmax offers a straightforward implementation.The survey identifies mode collapse—sacrificing diversity for quality—as a wider problem for adversarial models.Source: §4.1 · pp. 20–21

4.2. Text Generation using GANs and Reinforcement Learning

  • SeqGAN used REINFORCE and Monte Carlo searches to pass discriminator gradients to the generator, establishing an RL research hallmark despite limited diversity and generalization.Its initial comparison was limited to standard MLE-based training.Source: §4.2 · p. 21
  • VGAN outperformed recurrent-based language models and SeqGAN on three public literature datasets.It combines VAE-learned features and high-level latent variables with an RL-based generator update.Source: §4.2 · pp. 21–22
  • ORGAN’s domain-specific reward objectives and Wasserstein-distance training outperformed MLE and SeqGAN baselines [33].Its reward combines discriminator feedback with domain-specific objectives; the authors concluded it could generate domain-based data.Source: §4.2 · p. 22
  • RankGAN ranks collections of human- and machine-written sequences against a reference group rather than training a discriminator for binary predictions.Its ranking approach is described as inspired by web-search ranking and uses references sampled across the reference space.Source: §4.2 · p. 23
  • DP-GAN assigns lower rewards to repeated text and higher rewards to novel text to encourage diversity.It also proposes a discriminator better suited to distinguishing repeated from novel text.Source: §4.2 · p. 23
  • BGAN trained stably from scratch on discrete data, though its generation remained inadequate.The review presents this as evidence of BGAN’s stability and capacity to work with discrete data.Source: §4.2 · pp. 23–24
  • Reward signals may be infeasible in early epochs, prompting pre-training and joint-reward strategies such as BFGAN.BFGAN uses backward and forward generators, pre-trains with MLE, and then applies REINFORCE under discriminator-provided reward signals.Source: §4.2 · p. 24
  • REGANs [37] compare REBAR, RELAX, and a modified REINFORCE, with distinct reward functions intended to estimate distributions over generated tokens.Experiments used synthetic context-free grammar and compared the three alternatives.Source: §4.2 · p. 24
  • IRL uses denser rewards and entropy-regularized policy gradients to encourage diverse text without a discriminator, alleviating reward sparsity and mode collapse while matching prior state-of-the-art results such as LeakGAN.The generator maximizes expected rewards, increasing them for authentic text and decreasing them for generated text.Source: §4.2 · pp. 24–25
  • CTGAN is trained with REINFORCE and uses a conditional LSTM generator, while SentiGAN trains multiple generators simultaneously and uses a multi-class discriminator to help produce sentiment-specific text.Source: §4.2 · p. 25
  • CD-GAN uses annotated data from other domains to reduce the need for large-scale annotations and generate emotional text.Its Seq2Seq generator is guided by sentiment and domain discriminators, using both non-labeled input-domain data and labeled auxiliary-domain data.Source: §4.2 · pp. 25–26
  • LeakGAN passes high-level discriminator features to a MANAGER module, which guides a WORKER module generating text.Both modules are RNN-based and trained with REINFORCE; bootstrapped re-scaled activation adjusts rewards to mitigate vanishing.Source: §4.2 · p. 26
  • MaskGAN fills in missing text conditioned on surrounding context, using timestep rewards and increasing maximum sequence length after convergence criteria are met.It uses a Seq2Seq generator trained with REINFORCE and generates rewards from the generator’s full distribution to reduce variance.Source: §4.2 · pp. 26–27
  • TranGAN outperformed standard Seq2Seq and SeqGAN in almost all metrics while processing discrete sequences through an actor-critic architecture.Its transformer-based generator uses an encoder-decoder architecture to extract long-term dependencies from context.Source: §4.2 · p. 27
  • QuGAN targets Tibetan question-answering corpus generation with QRNNs and improved reward and Monte Carlo search strategies.The approach uses BERT after generation to correct grammatical mistakes.Source: §4.2 · p. 27
  • TG-SeqGAN seeks to improve text generation and make generated text look closer to real data, with a self-attention discriminator capturing long-term dependencies.The SAL [48] architecture is described as reducing reward sparsity and the risk of mode collapse.Source: §4.2 · pp. 27–28
  • FGGAN supplies discriminator feedback through a feature-guidance CNN, while TextGAIL combines pretrained language models with reinforcement learning and imitation replay to stabilize training.FGGAN also applies semantic rules to remove illogical tokens; TextGAIL uses RoBERTa as its final discriminator classifier and estimates relative realism contrastively.Source: §4.2 · p. 28

4.3. Text Generation using GANs and Modified Training Objectives

  • MaliGAN [13] uses a normalized maximum-likelihood and importance-sampling objective resembling MLE, with variance-reduction techniques that stabilize training.The survey also notes that these techniques aided discriminators in detecting whether a generator had learned too much.Source: §4.3 · p. 29
  • IWGAN replaces weight clipping with a gradient penalty to enforce the Lipschitz constraint, but the survey reports too few baselines to assess scaling to larger language models.The cited limitation is that weight clipping may fail to converge at a particular clipped value in hard optimization tasks.Source: §4.3 · pp. 29–30
  • AltGAN uses a continuous relaxation that multiplies softmax outputs by their embeddings, and learns from scratch without pre-training.The survey describes it as handling increasing, variable-length sequences and identifies it as the first model to learn from scratch without pre-training.Source: §4.3 · p. 30
  • TextGAN matches real and fake sentence latent-feature distributions with RKHS discrepancy, using soft-argmax and complementary techniques to alleviate mode collapse.The complementary techniques include initialization strategies and discretization approximations.Source: §4.3 · p. 30
  • LaTextGAN [53] uses an AutoEncoder and low-dimensional latent spaces to discriminate sequences; t-SNE plots and interpolations indicate adequate modeling and smooth sentence encoding.The approach aims to address the dimensionality curse through an encoding-decoding system.Source: §4.3 · p. 30
  • STGAN generates sentence embeddings through an encoder-decoder, feeding continuous embedded samples without soft-differentiation or RL, and benchmarks several training algorithms.Compared algorithms include minibatch discrimination, gradient penalty, WGAN, and WGAN-GP.Source: §4.3 · pp. 30–31
  • TextKD-GAN replaces one-hot encodings with continuous text representations, using a teacher–student AutoEncoder reconstruction approach that challenges the discriminator and strengthens the generator.The data passes through an AutoEncoder and is then reconstructed.Source: §4.3 · p. 31
  • FM-GAN matches real and fake latent-feature distributions with Feature-Mover’s Distance for an RL-free approach, but suffers mode collapse while baselines have high diversity and lower-quality sentences.The FMD is based on a variation of Earth-Mover’s Distance and can be applied to discrete-data objectives.Source: §4.3 · p. 31
  • JSD-GAN optimizes Jensen–Shannon divergence between the generator and empirical training distributions using a closed-form discriminator, but applies only to explicit representations.The survey identifies this as an RL-free alternative to min-max optimization, with that representation constraint limiting its applicable tasks.Source: §4.3 · p. 31
  • DGSAN uses an optimized closed-form solution for an optimal discriminator, combining generator and discriminator power in one network and enabling direct sampling without calculating gradients.Its objective relates the current generator, a new generator, and the optimal discriminator.Source: §4.3 · pp. 31–32

5. Conclusions and Future Insights

The conclusion frames adversarial learning as an alternative for systems struggling with modified or unknown data, while identifying discrete inputs and language structure as challenges for GAN-based text generation. It presents the survey as a reference for recent work and points to unresolved research gaps and Transformer-based GANs as a promising direction for 2021.

5. Conclusions and Future Insights

  • The survey critically analyzes recent GAN-based text-generation research, mostly from 2016 to 2020, and Table 1 helps readers find particular studies.The authors describe it as a unique source of recent research.Source: §5 · p. 32
  • The surveyed field is recent, spanning 2016 to 2020, and still lacks research and adequately defined pipelines.Source: §5 · p. 32
  • GANs were not designed for discrete data, whose non-differentiable outputs inhibit appropriate gradient calculation and updating.The authors identify this as a primary challenge for GAN-based text generation.Source: §5 · pp. 32–33
  • Most works address the discrete-data problem with modified objectives, Reinforcement Learning, or continuous-based outputs such as Soft-Argmax and Gumbel-Softmax.Source: §5 · p. 33
  • Text generation must capture grammar, syntax, semantics, and how characters and words connect through memories and contexts.Source: §5 · p. 33
  • The authors suggest a more robust pre-learning step in which pretrained embeddings, including BERT, ELECTRA, and GPT-2, guide networks instead of MLE.Source: §5 · p. 33
  • Transformer architectures initially diverted attention from GAN systems, but the authors suggest GANs might benefit as pretraining approaches or architectural components.Source: §5 · p. 33
  • The authors identify unresolved gaps in GAN-based text generation and describe Transformer-based GANs as promising approaches to pursue in 2021.Source: §5 · p. 33
Loading 2212.11119v1…