PaperlayerView exact revision on arXiv

Source-linked AI summary

DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models

Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, Lingpeng Kong

arXiv:2210.08933v3cs.CLcs.LG

TL;DR

Conditional sequence-to-sequence text generation with diffusion models remains underexplored because text is discrete. DiffuSeq adapts diffusion through partial noising and conditional denoising, and matches or surpasses competitive models while producing diverse outputs.

  • Problem

    Diffusion models had mainly been applied to unconditional discrete-text modeling, leaving conditional sequence-to-sequence generation underexplored.

  • Method

    DiffuSeq applies partial noising only to the target sequence and uses classifier-free conditional denoising for sequence-to-sequence generation.

  • Results

    Across 4 tasks and 6 baselines, DiffuSeq wins on at least one quality metric per comparison and consistently produces more diverse outputs.

  • Takeaways & Limitations

    DiffuSeq connects autoregressive, non-autoregressive, and diffusion models theoretically while demonstrating potential for complex conditional language generation.

  • Takeaways & Limitations

    The experimental setup initializes NAR targets from copied source or unknown-token embeddings, whereas diffusion targets start from N(0, I).

Abstract

from arXiv · show

Recently, diffusion models have emerged as a new paradigm for generative models. Despite the success in domains using continuous signals such as vision and audio, adapting diffusion models to natural language is under-explored due to the discrete nature of texts, especially for conditional generation. We tackle this challenge by proposing DiffuSeq: a diffusion model designed for sequence-to-sequence (Seq2Seq) text generation tasks. Upon extensive evaluation over a wide range of Seq2Seq tasks, we find DiffuSeq achieving comparable or even better performance than six established baselines, including a state-of-the-art model that is based on pre-trained language models. Apart from quality, an intriguing property of DiffuSeq is its high diversity during generation, which is desired in many Seq2Seq tasks. We further include a theoretical analysis revealing the connection between DiffuSeq and autoregressive/non-autoregressive models. Bringing together theoretical analysis and empirical evidence, we demonstrate the great potential of diffusion models in complex conditional language generation tasks. Code is available at \url{https://github.com/Shark-NLP/DiffuSeq}

1 INTRODUCTION

Diffusion models had mainly advanced in continuous-signal domains, while text applications focused on unconditional generation. DiffuSeq extends diffusion to conditional Seq2Seq generation and targets quality, diversity, and theoretical understanding.

  • Diffusion models had achieved major breakthroughs for continuous signals but remained under-explored for natural-language generation.
  • Existing text diffusion efforts primarily addressed unconditional language modeling, while classifier-guided approaches used separately trained classifiers for conditions.
  • Seq2Seq generation covers important tasks including open-ended sentence generation, dialogue, paraphrasing, and text style transfer.
  • DiffuSeq is a classifier-free diffusion model that supports Seq2Seq generation with one model for conditional target distributions.
  • DiffuSeq generates tokens in parallel and aims to provide sentence-level diversity without sacrificing generation quality.
  • The paper connects DiffuSeq theoretically to autoregressive and non-autoregressive models and presents empirical evidence for conditional language generation.

2 PRELIMINARY AND PROBLEM STATEMENT

Diffusion models for discrete text had focused on unconditional sequence modeling. This paper instead formulates conditional Seq2Seq generation as producing a target sequence from a source sequence.

  • Preliminary: Diffusion models typically corrupt data through a forward process and reconstruct it through a learned reverse denoising process.
  • Problem Statement: Prior efforts adapting diffusion models to discrete text focused on unconditional sequence modeling.
  • Problem Statement: The target problem is learning a diffusion model that produces an n-length target sequence conditioned on an m-length source sequence.

3 DIFFUSEQ

DiffuSeq adapts diffusion to conditional Seq2Seq generation by embedding source and target text, noising only the target, and conditionally denoising with one model. Its formulation connects diffusion with iterative non-autoregressive generation.

  • DIFFUSEQ: DiffuSeq extends vanilla diffusion models to conditional text generation through model-architecture and training-objective modifications.
  • Forward Process with Partial Noising: The model maps source-target text pairs into a shared continuous feature space through embedding transformation and concatenation.
  • Forward Process with Partial Noising: Partial noising injects noise only into the target representation while preserving the source representation as the conditioning signal.
  • Reverse Process with Conditional Denoising: During reverse diffusion, DIFFUSEQ denoises the target while conditioning on the source and requires no separately trained classifier.
  • Training Objective: Attention-based reconstruction of the target also affects source embedding learning, while shared embeddings jointly train source and target feature spaces.
  • Training and Inference Methods: Importance sampling allocates more training effort to diffusion steps with larger loss values to address insufficient training.
  • Training and Inference Methods: At inference, anchoring rounds continuous states to word-embedding space and replaces the recovered source portion with the original source representation.
  • Training and Inference Methods: MBR decoding selects among samples from different random seeds using expected risk, implemented with negative BLEU.

4 EXPERIMENTS

Experiments evaluate DiffuSeq on four Seq2Seq tasks against six AR/NAR baselines, measuring quality, diversity, and inference behavior. DiffuSeq generally matches or exceeds baseline quality while producing more diverse outputs, with analyses covering candidate selection, generation steps, speed, and joint training.

  • Experimental Setup: Experiments cover four Seq2Seq tasks and compare DiffuSeq with six strong autoregressive and non-autoregressive baselines.The evaluation includes open-domain dialogue, question generation, and other representative Seq2Seq tasks, with baseline families spanning encoder-decoder models, finetuned PLMs, and iterative NAR models.
  • Main Results: DiffuSeq achieves comparable or higher generation quality than strong baselines and consistently produces more diverse outputs for the same input.Quality is assessed with BLEU, ROUGE, and BERTScore, while diversity is assessed with self-BLEU, div-4, and dist-1.
  • Main Results: Across 6 baselines × 4 tasks, DiffuSeq wins on at least one quality metric, with relative improvements over 50% for BLEU in question generation and R-L in dialogue over LevT.DiffuSeq is superior to GPT2-base and comparable with GPT2-large, which has 8.2 times more parameters.
  • Main Results: DiffuSeq’s diversity is strongest at the sentence level, while GPT2 models remain better on token-level repetition and BERTScore.DiffuSeq outperforms four AR baselines on self-BLEU and div-4, but still trails pretrained GPT2 models on dist-1 and BERTScore.
  • Analysis: With larger candidate sets, DiffuSeq’s MBR quality improvement surpasses GPT2 because GPT2 generates highly similar candidates.DiffuSeq lags GPT2 at candidate sizes 1 and 3, then shows an increasingly stronger BLEU trend as |S| grows toward 20.
  • Analysis: During generation, DiffuSeq’s BLEU rises slowly before accelerating and surpassing LevT, while its diversity remains higher throughout; at 1,000 inference steps it exceeds GPT2-large’s BLEU at similar speed.The paper also reports that decoupling source and target embedding training produces poor performance compared with joint training.

5 RELATED WORK

Prior diffusion work for text focused mainly on unconditional generation or fine-grained classifier-guided control. DiffuSeq instead targets continuous-space diffusion for conditional Seq2Seq generation, conditioning on the full source sentence.

  • Diffusion Models for Conditional Generation: Diffusion-LM uses an extra classifier or plug-and-play guidance to impose fine-grained constraints on generated sentences.This approach does not naturally condition on the whole source sentence for Seq2Seq tasks.
  • Diffusion Models for Text Modeling: Text diffusion models include multinomial and absorbing-state approaches for discrete generation, plus continuous latent-representation models such as Diffusion-LM and Analog Bits.The cited discrete models generate unconditionally, while continuous models use mapping functions between discrete and continuous text spaces.
  • Diffusion Models for Conditional Generation: DiffuSeq is the first work described here to explore continuous-space diffusion models for conditional Seq2Seq text generation.It addresses source-conditioned generation rather than only unconditional text modeling.

6 CONCLUSIONS

DiffuSeq applies diffusion modeling to Seq2Seq generation and connects it theoretically to autoregressive and non-autoregressive models. Empirical results indicate a favorable quality–diversity trade-off and competitive performance across model families.

  • 6 CONCLUSIONS: DiffuSeq targets Seq2Seq generation with diffusion and is presented as having strong potential for balancing generation quality and diversity.The conclusion frames this as a new sequence-to-sequence learning paradigm.
  • 6 CONCLUSIONS: Theoretical analysis connects DiffuSeq with autoregressive and non-autoregressive models and identifies it as an extension of iterative NAR.This connection provides the paper’s theoretical account of DiffuSeq’s modeling position.
  • 6 CONCLUSIONS: Empirical results show DiffuSeq matching or surpassing competitive autoregressive, iterative NAR, and large-scale pretrained models on quality and diversity.The reported comparison covers both generation quality and diversity.

A OBJECTIVE DERIVATIONS OF DIFFUSEQ

The objective derivation defines diffusion as forward noising followed by reverse denoising in continuous latent space. A variational lower bound decomposes training into terms that can be parameterized through the learned reverse process.

  • Diffusion objective: Diffusion defines forward noising and reverse denoising processes over a continuous latent variable z.The forward process perturbs data structure, while the reverse process aims to recover the original latent z0.
  • Forward process: The forward process applies T random disturbances that transform z0 into partial Gaussian noise, with yT sampled from N(0, I).The schedule uses αt = 1 − βt and cumulative ᾱt terms.
  • Forward process: The noised latent distribution is q(zt|z0) = N(zt; √ᾱt z0, (1 − ᾱt)I).The cited formulation explicitly introduces Gaussian noise through the cumulative schedule.
  • Reverse process: The reverse transition is modeled as pθ(zt−1|zt) = N(zt−1; µθ(zt, t), σθ(zt, t)).The model predicts the mean and standard variation used for reverse denoising.
  • Training objective: Training minimizes a variational lower bound on negative log-likelihood, decomposed into KL-divergence and entropy terms.The parameterization of intermediate terms is chosen to minimize differences involving the forward and learned means.

B GRAPHICAL MODELS OF AR, FULLY NAR, ITERATIVE NAR AND DIFFUSEQ MODELS

The graphical-model analysis contrasts AR, fully NAR, iterative NAR, and DiffuSeq through their dependency structures and generation processes. DiffuSeq is derived as a generalized iterative-NAR model with Gaussian initialization and progressive full-context refinement.

  • AR and fully NAR: AR models use an initial prediction followed by progressive left-context prediction, whereas fully NAR models assume conditional independence for fast inference.These represent distinct dependency structures for Seq2Seq generation.
  • Iterative NAR: Iterative NAR models introduce intermediate sequences and progressively predict with full context rather than generating the entire output at once.Their refinement process is contrasted with the one-shot behavior of AR and fully NAR models.
  • Theoretical connection: The conditional total-correlation discrepancy between AR and fully NAR models is closed in iterative NAR under sufficient refinement steps.Because DiffuSeq extends iterative NAR, the paper argues it avoids this discrepancy for the same reason.
  • DiffuSeq: DiffuSeq represents discrete text with continuous features and uses rounding to map continuous vectors to discrete tokens at each diffusion step.The diffusion process operates on contiguous representations while retaining discrete text variables.
  • DiffuSeq: DiffuSeq generalizes iterative NAR before marginalizing intermediate continuous states, with equivalent computation orders but different initialization distributions.Iterative NAR initializes yT from copied source or unknown-token embeddings, while diffusion samples yT from N(0, I).
  • Graphical comparison: Iterative NAR and diffusion models use self-corrected generation, unlike AR and fully NAR models that generate text all at once.Figure 7 compares their source dependencies and generation structures.

D.1 PROCESSING OF QUESTION GENERATION DATASET

The question-generation data and evaluation setup combine extracted document-question pairs with model-specific sampling and diversity controls. Metrics include smoothed BLEU, ROUGE-L, and BERTScore, with acknowledged limits for n-gram metrics.

  • Dataset construction: 119K document-question training pairs were extracted from Quasar-T triplets when the answer exactly matched the document.The pairs were obtained after preprocessing.
  • Model comparison: Model comparisons consider parameter counts and how different output sentences are sampled.The comparison is summarized in Table 4.
  • Diversity settings: Diversity-generation settings vary by model, using up to nine iterations for NAR-LevT and tuned scalars for GPVAE-T5.Other baselines may lack an explicit diversity-control factor and therefore appear as single points.
  • Evaluation metrics: Evaluation uses sentence-level smoothed BLEU-1 to BLEU-4, ROUGE-L, and BERTScore based on DeBERTa-xlarge-mnli.The paper notes that n-gram metrics may miss semantic meaning, motivating BERTScore.

D.5 GENERATION RESULTS

Generation examples show that DiffuSeq produces diverse outputs across dialogue, question generation, and text simplification tasks. The examples also indicate a fluency trade-off relative to finetuned GPT2.

  • Generation results: DiffuSeq tends to generate diverse outputs across the reported task examples, though some outputs are less fluent than finetuned GPT2.This observation is drawn from Tables 5, 6, and 7.
  • Dialogue: Dialogue outputs are illustrated with different random seeds in Table 5.One example answers a question about how long hair dye lasts.
  • Question Generation: Question-generation outputs are illustrated with different random seeds in Table 6.The supplied passages identify the table but do not state a specific comparative outcome.
  • Text Simplification: Text-simplification examples transform a complex explanation of loneliness into a shorter statement about lacking childhood and teenage friends.The example preserves one stated cause while simplifying the wording.
Loading 2210.08933v3…