PaperlayerView exact revision on arXiv

Source-linked AI summary

PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Model

Yizhe Zhang, Jiatao Gu, Zhuofeng Wu, Shuangfei Zhai, Josh Susskind, Navdeep Jaitly

arXiv:2306.02531v3cs.CL

TL;DR

Autoregressive text generation can repeat or drift from intended semantics, while text diffusion can be costly and less fluent for long text. PLANNER diffuses paragraph-level semantic embeddings and decodes them autoregressively; the paper reports more diverse, less repetitive generation while maintaining fluency and relevance across tasks.

  • Problem

    Autoregressive generation can accumulate errors and repeat, while text diffusion for long text can be computationally expensive and less fluent than autoregressive models.

    Source: §1 · pp. 1–2

  • Method

    PLANNER learns paragraph-level semantic embeddings and uses latent diffusion to plan them before an autoregressive decoder generates the text.

    Source: §1 · p. 2, §4 · p. 3, §4.1 · p. 4

  • Results

    Across tasks, PLANNER is reported to reduce repetition and improve diversity while maintaining fluency and relevance against autoregressive and text-diffusion baselines.

    Source: §1 · p. 2, §7 · p. 10

  • Takeaways & Limitations

    The paper concludes that diffusion over paragraph embeddings can improve diversity and reduce repetition while generating fluent text compared with diffusion over tokens or token embeddings.

    Source: §7 · p. 10

    Paragraph reconstruction has conversion loss, with BLEUclean between 77 and 87 for hundreds of words, and some lexical mismatches include named-entity swaps.

    Source: §6.1 · p. 7

Abstract

from arXiv · show

Autoregressive models for text sometimes generate repetitive and low-quality output because errors accumulate during the steps of generation. This issue is often attributed to exposure bias - the difference between how a model is trained, and how it is used during inference. Denoising diffusion models provide an alternative approach in which a model can revisit and revise its output. However, they can be computationally expensive and prior efforts on text have led to models that produce less fluent output compared to autoregressive models, especially for longer text and paragraphs. In this paper, we propose PLANNER, a model that combines latent semantic diffusion with autoregressive generation, to generate fluent text while exercising global control over paragraphs. The model achieves this by combining an autoregressive "decoding" module with a "planning" module that uses latent diffusion to generate semantic paragraph embeddings in a coarse-to-fine manner. The proposed method is evaluated on various conditional generation tasks, and results on semantic generation, text completion and summarization show its effectiveness in generating high-quality long-form text in an efficient manner.

1 Introduction

Autoregressive generation can drift into repetition and away from intended semantics, while text diffusion brings fluency and computational challenges. PLANNER applies diffusion to paragraph-level semantic latents and pairs it with autoregressive decoding for more diverse, fluent generation with global control.

1 Introduction

  • Teacher-forced autoregressive generation cannot revise mistakes during decoding, so errors can accumulate as generation progresses.The paper links this mismatch between training and inference to exposure bias.Source: §1 · pp. 1–2
  • Repetitive prompts can trigger self-reinforcing repetition, and open-ended generations may drift from desired semantics, especially with poor prompts or high initial probabilities.The authors report these patterns as more prevalent in open-ended tasks.Source: §1 · p. 2
  • Text diffusion can revise outputs iteratively, but prior models may be less fluent and costly for long text, while token-to-embedding conversion can cause rounding errors.Earlier remedies such as adversarial models, reinforcement learning, and repetition penalties still follow autoregressive generation with local adjustments.Source: §1 · p. 2
  • PLANNER combines non-autoregressive semantic diffusion with autoregressive decoding to generate fluent paragraphs while retaining global control.Diffusion plans paragraph semantics through coarse-to-fine refinement; the decoder translates those semantics into text.Source: §1 · p. 2
  • PLANNER’s iterative refinement is reported to reduce repetition and improve diversity while maintaining fluency and relevance relative to autoregressive and text-diffusion baselines.The paper also reports more diversified generations robust to repetitive or ill-composed prompts where fine-tuned GPT-2 failed.Source: Figure 1 · p. 1, §1 · p. 2

2 Preliminary

The preliminary section describes diffusion as progressive noising followed by learned denoising, then introduces latent diffusion’s use of an autoencoder to model a lower-dimensional representation.

2 Preliminary

  • A diffusion model progressively corrupts data with scheduled noise, then learns a parametric function to denoise corrupted inputs toward clean data.Sampling may use ancestral DDPM or deterministic DDIM procedures.Source: §2 · p. 3
  • The diffusion model is trained with a weighted reconstruction loss, and its weighting uses the signal-to-noise ratio.The denoising backbone is represented by Fθ.Source: §2 · p. 3
  • Latent diffusion separates compression and generation by encoding data into a low-dimensional space and decoding generated latent codes back to the original space.The latent representation is intended to reduce computational complexity while retaining perceptual quality.Source: §2 · p. 3

3 Related Work

Prior text-diffusion work explores discrete and continuous representations, while this paper motivates paragraph embeddings as a fixed, shorter alternative and relates its approach to text VAEs.

3 Related Work

  • Prior text diffusion has used discrete token noising, continuous token embeddings with rounding, analog bits, contextualized BART embeddings, and image-captioning applications.The cited examples include Diff-LM, DiffuSeq, Analog Bits, and work on contextualized BART embeddings.Source: §3 · p. 3
  • Existing text diffusion must handle variable sequence lengths and can produce disfluent or repetitive text, especially for long generations.The paper instead models shorter, fixed-sized paragraph embeddings, which it presents as beneficial for computational cost and fluency.Source: §3 · p. 3
  • Text VAEs can learn smooth, interpretable representations and diverse text, but KL vanishing can lead decoders to disregard latent codes.The authors frame their approach as using a more flexible prior to help ensure codes influence the output distribution.Source: §3 · p. 3

4 PLANNER: A Language Diffusion Model on Paragraph Embeddings

PLANNER learns paragraph-level latent codes with a variational embedder, then uses conditional latent diffusion to plan those codes before decoding them into text.

4 PLANNER: A Language Diffusion Model on Paragraph Embeddings

  • PLANNER uses a variational paragraph embedder to learn a smooth latent space, then a diffusion model to generate codes representing longer-text semantics.The overview presents a two-part model: paragraph embedding followed by latent-code generation and text decoding.Source: §4 · p. 3, Figure 2 · p. 4

4.1 Learning a Variational Paragraph Embedder

  • Paragraph embeddings are fixed-length latent codes that capture the semantics of target texts up to 512 tokens, with examples using k = 16 codes.The codes are shorter than the original text and are represented as vectors of dimension h.Source: §4.1 · p. 4
  • A transformer encoder-decoder forms a bidirectional mapping: the encoder’s first k final-layer states become paragraph embeddings, and the decoder reconstructs the input text.Pretrained BERT and GPT-2 initialize the encoder and decoder, respectively.Source: §4.1 · p. 4
  • Substituting input tokens during denoising-autoencoder training improves local smoothness, but substitution probability trades reconstruction accuracy against smoothness.The paper evaluates clean reconstruction with BLEUclean and robustness to corrupted input with BLEUrobust.Source: §4.1 · p. 4
  • A text VAE regularizes the posterior toward a Gaussian distribution to make the paragraph-embedding distribution easier for diffusion to learn.The regularization strength β is kept small to alleviate posterior collapse; interpolated embeddings are assessed with GPT-2 perplexity.Source: §4.1 · pp. 4–5

4.2 Planning then Decoding: A Latent Diffusion Model for Paragraph Embeddings

  • With the embedder frozen, conditional diffusion is trained on mean paragraph embeddings, using class labels or encoded raw text as conditioning signals.The denoising transformer receives conditioning and time embeddings through cross-attention and adaptive layer normalization.Source: §4.2 · p. 5
  • At inference, the model starts from Gaussian embeddings, denoises for a fixed number of steps, then decodes the resulting codes with deterministic decoding.Classifier-free guidance and dynamic thresholding are applied during inference; the authors omit Imagen’s rescaling step to avoid altering latent semantics.Source: §4.2 · p. 5

5 Experimental Setups

PLANNER is evaluated on three conditional-generation tasks, with configurations and baselines specified for long-form text. The setup also examines paragraph-embedding noise and uses metrics for fluency, diversity, repetition, relevance, and denoising.

5 Experimental Setups

  • The evaluation covers sentiment-guided generation, long-form completion, and summarization, using TripAdvisor, C4, CNN/DailyMail, and XSum data.Completion targets the remainder of a paragraph after its first two sentences are supplied as context.Source: §5 · pp. 5–6
  • Fluency, lexical diversity, repetition, and relevance are assessed with PPL, Ent-n and DIST-n, self-BLEU, REP-n, SacreBLEU, ROUGE-L, and BERTScore.Ent-n and DIST-n are reported at n = 1.Source: §5 · p. 6
  • AuBLEU summarizes BLEU across corruption levels as the area under a curve, providing a metric for comparing text-diffusion models' denoising ability.The curve plots BLEU for denoised inputs against α^2, which ranges from 0 to 1 as signal-to-noise ratio increases.Source: §5 · p. 6
  • The latent diffusion model uses 30 DDIM steps, 16 paragraph embeddings, and a 533M-parameter diffusion model; deterministic decoding is used to convert embeddings into text.The encoder and decoder are initialized from BERT-large and GPT-medium, respectively.Source: §5 · pp. 5–6
  • Baselines include GPT-2 fine-tuning, encoder-decoder and VIB models, T5-large for summarization, and the Diff-LM and Genie text-diffusion models.GPT-2 baselines use top-p sampling or beam search; encoder-decoder and VIB generations use top-p sampling.Source: §5 · p. 6
  • Large substitution noise on XSum worsens BLEUclean but improves BLEUrobust and PPLint.Figure 3 examines the impact of injected-noise proportion when learning paragraph embeddings.Source: Figure 3 · p. 6

6 Results

Across sentiment-guided generation, long-form completion, and summarization, PLANNER generally improves diversity and reduces repetition while maintaining relevant or comparable-quality output. Its paragraph representations balance reconstruction accuracy and smoothness, though conversion loss and occasional hallucinations remain.

6.1 Paragraph Representation Learning

  • Paragraph representation learning balances reconstruction accuracy against local and distributional smoothness, while unavoidable conversion loss can include name-entity swaps.The selected substitution probability was p = 0.3 for most datasets; BLEUclean was 77–87 when reconstructing hundreds of words.Source: §6.1 · p. 7

6.2 Sentiment-Guided Generation

  • In sentiment-guided generation, PLANNER achieves higher sentiment accuracy than baselines, with diversity comparable to FT top-p sampling and lower Rep-4 under greedy decoding.The sentiment classifier used for assessment achieved 96.75% held-out test accuracy.Source: §6.2 · p. 7, Table 1 · p. 7

6.3 Long-form Text Completion

  • PLANNER improves diversity while maintaining comparable relevance against Diff-LM and other baselines in long-form completion, and remains robust to repetitive or ill-formed prompts.The Diff-LM comparison was not run on C4 because adequate training would require thousands of GPU hours.Source: §6.3 · pp. 7–8, Table 2 · p. 8
  • Human judges moderately preferred PLANNER over FT and VIB for hotel reviews, except for human-likeness versus VIB; they still preferred human responses over system outputs.The pairwise evaluation covered 300 examples per system, each judged by three people.Source: §6.3 · p. 8, Table 3 · p. 8

6.4 Summarization

  • For summarization, PLANNER matches Rouge-L scores while improving other metrics over T5-sample and Genie, and produces more diverse, less repetitive summaries than T5-search.The diversity and repetition gains over T5-search are less conspicuous than in open-ended completion.Source: §6.4 · pp. 8–9, Table 4 · p. 9
  • Generated summaries may contain hallucinations or errors, especially in named entities and digits, potentially from conversion or paragraph-embedding generation.The authors identify both sources as requiring further investigation.Source: §6.4 · p. 9

6.5 Analysis

  • On hotel-review completion, 256 PLANNER samples took 378 seconds versus 1,693 seconds for unbatched FT; length-sorted, maximally batched FT took 338 seconds.PLANNER avoids length bucketing; on CNN-DM, it generated 11,392 summaries in 8.4 GPU hours versus Genie’s 47.2 GPU hours.Source: §6.5 · p. 9
  • As diffusion proceeds, generations become more detailed and most metrics improve, but hallucinations often arise late when added details or overemphasized aspects alter correct content.The late-stage errors require further investigation.Source: §6.5 · pp. 9–10

7 Conclusion

PLANNER combines paragraph embeddings with latent diffusion and an autoregressive decoder; the appendices examine substitution-rate and latent-code choices, along with reconstruction and interpolation behavior.

7 Conclusion

  • PLANNER combines an autoencoder that compresses lengthy texts into paragraph embeddings with continuous-time diffusion that models their distribution.The conclusion also identifies possible toxicity or fairness issues in the dataset, and flags the autoregressive decoder and classifier-free guidance distribution discrepancy for further investigation.Source: §7 · p. 10

A.1 Selection of substitution rate p

  • For substitution rates p in (0, 0.7), Soverall correlates with generation PPL, but higher p can increase conversion error and weaken that correspondence.The authors use Soverall as a surrogate to avoid costly task-specific grid searches.Source: §A.1 · p. 14

A.2 Selection of number of latent code k

  • Smaller k makes diffusion easier and more computationally efficient but can lose information; increasing k brings no dramatic performance gain, so most experiments use k = 16.The latent-code selection ablation found no significant difference among first, last, or interleaved hidden vectors; the authors chose the first k.Source: §A.2 · p. 14, Table 5 · p. 14

A.3 Reconstruction, denoising and interpolation examples

  • Paragraph-embedder examples show clean and denoised reconstructions, while interpolations are usually coherent and combine content from both input sentences.The denoising reconstructions use inputs corrupted with 30% substitution noise.Source: §A.3 · pp. 14–15, Table 6 · p. 15

B Generation from PLANNER across multiple time steps

Across 256 CNN-DM summarization samples, most metrics improve as generation advances. Examples show coarse-to-fine refinement, though adding detail can introduce hallucinations.

B Generation from PLANNER across multiple time steps

C PLANNER with stochastic decoding

Stochastic decoding can improve diversity and repetition metrics, but may reduce relevance and accuracy and increase hallucinations. The reported hotel-review experiment used nucleus sampling.

C PLANNER with stochastic decoding

  • The hotel-review experiment used nucleus sampling with p = 0.92 and K = 50.The passage introduces stochastic decoding as straightforward to implement in PLANNER.Source: C PLANNER with stochastic decoding · p. 16
  • Stochastic decoding can improve diversity and repetition metrics at the expense of relevance and accuracy, with increased hallucinations.The decoder is intended to translate the latent code faithfully rather than perform compositional planning.Source: C PLANNER with stochastic decoding · p. 17

D Denoising strength comparison

PLANNER shows stronger denoising at relatively low SNR and higher AuBLEU than word-embedding-space baselines. The appendix also describes generation examples, evaluation metrics, model settings, and baseline-training costs.

D Denoising strength comparison

E Generation examples

  • Generation examples cover each task and include problematic outputs with hallucinations or errors, particularly in named entities and digits.The generation examples are presented in Tables 12 and 13.Source: E Generation examples · p. 18

F.1 Metrics

  • Evaluation uses GPT-2 large perplexity for fluency, Ent-n and DIST-n for lexical diversity, self-BLEU for cross-example diversity, and REP-n for repetition.Ent-n and DIST-n are reported only at n = 1 because of their strong correlation.Source: §F.1 · pp. 18–19

F.2 Model setups

  • PLANNER uses BERT-large and GPT-medium for its encoder and decoder, respectively, with k = 16 paragraph embeddings; its 533M-parameter diffusion model uses DDIM.Training used four Nvidia A100 GPUs; CFG weights were 2 for text completion and 5 for summarization, selected on validation data.Source: §F.2 · p. 19

F.3 Text diffusion baseline configurations

  • Training Diff-LM for 800,000 iterations on C4 would require at least 7,448 GPU hours, based on the reported ROCStories and C4 dataset-size comparison.The cited estimates are about 5 hours for 200,000 E2E iterations and over 20 hours for 800,000 ROCStories iterations.Source: §F.3 · p. 19

F.4 Ablations on DDPM, diffusion steps and noise scheduler

The ablations show trade-offs among diffusion settings, while repetitive-prompt examples contrast PLANNER with fine-tuned baselines and sampling. More diffusion steps tend to raise diversity but reduce relevance and inference speed.

F.4 Ablations on DDPM, diffusion steps and noise scheduler

  • DDIM outperforms DDPM on most metrics, while more diffusion steps typically improve diversity at a cost to relevance and inference speed.The cosine scheduler worked better than the beta linear scheduler; a noise shift of 4 improved summarization Rouge-L by around 5% versus a vanilla noise shift of 1.Source: §F.4 · p. 20

G Text completion with repetitive prompt

  • PLANNER generates significantly less repetitive text than fine-tuned baselines on ill-composed hotel-review prompts, where sampling only partly mitigates repetition.Self-reinforcement of repetition still occurs during sampling.Source: G Text completion with repetitive prompt · p. 20

H Human evaluation

The human evaluation screened judges with random questions and reported interrater agreement. The supplied materials also include a judging template and example-generation tables, but do not report human-evaluation outcomes.

H Human evaluation

  • Judges had to pass 80% of 10 random screening questions, and Krippendorff’s alpha for interrater agreement was 0.5958.The human-evaluation template is provided in Figure 7.Source: H Human evaluation · p. 20, Figure 7 · p. 21
  • Table 10 shows a CNNDM generation with a hallucinated “forensic DNA” phrase in the last two of 20 diffusion steps, while earlier generations are described as more reasonable.Table 11 indicates that stochastic decoding yields higher diversity at a cost to other metrics.Source: Table 10 · p. 22, Table 11 · p. 23
  • The supplied tables include generated and problematic examples, hotel-review completion ablations, and generations prompted with repetitive text.The captions identify Tables 12–15 but do not state their comparative outcomes.Source: Table 12 · p. 23, Table 13 · p. 24, Table 14 · p. 24
Loading 2306.02531v3…