PaperlayerView exact revision on arXiv

Source-linked AI summary

DINOISER: Diffused Conditional Sequence Learning by Manipulating Noises

Jiasheng Ye, Zaixiang Zheng, Yu Bao, Lihua Qian, Mingxuan Wang

arXiv:2302.10025v2cs.CLcs.AIcs.LG

TL;DR

Diffusion models applied to discrete sequences still struggle to achieve satisfactory generation quality, even when tokens are embedded in continuous spaces. DINOISER manipulates noise scales during training and inference, and reports consistent improvements over prior diffusion-based models across conditional sequence benchmarks.

  • Problem

    Embedding discrete tokens into continuous spaces does not fully remove discreteness, and prior diffusion-based sequence models fall short of satisfactory conditional generation quality.

    Source: §1 · p. 1

  • Method

    DINOISER adaptively clips training noise scales and amplifies noise during inference to encourage the denoiser to use source conditions.

    Source: §1 · p. 2

  • Results

    DINOISER reports consistent improvement over previous diffusion-based sequence models across several conditional sequence modeling benchmarks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors conclude that DINOISER makes training more effective and enables better use of source conditions for prediction.

    Source: §7 · p. 13

Abstract

from arXiv · show

While diffusion models have achieved great success in generating continuous signals such as images and audio, it remains elusive for diffusion models in learning discrete sequence data like natural languages. Although recent advances circumvent this challenge of discreteness by embedding discrete tokens as continuous surrogates, they still fall short of satisfactory generation quality. To understand this, we first dive deep into the denoised training protocol of diffusion-based sequence generative models and determine their three severe problems, i.e., 1) failing to learn, 2) lack of scalability, and 3) neglecting source conditions. We argue that these problems can be boiled down to the pitfall of the not completely eliminated discreteness in the embedding space, and the scale of noises is decisive herein. In this paper, we introduce DINOISER to facilitate diffusion models for sequence generation by manipulating noises. We propose to adaptively determine the range of sampled noise scales for counter-discreteness training; and encourage the proposed diffused sequence learner to leverage source conditions with amplified noise scales during inference. Experiments show that DINOISER enables consistent improvement over the baselines of previous diffusion-based sequence generative models on several conditional sequence modeling benchmarks thanks to both effective training and inference strategies. Analyses further verify that DINOISER can make better use of source conditions to govern its generative process.

1 Introduction

Diffusion models for discrete sequences remain short of satisfactory generation quality despite embedding tokens in continuous spaces, because discreteness persists and noise scale matters. DINOISER manipulates noise scales in training and inference; experiments report strong performance across conditional sequence tasks.

  • Embedding discrete tokens into continuous space enables diffusion for sequence generation but still falls short of competitive performance on conditional tasks.Source: §1 · p. 1
  • Residual discreteness in embeddings creates low-density regions, especially under small noise, while larger embedding dimensions make eliminating discreteness harder.The authors identify small training noises and a non-adaptive noise schedule as problems, and say larger inference noises can help models use source conditions.Source: §1 · pp. 1–2
  • DINOISER uses adaptive noise-scale clipping to avoid small-noise training and amplifies noise during inference to encourage use of source conditions.Source: §1 · p. 2
  • Experiments report strong DINOISER performance across bilingual and multilingual machine translation, text simplification, and paraphrasing, from low- to high-resource settings.Ablations attribute considerable gains to both the training and inference approaches, and further analysis finds that the condition-enhanced denoiser better uses source conditions.Source: §1 · p. 2

2 Background

The background describes conditional sequence generation and diffusion, then explains how diffusion models are adapted to sequence embeddings and used for conditional generation. Sampling iteratively reverses the diffusion process from noise toward a target sequence.

  • Conditional sequence learning generates a target sequence from source conditions, either autoregressively, in parallel under token independence, or through iterative refinement.In iterative refinement, each cycle uses the source and the preceding intermediate prediction.Source: §2 · p. 2
  • A diffusion process corrupts an initial representation with Gaussian noise according to a schedule, with increasing timesteps producing more corruption.Source: §2 · p. 2
  • For sampling, a learned model estimates the clean representation, and an ODE solver such as DDIM steps from Gaussian noise toward the final prediction.Source: §2 · p. 3
  • DiffusionLM applies diffusion in continuous embedding space and formulates training as a variational autoencoder objective with diffusion and reconstruction terms.The source condition can be incorporated into the denoiser while the diffusion process remains unchanged.Source: §2 · p. 3

3 The Pitfall of Discreteness: The Noise Scale Matters

Preliminary studies link diffusion models’ weaknesses in conditional sequence learning to residual discreteness in token embeddings and the noise scales used in training and inference. The authors motivate avoiding small training noises and amplifying noise at inference to improve source-condition use.

3 The Pitfall of Discreteness: The Noise Scale Matters

  • As noise scales decrease, nearest-neighbor recovery approaches 100%, while larger scales make predictions more faithful to source conditions.The authors connect low-noise recovery to hallucinations; Table 1 illustrates source-irrelevant DiffusionLM outputs alongside DINOISER predictions.Source: Figure 1 · p. 4, §3 · pp. 4–5, Table 1 · p. 5
  • DiffusionLM trails CMLM on IWSLT14 DE→EN, while its loss approaches zero at small noise scales, consistent with easy recovery around separated embedding modes.The comparison uses the vanilla conditional DiffusionLM setup and its sqrt noise schedule.Source: §3 · pp. 3–4, Figure 1 · p. 4
  • Recovering embeddings by nearest neighbor becomes easier as embedding dimensions grow, indicating that scaling up the space makes discreteness harder to eliminate.The study reports high nearest-neighbor recovery even at considerably large noise scales in higher-dimensional spaces.Source: §3 · p. 4, Figure 1 · p. 4

C1. We should not train on too small noise scales

  • Small-noise training can leave low-density gaps between token modes that may be sampled at inference, causing error accumulation and impeding conditional learning.For an ambiguous translation, averaging the embeddings of two possible target tokens places the prediction in the blank region between them.Source: C1. We should not train on too small… · p. 5
  • The authors recommend avoiding small training noises, adapting noise schedules to embedding dimensionality, and amplifying inference noise to encourage source-condition use.They note that existing methods use fixed schedules across embedding dimensions, which they describe as lacking scalability.Source: C1. We should not train on too small… · p. 5

4 DINOISER

DINOISER manipulates noise scales in both training and inference: adaptive clipping avoids small-noise training, while CEDI uses larger timesteps to encourage source-conditioned predictions.

4.1 Noise Scale Clipping: Manipulating Noises for Counter-Discreteness Training

  • Clipping adaptively sets the minimum noise scale from embedding distances, then trains only at noise scales at least that threshold.The threshold is updated during training based on minimum pairwise distances within the vocabulary.Source: §4.1 · pp. 5, 7

4.2 CEDI: Manipulating Noises for Condition-Enhanced Sampling

  • At the larger timestep 0.995, predictions are closer to the ground-truth target embedding and farther from a misleading negative sample.The synthesis experiment compares predictions at τ=t and τ=0.995.Source: §4.2 · p. 7
  • CEDI feeds the denoiser larger timesteps during sampling to encourage it to use source conditions in its predictions.The sampler uses one timestep sequence that decreases toward zero and a second that ends at a large timestep.Source: §4.2 · p. 7

5 Experiment

Experiments evaluate DINOISER on translation, simplification, and paraphrasing, finding strong results across tasks and examining how noise clipping and condition-enhanced sampling affect performance.

5.1 Experimental Setup

  • Experiments cover six machine translation tasks plus Wiki simplification and QQP paraphrasing, comparing against Transformer, CMLM, and diffusion-based baselines.Diffusion methods use MBR decoding; the authors also report models with and without knowledge distillation.Source: §5.1 · p. 8

5.2 Main Results

  • DINOISER achieves state-of-the-art results among diffusion-based machine translation models and surpasses DiffuSeq on text simplification and paraphrasing.The reported comparisons appear in Table 2 for translation and Table 3 for Wiki and QQP.Source: §5.2 · p. 9, Table 2 · p. 9, Table 3 · p. 10
  • DINOISER surpasses CMLM on almost all large- and small-scale translation scenarios, while vanilla DiffusionLM struggles more with larger datasets.The authors describe this as evidence that DINOISER improves the scalability of diffusion-based sequence learners.Source: §5.2 · p. 9
  • DINOISER matches or exceeds CMLM with MBR=1 and needs fewer NFEs, with 20 steps costing 1% to 10% of prior methods’ computation and latency.The paper characterizes the result as greater accuracy and efficiency than previous diffusion-based sequence models.Source: §5.2 · p. 10

5.3 Effect of Noise Scale Clipping for Training Ablation study on noise clipping.

  • Noise scale clipping consistently improves performance; adaptive clipping balances the degradation seen when fixed thresholds are too small or too large.Replacing the linear noise schedule with the sqrt schedule has negligible final-performance effect, but makes clipping’s improvement smaller.Source: §5.3 · p. 10, Figure 4 · p. 10

5.4 Effect of Condition Enhancement for Sampling

  • CEDI outperforms DDIM, especially with small MBR candidate sizes, and maintains high source contribution while its prediction accuracy rises steadily.DDIM’s accuracy fails to improve early in sampling and remains behind CEDI; the authors report a correlation between higher source contribution and performance improvement.Source: §5.4 · pp. 10–11, Figure 5 · p. 11
  • In multilingual one-to-many translation, CEDI has higher language accuracy than DDIM, and full DINOISER produces fluent outputs where DDIM may miss the target-language condition.The experiments also test many-to-one translation with source sentences from four languages.Source: §5.4 · p. 11, Table 5 · p. 11, Table 6 · p. 12

6 Related Work

Related work spans non-autoregressive sequence generation and diffusion-based sequence learning. DINOISER differs by using continuous diffusion and studying both training and inference.

Non-autoregressive Sequence Generative Models

  • Non-autoregressive models predict tokens in parallel but assume conditional independence, risking ignored target dependencies and multimodality-related quality problems.Iterative refinement mitigates the strong independence assumption by repeatedly refining a sequence.Source: §6 · p. 12
  • Some iterative-refinement approaches formalize refinement as discrete-state Markov processes, whereas this paper focuses on continuous diffusion in embedding space.Source: §6 · p. 12

Diffusion Models for Sequence Learning

  • Prior sequence-diffusion work explores modeling components, pretrained latent spaces, or discrete token-prediction objectives; DINOISER emphasizes both training and inference.Source: §6 · p. 12

7 Conclusion

DINOISER treats noise schedules as central to conditional sequence learning and determines suitable noise scales for training and inference. It reports more effective training and better use of source conditions, with considerable performance improvements.

7 Conclusion

  • DINOISER determines suitable noise scales for both training and inference in conditional sequence learning.The paper frames noise schedules as crucial to this task, based on a systematic empirical study.Source: §7 · p. 12
  • The paper uses systematic empirical study to examine the role of noise schedules in conditional sequence learning.Source: §7 · p. 12
  • DINOISER makes training more effective and enables models to better utilize source conditions for prediction.The authors report that these outcomes lead to considerable performance improvements.Source: §7 · p. 13

A More Results on Machine Translation

With knowledge distillation, DINOISER has a small performance gap from the autoregressive Transformer on machine translation. The section also reports tokenized BLEU results to facilitate comparisons with prior studies.

A More Results on Machine Translation

B Implementation Details

The implementations use Transformer-based architectures, with a smaller setup for IWSLT14, and apply self-conditioning and optional knowledge distillation. Inference uses beam-based decoding, while preprocessing and batch sizes vary by dataset.

B Implementation Details

  • Implementations use Transformer-base except on IWSLT14, which uses four attention heads and 1024-dimensional feedforward layers; diffusion embeddings are 16-dimensional there and 64-dimensional elsewhere.Source: B Implementation Details · p. 17
  • The method applies self-conditioning and is trained both with and without knowledge distillation.Source: B Implementation Details · p. 17
  • Inference uses length beam 5 for non-autoregressive machine translation except CDCD and DiffuSeq, while diffusion methods use MBR decoding and DiffusionLM and DINOISER sample with 20 steps.Source: B Implementation Details · pp. 17–18
  • Preprocessing follows fairseq instructions for IWSLT14, uses preprocessed WMT14/WMT16 data, and applies byte-pair encoding to Wiki and QQP.Source: B Implementation Details · p. 18
  • Training batches are 128K for WMT14/WMT16 and 32K for other datasets; checkpoint averaging is not applied because it was empirically unnecessary.Source: B Implementation Details · p. 18

C Relationship Between Different Noise Schedules and Time Samplers

The training objective can be expressed using noise scales rather than timesteps. Different noise schedules and time samplers are interchangeable through different weighting functions.

C Relationship Between Different Noise Schedules and Time Samplers

D Effect of Sizes of Length Beam and MBR

DINOISER’s validation performance depends on tuning length beams and MBR size: performance rises and then falls as beam size increases, while MBR decoding further improves each beam setting. The two sampling choices have complementary effects and offer diversity and computational-budget advantages over deterministic or maximum-length-limited decoding.

D Effect of Sizes of Length Beam and MBR

Loading 2302.10025v2…