Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!

1 Introduction

• Training uses both forward and reverse processes, but inference uses only the reverse process, so predictions rely on the model’s prior noise output rather than forward-process noise.The passage describes this discrepancy as exposure bias and links it to error accumulation as inference denoising steps grow.Source: §1 · p. 1
• Downsampling accelerates inference but can reduce performance because its shorter trajectory mismatches the full diffusion trajectory used in training.The introduction contrasts diffusion inference taking around 12 hours for 10K 128-token sentences on one A100 with CMLM taking a few minutes.Source: §1 · p. 2
• Across 6 generation tasks and 3 settings, Distance Penalty and Adaptive Decay Sampling achieve 100× speedup with performance improvement or 200× acceleration with competitive results.The methods do not change the original diffusion-model architecture or add parameters.Source: §1 · p. 2
• The preliminary study finds that injecting model-predicted noise during training improves performance, while uniform downsampling harms it and motivates adaptive sampling.These observations motivate Distance Penalty and Adaptive Decay Sampling.Source: §1 · p. 2

2.1 Diffusion Model

• The forward process gradually perturbs data toward Gaussian noise, while the reverse process predicts noise to recover the initial data distribution step by step.Training maximizes the data marginal likelihood using a simplified objective; intermediate training noise can be obtained directly without previous history.Source: §2.1 · p. 2

2.2 Diffusion Model for Text Generation

• Text generation adds an embedding step to map discrete tokens into continuous space and a final step to recover text from the continuous representation.The forward addition is q(z0 | w), and the reverse process ends with pθ(w | z0).Source: §2.2 · pp. 2–3

2.3 Inference Speedup

• Downsampling selects a subset of the full diffusion trajectory to accelerate inference, which uses a long reverse trajectory.The selected subset is denoted T′ = {t′1, t′2, ···, t′k}.Source: §2.3 · p. 3

3 Gaps between Training and Inference

• The two gaps are model-predicted noise conditioning at inference and the mismatch between its downsampled time subset and training’s full trajectory.The study is introduced to calibrate the effects of these training–inference gaps.Source: §3 · p. 3

3.2 Analysis

• Replacing forward noise with predicted noise improves performance over vanilla training, and injecting predicted noise performs better than replacing it.The study evaluates different γ1 and γ2 settings on ROC Stories using BLEU, Lexical Repetition, ROUGE, and Perplexity.Source: §3.2 · p. 3, §3.1 · p. 3, Figure 2 · p. 3
• Non-uniform sampling that allocates more steps to κ1 outperforms other tested allocations and even the original full reverse steps.The allocations keep the total number of downsampled steps fixed; the figure plots evaluation metrics against denoising steps and marks downsampled-step counts with dots.Source: §3.2 · pp. 3–4, Figure 3 · p. 4
• As time step t increases, predicted reverse noise diverges more from forward noise, and the discrepancy also grows for noise predicted across a wider step range.The authors infer that the range of downsampled reverse-noise steps should narrow as t increases.Source: §3.2 · p. 4, Figure 4 · p. 4

3.3 Potential Improvement

• The analysis concludes that predicted-noise training can mitigate the gaps, uniform downsampling harms performance, and downsampled reverse noise can be injected during training with a narrowing step range.The proposed direction combines predicted-noise injection with a range that decreases as the time step increases.Source: §3.3 · pp. 4–5

4.1 Distance Penalty

• Distance Penalty injects downsampled predicted reverse noise during post-training to bridge training–inference gaps.The method is applied to a converged model so each reverse step can produce a rational perturbation noise.Source: §4.1 · p. 5
• The Distance Penalty constrains the total noise in each step’s downsampling range using a threshold scaled by accumulated noise and the predefined inference step count.The training objective adds a penalty to the original diffusion-model objective, with γ controlling constraint strength.Source: §4.1 · p. 5

4.2 Adaptive Decay Sampling

• Adaptive Decay Sampling splits reverse diffusion into three stages and assigns more downsampled steps to stages with more added noise.The stage allocation is controlled by the cumulative noise schedule, and the total downsampled steps are divided among the stages.Source: §4.2 · p. 5

5.1 Settings

• Experiments cover six tasks across directed, open-ended, and controllable generation, using DIFFSEQ or Diffusion-LM baselines and comparisons with language models.The tasks include text simplification, paraphrase, story generation, question generation, semantic-content control, and syntax-spans control.Source: §5.1 · p. 6

5.2 Results

• Open-ended generation improves over DIFFSEQ on most metrics at 2,000 steps and over Respace on most metrics at 20 steps, except diversity metrics.The authors particularly report gains in PPL and Mauve, and better performance than BART and CMLM.Source: §5.2 · pp. 6–7, Table 1 · p. 7
• Directed-generation results show better performance than DIFFSEQ at 2,000 steps and Respace at 20 steps, especially on PPL.Table 2 reports directed text-generation results and marks the 2,000-step and 20-step settings.Source: §5.2 · p. 7, Table 2 · p. 7
• Controllable generation achieves higher Ctrl scores and lower PPL than the baselines, but has lower diversity.Table 3 reports controllable text-generation results.Source: §5.2 · p. 7, Table 3 · p. 7

5.3 Ablation Study

• Increasing the Distance Penalty weight from 1 to 6 improves fluency and precision at the cost of diversity, while the method outperforms post-training without the penalty.The authors attribute the improvement over ω = 0 to Distance Penalty rather than post-training alone.Source: §5.3 · p. 8, Table 4 · p. 8
• A larger downsampling range produces more fluent and precise results, while adaptive adjustment is needed to trade off metrics.The reported fluency and precision indicators are lower PPL and higher B-2 and R-2, respectively.Source: §5.3 · p. 8, Table 5 · p. 8
• Adaptive Decay Sampling outperforms Respace on B-2, R-2, and fluency, and DDIM on diversity; its performance decline is also smaller.The authors describe this smaller decline as evidence of robustness.Source: §5.3 · p. 8

5.4 Human Evaluation

• Human evaluators prefer the method over vanilla diffusion at both inference settings, with the advantage becoming greater as reverse steps decrease.Three annotators assessed fluency, coherence, and relevance on sampled pairs across six tasks; reported inter-annotator agreement was substantially consistent (ζ ∈[0.6, 1]).Source: §5.4 · p. 8, Table 6 · p. 8, Table 7 · p. 8

6.1 Text Generation via Diffusion Model

• Continuous-space diffusion approaches connect text and continuous representations, while discrete-space approaches use corruption processes but face one-hot-vector scaling limitations.Diffusion text-generation work also combines diffusion with iterative non-autoregressive and semi-autoregressive models.Source: §6.1 · pp. 8–9

6.2 Inference Acceleration of Diffusion Model

• Prior acceleration methods include DDIM, faster SDE solvers, distillation, and Denoising MCMC, but the cited approaches target computer vision rather than text generation.The passage frames text-generation inference acceleration as unexplored by those methods.Source: §6.2 · p. 9

6.3 Exposure Bias of Autoregressive Model

• Autoregressive exposure-bias work addresses train-test discrepancies through new objectives, regularization, or reinforcement learning to minimize expected loss.The cited strategies include Minimum Risk Training.Source: §6.3 · p. 9

• The paper targets training-inference gaps in diffusion models for text generation.Source: §7 · p. 9
• Distance Penalty and Adaptive Decay Sampling are proposed to mitigate those gaps.Source: §7 · p. 9
• Experiments across six text-generation tasks and three settings show better performance and inference speedup with the methods.Source: §7 · p. 9

• Generated results have lower diversity than language-model outputs, which the authors attribute to clamp sampling.Source: §8 · p. 9
• Post-tuning diffusion steps must match training steps, yet a gap remains between those steps and inference steps.Source: §8 · p. 9
• The diffusion model can generate incoherent sentences in open-ended tasks, a limitation the authors relate to the strong conditional-independence assumption of non-autoregressive models.Source: §8 · p. 9

Ethics Statement

• The authors replace corpus names with special placeholders to mitigate problematic generation biases, while warning that the issue cannot be fully solved and urging cautious real-world use.They also state that the data are publicly available and that the same evaluation scripts are used across comparisons.Source: Ethics Statement · p. 9

A Preliminary of Diffusion Model

• Training obtains intermediate forward-process states directly through accumulated multiplication, with 1 − ᾱ_t denoting the accumulated-noise variance.The training objective is to maximize the marginal likelihood, and the variational lower bound is introduced as its formulation.Source: A Preliminary of Diffusion Model · p. 13

B Diffusion Models for Text Generation

• For text generation, discrete text is mapped into continuous embeddings and recovered by selecting the most probable word; clamping commits predictions to their nearest word vectors during reverse steps.For sequence-to-sequence tasks, partially noising perturbs the target while replacing the source-text portion with its anchor vector after each perturbation.Source: B Diffusion Models for Text Generation · pp. 13–14

C Implementation Details

• The implementation-details section introduces dataset processing, baseline settings, and evaluation metrics.The supplied passage provides no further specifics for this section.Source: C Implementation Details · p. 14

D.2 Extensive Trials

• Models trained for 40K, 80K, and 120K steps show a similar predicted-versus-forward-noise distance trend to the earlier result, supporting predicted-noise injection early in training.The comparison also includes a randomly initialized model trained for zero steps.Source: §D.2 · p. 15, Figure 7 · p. 15

E Adaptive Decay Sampling

• ADS divides downsampled steps among three denoising-stage subsets, weighted by η_i, and is compared with DDIM, which skips reverse steps using a non-Markov sampling process.The DDIM comparison sets σ_t = 0 for every time step.Source: E Adaptive Decay Sampling · p. 15, §E.3 · p. 16

E.1 Noise Scheduler

• The square-root noise scheduler increases noise rapidly during the first 500 forward steps, then more slowly later; across three diffusion stages, model training uses more steps to solve high-noise cases.Source: §E.1 · p. 15

E.2 Correlation between ADS and Noise Scheduler

• ADS allocates more attention to high-noise denoising than Respace, and its denoising curve drops rapidly in Stage 3.The authors say the curve supports their preliminary observation that assigning more downsampled steps to early denoising can improve performance.Source: §E.2 · p. 16

F.3 Sampling Strategy Comparison

• ADS has a lower metric-change rate than Respace as downsampled inference steps decrease, indicating greater robustness to fewer steps.Figure 11 plots the change rate for each evaluation metric.Source: Figure 11 · p. 16

F.4 Speed up the Training

• Distance Penalty post-training improves ROC performance even after a 40K-step warm-up followed by 30K post-training steps, with larger improvements from more post-training.The experiment uses MBR candidate size |S| = 1 for convenience.Source: §F.4 · p. 17

F.5 Robustness of Adaptive Decay Sampling

• ADS has a lower metric-change rate than Respace as the number of downsampled inference steps decreases, indicating greater robustness.The rate of change is plotted for each evaluation metric in Figure 11.Source: §F.5 · p. 17

G Human Evaluation

• Annotators compare a prompt with two model outputs and may choose a tie; three annotators each receive $0.05 per pair, with about 30 seconds allotted per comparison.The authors state that annotators work independently and that the process is fair.Source: G Human Evaluation · p. 17

H Case Study

• The case-study examples are sampled from 2,000-step and 20-step diffusion models and their corresponding Distance Penalty and ADS variants; colors mark prompt-related phrases, contradictions, and grammatical errors.Green marks key prompt-related phrases, red marks contradictions, and blue marks serious grammatical errors.Source: H Case Study · p. 17
• The case study reports more fluent, higher-quality text with the proposed methods, while the original model often produces repetitions or meaningless tokens.The authors caution that GPT-2 PPL may favor repetitive text, while meaningless or hard-to-read text may receive better Distinct and Lexical Repetition scores.Source: H Case Study · p. 18