PaperlayerView exact revision on arXiv

Source-linked AI summary

Prefix-Denoising Consistency: Test-Time Verification for Diffusion Language Models

Yuki Ichihara, Naoto Iwase, Mohammad Atif Quamar, Junpei Komiyama

arXiv:2608.25311v1cs.LG

TL;DR

DLMs' low-temperature repeated generations can follow near-identical denoising trajectories, making naive self-consistency a limited verification signal. PDC instead regenerates masked suffixes from fixed prefixes, using the greater reproducibility of correct trajectories to improve performance across reasoning benchmarks, with robustness across unmasking strategies and hyperparameter choices.

  • Problem

    Low-temperature DLM decoding can yield near-identical denoising trajectories, making agreement among repeated full generations a limited independent check for answer reliability.

  • Method

    PDC fixes an exact prefix, remasks the remaining positions, re-denoises them at several keep rates, and aggregates regenerated answers by majority vote.

  • Results

    Across math and commonsense benchmarks, PDC consistently improves over the initial sample, compares favorably with independent generations under compute constraints, and remains robust across unmasking strategies and hyperparameter choices.

  • Takeaways & Limitations

    Prefix-conditioned regeneration is an effective DLM-specific primitive for test-time verification and may help improve diffusion language model reliability.

  • Takeaways & Limitations

    The theoretical benefit analysis assumes a binary answer space with no null answers, while the method itself treats the DLM as a black-box generator.

Abstract

from arXiv · show

Diffusion Language Models (DLMs) have recently become increasingly competitive with autoregressive (AR) models, and even outperform them on certain tasks. Unlike AR models, DLMs produce output through iterative denoising without a left-to-right order. To further improve the performance of DLMs, we introduce PDC (\emph{Prefix-Denoising Consistency}), a test-time self-verification method for DLMs. PDC exploits a distinctive test-time signal in DLMs under prefix conditioned regeneration, correct trajectories are more stable and reproducible than incorrect ones. Concretely, given an initially generated sample, PDC splits the sentence at an intermediate position and regenerates the remaining tokens conditioned on the fixed prefix. Across mathematical reasoning and commonsense reasoning benchmarks, PDC consistently improves upon the initial sample, outperforms independent generations under a computational constrained comparison, and is robust to different unmasking strategies and parameter settings. These results highlight prefix-conditioned regeneration as an effective DLM-specific primitive for test-time verification.

1 INTRODUCTION

DLMs generate outputs through iterative denoising, creating verification challenges that naive repeated sampling may not solve. PDC addresses this by regenerating suffixes from fixed prefixes and exploits greater reproducibility of correct answers to improve test-time verification.

  • Motivation: DLMs iteratively denoise outputs rather than generating tokens strictly left to right, enabling global revision but complicating direct transfer of autoregressive verification methods.Low-temperature DLM decoding can produce near-identical denoising trajectories, so repeated generations may provide limited independent evidence.
  • PDC: PDC keeps an exact prefix, remasks the remaining positions, and re-denoises them to generate structured alternative trajectories.It repeats this intervention at several keep rates and aggregates extracted answers by majority vote, excluding the initial answer from the vote.
  • Diagnostic signal: Initially correct answers are more reproducible than initially wrong answers under prefix-conditioned regeneration, motivating PDC's verification signal.Figure 2 reports the reproduction-rate comparison under ρ = 0.5 regeneration across generation lengths L = 128, 256, 512 on three math benchmarks.
  • Results: In the full diffusion setting across math and commonsense benchmarks, PDC consistently improves over the initial sample and TIF.TIF is a temporal-voting baseline that extracts answers from intermediate denoising steps.
  • Results: PDC outperforms standard majority voting with a smaller total denoising-step budget and remains improved across different unmasking schemes and settings.These comparisons address both compute efficiency and robustness to design choices.

2 PREFIX-DENOISING CONSISTENCY

DLMs generate sequences through iterative, non-left-to-right denoising, while PDC verifies an initial answer by regenerating masked suffixes under fixed prefixes and voting over the regenerated answers. The method is motivated by the finding that correct answers are more reproducible than incorrect answers, with improvement characterized by the reproduction gap.

  • 2.1 STANDARD DLM INFERENCE TIME: DLMs iteratively predict masked output positions in parallel, unmasking subsets according to a denoising schedule until the sequence is complete.Positions may be resolved in arbitrary rather than left-to-right order.
  • 2.2 PREFIX-DENOISING CONSISTENCY (PDC): PDC holds a prefix of the initial output, remasks the remaining positions, regenerates them with the same DLM, and majority-votes over regenerated answers.The initial answer is recorded for diagnostics but excluded from the final vote.
  • 2.2 PREFIX-DENOISING CONSISTENCY (PDC): The fixed keep rates 0.1, 0.5, and 0.9 probe weak, intermediate, and strong prefix conditioning around the same initial output.The rates require three regenerations while varying how much of the original trajectory remains fixed.
  • 2.2 PREFIX-DENOISING CONSISTENCY (PDC): Correct initial answers are more likely to be preserved than incorrect initial answers are to be repeated across experiment settings.These reproduction rates provide the signal PDC uses to assess initial-answer reliability.
  • 2.3 WHEN DOES PDC IMPROVE ACCURACY?: A sufficient improvement condition is that the majority-vote probability V3 increases from same-wrong-answer persistence rates R− to correct-answer reproduction rates R+, with strict inequality for at least one regeneration.The argument assumes a binary answer space and conditionally independent regenerated answers.
  • 2.3 WHEN DOES PDC IMPROVE ACCURACY?: PDC improves upon the initial answer precisely when the probability of correcting wrong initial answers exceeds the probability of spoiling correct initial answers.Under the binary-answer analysis, improvement is equivalent to V3(R+) > V3(R−).

3 EXPERIMENTS

Experiments test PDC’s diagnostic signal, accuracy gains, compute efficiency, and robustness across DLM models, datasets, decoding settings, and unmasking rules.

  • 3 EXPERIMENTS: The experiments evaluate reproduction rates, accuracy against INIT and TIF, compute-constrained self-consistency, and robustness across unmasking and inference settings.Models include Dream-7B and LLaDA-family systems, with full-diffusion and semi-AR evaluations.
  • 3.1 CORRECT ANSWERS ARE MORE REPRODUCIBLE THAN WRONG ANSWERS: Initially correct answers are typically reproduced under prefix-conditioned regeneration, whereas initially wrong answers are less likely to persist as the same wrong answer.This diagnostic comparison motivates PDC’s use of regeneration consistency for verification.
  • 3.2 PDC IMPROVES THE INITIAL SAMPLE: PDC attains the best or tied-best accuracy in all 15 settings and improves over the selected INIT baseline in 14, with gains up to +5.68 accuracy points.The generation length is selected using INIT accuracy rather than optimized for PDC.
  • 3.3.1 PDC OUTPERFORMS SELF-CONSISTENCY AT LOWER INFERENCE COST: PDC improves over INIT x4 in 13 of 15 completed model–dataset pairs and ties one, despite INIT x4 using a 4T denoising-step budget.PDC uses one initial generation followed by three prefix-conditioned regenerations.
  • 3.3.2 ROBUSTNESS: PDC remains effective under semi-AR inference, more stochastic decoding at τ = 1.0, and alternative unmasking rules.Under alternative unmasking on MATH-500, PDC achieves the best accuracy across all reported model and length settings.

4 CONCLUSION

The paper presents PDC as a DLM-specific test-time verification method based on prefix-conditioned regeneration. Across math and commonsense benchmarks, it improves initial samples, compares favorably with independent generations under compute constraints, and remains robust across settings.

  • 4 CONCLUSION: PDC is a test-time self-verification method that exploits the greater stability and reproducibility of correct reasoning trajectories under prefix-conditioned regeneration.It is presented as a DLM-specific approach.
  • 4 CONCLUSION: Across math and commonsense benchmarks, PDC consistently improves over the initial sample and compares favorably with independent generations under compute constraints.The conclusion reports the pattern across both benchmark categories.
  • 4 CONCLUSION: PDC remains robust across unmasking strategies and hyperparameter choices.The paper contrasts PDC’s black-box test-time procedure with related methods that modify models or inference schedules.

B FULL REPRODUCTION RATES RESULTS

Across CSQA and SQA, correct-answer retention is generally higher than persistence of the same wrong answer across keep rates 0.1, 0.5, and 0.9.

  • Correct-answer retention is generally higher than same-wrong persistence across CSQA and SQA and keep rates 0.1, 0.5, and 0.9.These additional datasets and rates are reported in the full conditional-stability diagnostics.

C FULL-DIFFUSION ACCURACY SWEEP

The full-diffusion sweep evaluates PDC across completed generation lengths and reports accuracy changes relative to INIT, alongside TIF comparisons.

  • PDC consistently improves INIT accuracy across completed generation lengths L = 128, 256, 512.The results are described as robust to block-length selection within the full-diffusion sweep.
  • PDC is evaluated with three suffix-regenerated completions using keep rates 0.1, 0.5, and 0.9.Signed values report accuracy-point changes relative to INIT; TIF changes are averaged over Fixed, Linear, and Exp. α = 5 variants.

D SEMI-AR LLADA-FAMILY RESULTS

In the semi-autoregressive blockwise setting, the proposed method remains effective and improves INIT accuracy for LLaDA-family models across generation lengths.

  • The semi-autoregressive runs use block length 32, with generation lengths 128, 256, and 512.Diffusion is performed independently within each block.
  • The proposed method improves INIT accuracy for LLaDA-family models across generation lengths in the semi-autoregressive blockwise setting.This indicates that its benefits are not limited to the full-diffusion setting used for the main claims.

E CASE STUDY: HOW PREFIX REGENERATION REPAIRS ERRORS

Prefix-conditioned regeneration can preserve correct trajectories while allowing incorrect intermediate steps to be replaced, including in semi-autoregressive blockwise evaluation and illustrative math cases.

  • Mechanism: High R+ and low R− mean correct initial answers tend to remain correct, while wrong answers often do not reproduce the same mistake.Escaping the same wrong answer does not guarantee correctness, but remasking an erroneous intermediate can permit a corrected continuation.
  • Semi-AR evaluation: In the semi-autoregressive evaluation, Table 6 uses block length 32 and generation lengths 128, 256, and 512, with percentages and within-row accuracy changes.Bold marks the largest delta within each row, while negative deltas are shown in red.
  • Case study: Table 8 presents two MATH-500 examples where remasking the wrong intermediate step avoids the same mistake and produces a corrected continuation.Incorrect initial continuations are marked red and corrected regenerated continuations green.
  • Case study: The figure-skater example changes from an initial south answer to east after regeneration recomputes 2250 − 6 · 360 = 90 degrees.The initial sample made a local modulo-arithmetic error; regeneration corrected the suffix.

F MODEL AND DATASET SUMMARY

The experiments use Dream-v0-Instruct-7B, LLaDA-1.5, and LLaDA-8B-Instruct, with datasets spanning mathematical, multiple-choice, and binary reasoning tasks. The main paper focuses on full diffusion, while semi-autoregressive LLaDA-family runs are reported separately.

  • Math-style datasets use canonical answer extraction with symbolic or numeric equivalence when applicable.
  • Multiple-choice and binary reasoning datasets use canonicalized option extraction.

G RESULTS AT DIFFERENT SAMPLING TEMPERATURES

PDC remains effective when sampling becomes more stochastic, achieving the highest accuracy in most tested settings and a larger mean margin over four independent generations. Its gains are strongest on mathematical reasoning tasks, though it trails the comparison baseline on all tested CSQA settings.

  • 12 of 15 model–dataset settings achieve the highest numerical accuracy with PDC at τ = 1.0.PDC also outperforms INIT ×4 in the same 12 settings.
  • PDC exceeds INIT ×4 by 4.98 accuracy points averaged uniformly over 15 settings at τ = 1.0.The largest dataset-level mean gains are 11.45 points on GSM8K and 7.47 points on MATH-500.
  • PDC’s unweighted mean accuracy changes from 65.35 at τ = 0.2 to 66.16 at τ = 1.0, while INIT and INIT ×4 decrease.The mean margin over INIT ×4 increases from 2.11 to 4.98 points.
  • On all three CSQA settings, PDC trails INIT ×4 by 0.74–2.12 points.The reported results therefore identify mathematical reasoning tasks as the stronger setting for PDC at the tested higher temperature.

H REPRODUCIBILITY DETAILS

The experiments use low-confidence and entropy-based unmasking with temperature 0.2 and generation lengths matched to denoising budgets. Outputs follow explicit reasoning and answer-format instructions, while extraction and evaluation apply task-specific normalization and equivalence checks.

  • Inference settings: The evaluation uses entropy-based and low-confidence unmasking at token temperature 0.2.Generation length L equals the denoising step budget T ∈ {128, 256, 512}.
  • Inference settings: The main tables use block length equal to generation length, while semi-autoregressive LLaDA-family runs with block length 32 appear in Appendix D.The main paper exclusively considers the full diffusion setting.
  • Regeneration budget: During full-diffusion regeneration, the effective denoising budget equals the remasked suffix length when T = L.The general proportional rule uses Tρ = max{1, round(TLρ/L)}.
  • Regeneration budget: Dream uses the floor version Tρ = max{1, ⌊TLρ/L⌋}, differing from the rounded rule by at most one denoising step.Reported tables use saved records for the corresponding run.
  • Prompt and output format: Math prompts require step-by-step reasoning with the final answer inside \boxed{}, while multiple-choice prompts require reasoning before a single boxed option letter.The multiple-choice setting is therefore not direct-answer-only.
  • Answer evaluation: Math answers are normalized and compared using exact strings, SymPy equivalence when needed, or 10^-6 numeric tolerance; multiple-choice outputs are canonicalized to letters.Null initial predictions count as incorrect for INIT, while conditional persistence measures exclude them.
Loading 2608.25311v1…