PaperlayerView exact revision on arXiv

Source-linked AI summary

dVoting: Fast Voting for dLLMs

Sicheng Feng, Zigeng Chen, Xinyin Ma, Gongfan Fang, Xinchao Wang

arXiv:2602.12153v1cs.CLcs.AI

TL;DR

Test-time scaling in dLLMs faces high inference costs; dVoting uses consistency-based voting to refine generations. Evaluations report consistent performance gains across reasoning benchmarks.

  • Problem

    Test-time scaling in dLLMs incurs high inference cost, motivating a faster approach to improve reasoning performance.

    Source: §6 · p. 8

  • Method

    dVoting identifies uncertain tokens through consistency analysis, iteratively remasks and regenerates them, and aggregates candidate answers by voting.

    Source: §1 · p. 2

  • Results

    dVoting consistently improves performance across benchmarks on LLaDA and Dream, with LLaDA gains of 6.22%–7.66% on GSM8K.

    Source: §5.2 · p. 6

  • Takeaways & Limitations

    The work provides a foundation for future test-time scaling in dLLMs.

    Source: §6 · p. 9

Abstract

from arXiv · show

Diffusion Large Language Models (dLLMs) represent a new paradigm beyond autoregressive modeling, offering competitive performance while naturally enabling a flexible decoding process. Specifically, dLLMs can generate tokens at arbitrary positions in parallel, endowing them with significant potential for parallel test-time scaling, which was previously constrained by severe inefficiency in autoregressive modeling. In this work, we introduce dVoting, a fast voting technique that boosts reasoning capability without training, with only an acceptable extra computational overhead. dVoting is motivated by the observation that, across multiple samples for the same prompt, token predictions remain largely consistent, whereas performance is determined by a small subset of tokens exhibiting cross-sample variability. Leveraging the arbitrary-position generation capability of dLLMs, dVoting performs iterative refinement by sampling, identifying uncertain tokens via consistency analysis, regenerating them through voting, and repeating this process until convergence. Extensive evaluations demonstrate that dVoting consistently improves performance across various benchmarks. It achieves gains of 6.22%-7.66% on GSM8K, 4.40%-7.20% on MATH500, 3.16%-14.84% on ARC-C, and 4.83%-5.74% on MMLU. Our code is available at https://github.com/fscdc/dVoting

1 Introduction

dVoting is a training-free test-time voting strategy for dLLMs that exploits flexible remasking to refine uncertain tokens and aggregate candidate answers. The introduction reports gains across reasoning benchmarks and presents the method as a foundation for efficient test-time scaling in dLLMs.

Motivation

  • Unlike training-time reasoning enhancements, dVoting targets inference, where prior dLLM test-time methods HEX (Lee et al., 2025) and RFG (Chen et al., 2025a) typically increase computation.HEX varies block sizes to activate implicit semi-autoregressive experts, while RFG guides generation at the logit level with an additional fine-tuned model.Source: §1 · p. 1
  • Repeated tokens frequently recur across samples for the same input, and dLLMs can remask and regenerate arbitrary decoded tokens at arbitrary positions to reduce this redundancy.The introduction points to Figure 2(b) and Table 1 for the observation and its quantification.Source: §1 · p. 1

Method

  • dVoting identifies uncertain tokens through consistency analysis, iteratively remasks and regenerates them, then aggregates candidate answers by voting.It preserves consistent tokens from previous generations, remasks the rest for subsequent sampling, and can terminate early when candidate answers meet consistent criteria.Source: §1 · p. 2, Figure 1 · p. 2

Experiments

  • On LLaDA, dVoting yields performance gains of 6.22%–7.66% on GSM8K and 4.40%–7.20% on MATH500.Experiments across two popular dLLMs report improvements on mathematical, scientific, and general reasoning benchmarks, alongside a leading performance–efficiency trade-off and robustness under varied configurations.Source: §1 · p. 2

Contributions

  • The paper presents dVoting as a training-free voting strategy and the first baseline for efficient test-time scaling in dLLMs.Its empirical study identifies repeated tokens across sampling runs and connects that observation to dLLM remasking.Source: §1 · p. 2

2 Related Work

The related work presents dLLMs as a growing parallel-generation paradigm and identifies efficient test-time scaling as an underexplored gap.

Overview of Diffusion Language Models

Test-Time Scaling in Language Models

3 Preliminaries

Masked diffusion language models iteratively denoise corrupted sequences, predicting masked tokens in parallel. This makes dLLMs suitable for parallel test-time scaling, which the section presents as a training-free alternative to RL, albeit with added inference overhead.

Preliminaries

  • MDLMs iteratively reconstruct sequences corrupted by independently masking positions, while their denoising model predicts masked tokens conditionally independently given the corrupted sequence.The masking intensity t ranges from 0 to 1.Source: §3 · p. 3
  • MDLM training focuses on reconstructing original content at masked tokens, using a reconstruction loss that upper-bounds negative log-likelihood.Source: §3 · p. 3

Properties of dLLMs Fit Parallel Test-Time Scaling

  • During inference, dLLMs predict all masked positions in parallel, selectively remask uncertain tokens, and reuse reliable context across sampling runs to refine earlier decisions.The section says this parallel decoding also reduces inference latency.Source: §3 · p. 3

Parallel Test-Time Scaling Can Work as an Alternative to Reinforcement Learning in dLLMs

  • Parallel test-time scaling with voting achieves performance comparable to advanced RL-based methods across reasoning and general tasks, but increases inference overhead.The section frames voting as selecting the most consistent output from multiple samples, approximating RL’s effect of favoring higher-quality trajectories; this perspective is also supported by Yue et al. (2025).Source: §3 · p. 4

4 Methods

The methods section motivates DVOTING as a cost-effective alternative to RL in dLLMs, then uses answer- and token-level consistency to design remask sampling.

4.1 Key Observations

  • 84.58% of samples correctly solved by both baseline and voting fell in the 4/5 or 5/5 consistency bins, while 66.07% in other categories received 1/5 or 2/5 votes.The authors interpret higher consistency as characteristic of easier questions, where sampling redundancy can be reduced with minimal impact on performance.Source: §4.1 · p. 4
  • NUPR@2 is around 50% and NUPR@3 around 20%, indicating substantial token-level redundancy across GSM8K, MATH500, and ARC-C at different generation lengths.NUPR@k measures the fraction of token positions where at least k of K sampled answers share an identical token.Source: §4.1 · pp. 4–5, Table 1 · p. 4

4.2 Remask Sampling

  • dVoting stops sampling when answers are consistent, preserves agreed tokens and tokens from high-consistency answers, then resamples uncertain positions conditioned on preserved tokens until the voting answer converges.The design combines answer-level early stopping with selective token remasking rather than generating a full response at each sampling action.Source: §4.1 · p. 4, §4.2 · p. 5
  • For parallel decoding, dVoting commits token positions whose entropy falls below threshold α, set to 0.3 in the main experiments.An ablation study of the threshold is provided.Source: §4.2 · p. 5

5 Experiments

Across LLaDA and Dream experiments, DVOTING improves benchmark performance while using fewer inference steps than test-time scaling baselines. Its computation adapts to problem difficulty, and ablations show robustness across several hyperparameter settings.

5.1 Experimental Setups

  • Experiments compare original results and majority voting with HEX and RFG across mathematical, scientific, and general reasoning benchmarks including GSM8K, MATH500, ARC-C, MMLU, and GPQA.The study evaluates LLaDA and Dream models using zero-shot simple-eval prompts that elicit step-by-step reasoning.Source: §5.1 · pp. 5–6

5.2 Main Results

  • On Dream, DVOTING improves performance across all benchmarks with the fewest steps among test-time scaling baselines; on RL-enhanced LLaDA-1.5, it also improves performance with minimal additional inference cost.On Dream, speedups are 1.0–2.7× over majority voting, 1.0–1.8× over RFG, and 5.0–13.4× over HEX; the LLaDA-1.5 evaluation covers four benchmarks.Source: Table 4 · p. 6, §5.2 · pp. 6–7, Table 5 · p. 7
  • Across all settings, DVOTING achieves a superior benefits-per-cost trade-off, with larger BPC improvements than other test-time scaling baselines as LLaDA generation length increases.BPC measures performance gain over the original model per additional denoising step.Source: Figure 4 · p. 6, §5.2 · p. 7
  • On LLaDA-8B-Instruct, DVOTING gains 6.22%–7.66% on GSM8K, 4.40%–7.20% on MATH500, 3.16%–14.84% on ARC-C, 3.57%–4.73% on GPQA, and 4.83%–5.74% on MMLU.It also uses the fewest steps among test-time scaling baselines, with 1.1–4.4× speedup over majority voting, 1.1–2.2× over RFG, and 5.5–22.1× over HEX.Source: §5.2 · p. 6

5.3 Visualization

  • In two GSM8K cases, DVOTING stops early when the base answer is correct and uses five sampling iterations and voting to correct an incorrect answer.The examples illustrate computation adapting to problem difficulty: simple cases receive less inference effort, while harder cases receive more.Source: Figure 5 · p. 7, §5.3 · pp. 7–8

5.4 Ablation Study

  • On GSM8K and MATH500 at generation lengths 128 and 256, performance improves with larger sampling upper bounds n until reaching saturation.The ablation evaluates n values of 1, 5, 9, 13, and 17 while keeping other hyperparameters fixed.Source: Table 6 · p. 8, §5.4 · p. 8
  • Across block sizes 4–64, DVOTING outperforms both the original results and majority voting, showing robustness to block size.The ablation varies block size while keeping other hyperparameters fixed.Source: Table 7 · p. 8, §5.4 · p. 8
  • Across entropy thresholds 0.1–0.7, DVOTING maintains strong performance and favorable efficiency; at α=0.9 performance drops, but remains above the original results.The ablation also reports a test-time scaling trend in which increased computation leads to improved performance.Source: Table 8 · p. 8, §5.4 · p. 8

6 Conclusion

dVoting addresses the high inference cost of test-time scaling in dLLMs by using repeated-token consistency and remasking for fast voting, improving reasoning with modest extra inference cost.

6 Conclusion

  • dVoting consistently enhances reasoning performance with modest extra inference cost, achieving a leading performance–efficiency trade-off.The conclusion presents the method as a foundation for future test-time scaling and a way to unlock dLLMs’ inference-time potential.Source: §6 · pp. 8–9
  • dVoting leverages dLLMs’ remasking mechanism to provide a fast voting strategy tailored to these models.The method is proposed to address the high inference cost incurred by test-time scaling in dLLMs.Source: §6 · p. 8
  • The approach builds on the empirical observation that repeated tokens frequently emerge across multiple samples for the same prompt.The conclusion presents this observation as the insight motivating dVoting.Source: §6 · p. 8

Impact Statement · Appendix

dVoting uses remasking to direct test-time computation toward uncertain tokens, improving reasoning while reducing inference cost and resource consumption. The appendix describes the method, additional case analyses, future work, and the LLM usage statement.

Impact Statement

  • dVoting uses a voting strategy that leverages remasking to focus test-time computation on uncertain tokens.The paper frames this as an approach to efficient test-time scaling for diffusion large language models.Source: Impact Statement · p. 9
  • The approach improves reasoning performance while reducing inference cost.The impact statement presents these gains as making test-time scaling more practical under limited computational budgets.Source: Impact Statement · p. 9
  • By improving inference efficiency, dVoting reduces computational overhead and resource consumption.The statement says this makes large-scale model deployment more feasible in practice.Source: Impact Statement · p. 9
  • The method introduces no additional ethical or societal concerns beyond those commonly associated with large language models.Source: Impact Statement · p. 9

Appendix

  • Appendix A.1 presents the detailed process of dVoting.Source: Appendix · p. 13
  • Appendix B provides additional case analysis to further demonstrate dVoting’s effectiveness.Source: Appendix · p. 13
  • Appendix C discusses future work, and Appendix D presents the LLM usage statement.Source: Appendix · p. 13

A The Details of DVOTING · A.1 The Detailed Algorithm of the Consistency-Guided Designs

dVoting iteratively samples generations, uses token-level consistency to decide what to fix or remask, and stops when no further remasking is needed. It then votes over the sampled answers to produce a final prediction.

A.1 The Detailed Algorithm of the Consistency-Guided Designs

  • Algorithm 1 specifies dVoting’s pipeline, implemented with PyTorch.The algorithm takes a prompt, dLLM, maximum sample count n, and generation length L as inputs.Source: §A.1 · p. 13
  • For each sample, dVoting initializes a sequence with prompt tokens and masked generation positions, alongside a Boolean array tracking which positions to remask.The array m has length L and records whether each generation position should be remasked.Source: §A.1 · p. 13
  • Using previous samples, dVoting fixes positions that satisfy its consistency criterion and masks positions that do not.It computes a token consistency score at each generation position to make this decision.Source: §A.1 · p. 13
  • If global token-level consistency indicates that no position needs remasking, dVoting fixes all tokens and stops further sampling.The algorithm breaks when all entries of m are False.Source: §A.1 · p. 13
  • When masked tokens remain, dVoting uses entropy-threshold-based parallel decoding with a semi-autoregressive strategy to complete the sequence.Each completed sequence is appended to the collection of sampled generations.Source: §A.1 · p. 13
  • After sampling, dVoting votes over candidate answers from the collected generations to obtain its final prediction.Sampling continues up to the maximum n unless the algorithm stops earlier when no positions require remasking.Source: §A.1 · p. 13

B Case Analysis

Additional ARC-C and MATH500 cases show dVoting stopping once answers are consistent: it quickly returns a correct answer on an easy problem and refines an initially incorrect answer on a harder one.

B Case Analysis

C Future Work · D Large Language Model Usage Statement

The work focuses on language and identifies parallel test-time scaling beyond language as future work; LLMs were used only for minor language editing.

C Future Work

D Large Language Model Usage Statement

Related papers

Loading 2602.12153v1…