PaperlayerView exact revision on arXiv

Source-linked AI summary

dVoting: Fast Voting for dLLMs

Sicheng Feng, Zigeng Chen, Xinyin Ma, Gongfan Fang, Xinchao Wang

arXiv:2602.12153v1cs.CLcs.AI

TL;DR

dLLM test-time scaling seeks stronger reasoning without the inefficiency of autoregressive parallelization. DVOTING uses token-consistency analysis to remask and regenerate uncertain positions, then aggregates candidates by voting. It consistently improves reasoning benchmarks while maintaining a favorable performance–efficiency trade-off.

  • Problem

    Test-time scaling in dLLMs incurs high inference cost, while the extent of repeated token predictions across samples creates an opportunity for more efficient scaling.

  • Method

    DVOTING is a training-free remask sampling strategy that preserves consistent tokens, iteratively regenerates uncertain positions, and aggregates candidate answers through voting.

  • Results

    DVOTING consistently improves reasoning performance across benchmarks and dLLM families while using fewer steps and achieving strong speedups over test-time scaling baselines.

  • Takeaways & Limitations

    DVOTING provides a foundation for efficient test-time scaling and advances the performance–efficiency trade-off in dLLMs.

Abstract

from arXiv · show

Diffusion Large Language Models (dLLMs) represent a new paradigm beyond autoregressive modeling, offering competitive performance while naturally enabling a flexible decoding process. Specifically, dLLMs can generate tokens at arbitrary positions in parallel, endowing them with significant potential for parallel test-time scaling, which was previously constrained by severe inefficiency in autoregressive modeling. In this work, we introduce dVoting, a fast voting technique that boosts reasoning capability without training, with only an acceptable extra computational overhead. dVoting is motivated by the observation that, across multiple samples for the same prompt, token predictions remain largely consistent, whereas performance is determined by a small subset of tokens exhibiting cross-sample variability. Leveraging the arbitrary-position generation capability of dLLMs, dVoting performs iterative refinement by sampling, identifying uncertain tokens via consistency analysis, regenerating them through voting, and repeating this process until convergence. Extensive evaluations demonstrate that dVoting consistently improves performance across various benchmarks. It achieves gains of 6.22%-7.66% on GSM8K, 4.40%-7.20% on MATH500, 3.16%-14.84% on ARC-C, and 4.83%-5.74% on MMLU. Our code is available at https://github.com/fscdc/dVoting

1 Introduction

dLLMs enable flexible parallel decoding, motivating test-time reasoning methods that operate during inference rather than training. DVOTING exploits repeated token predictions by selectively remasking uncertain tokens, improving reasoning performance across benchmarks with modest overhead.

  • 1 Introduction: dLLMs support parallel decoding in flexible orders beyond left-to-right generation, creating potential for test-time scaling.Their iterative unmask-and-remask process can generate tokens at arbitrary positions.
  • 1 Introduction: Existing dLLM reasoning enhancements mainly emphasize reinforcement learning during training, while test-time decoding methods remain comparatively limited.Prior test-time methods include varying masking schedules and logit-level guidance.
  • 1 Introduction: Repeated tokens frequently occur across samples for the same prompt, while arbitrary-position remasking enables reuse of consistent tokens and refinement of uncertain ones.The method connects token-level consistency with dLLM remasking capabilities.
  • 1 Introduction: DVOTING preserves consistent tokens, remasks the remainder, iteratively regenerates candidates, and stops early when candidate answers satisfy consistency criteria.The process is summarized as a remask sampling strategy followed by voting over candidate answers.
  • 1 Introduction: 6.22%–7.66% gains on GSM8K and 4.40%–7.20% gains on MATH500 were reported on LLaDA, alongside improvements across mathematical, scientific, and general reasoning benchmarks.The method also reports a leading performance–efficiency trade-off and robustness under varied configurations.
  • 1 Introduction: DVOTING is presented as a training-free baseline and foundation for efficient test-time scaling in dLLMs.The authors frame it as helping unlock inference-time potential.

2 Related Work

The related work situates dLLMs within expanding diffusion-language research and contrasts their underexplored test-time scaling with established autoregressive approaches. Existing dLLM methods include masking-schedule aggregation and auxiliary-model logit guidance.

  • 2 Related Work: Diffusion language modeling extends diffusion generation from continuous domains to discrete text, with growing applications in multimodal generation and code synthesis.The passage characterizes the research direction as increasingly broad and mature.
  • 2 Related Work: Test-time scaling allocates additional inference computation to elicit stronger reasoning, using methods such as chain-of-thought, best-of-N sampling, and self-consistency in autoregressive models.The passage contrasts this extensive autoregressive literature with more limited work for dLLMs.
  • 2 Related Work: HEX aggregates diverse masking schedules, whereas RFG guides generation at the logit level with an additional fine-tuned model.These methods provide early references for dLLM test-time scaling.

3 Preliminaries

Masked diffusion language models reconstruct partially corrupted sequences rather than generating strictly left to right. During inference, dLLMs update masked positions in parallel and selectively remask uncertain tokens, supporting parallel test-time scaling and voting-based aggregation.

  • 3 Preliminaries: MDLMs generate text by iteratively reconstructing a partially corrupted sequence rather than following an autoregressive token order.The forward process independently masks sequence positions with intensity t, and a denoising network reconstructs masked tokens.
  • 3 Preliminaries: Training focuses on reconstructing original content from corrupted sequences, with the reconstruction loss serving as a surrogate objective that upper-bounds negative log-likelihood.This describes the training objective for the masked diffusion formulation.
  • 3 Preliminaries: During inference, dLLMs update all masked positions in parallel and selectively remask uncertain positions across iterations.This mechanism permits multiple outputs to be generated in parallel and later aggregated.
  • 3 Preliminaries: Voting performs multiple sampling runs and selects the most consistent output, providing a training-free alternative that approximates reinforcement-learning effects on sampling efficiency.The paper presents test-time computation as achieving performance comparable to RL-enhanced approaches.

4 Methods

DVOTING is motivated by answer- and token-level redundancy across samples, then uses dLLM remasking to selectively refine uncertain positions and stop when voting converges.

  • Key observations: DVOTING begins with an empirical study of majority voting using entropy-threshold parallel decoding on LLaDA with GSM8K.
  • Key observations: Voting consistency level measures the fraction of votes received by the most frequent answer.
  • Key observations: 84.58% of samples solved correctly by both baseline and voting methods fall into the 4/5 or 5/5 consistency bins.
  • Key observations: NUPR@k measures the fraction of token positions where at least k of K samples share an identical token.Across GSM8K, MATH500, and ARC-C, NUPR@2 is around 50% and NUPR@3 around 20%.
  • Remask sampling: The remask sampling strategy retains agreed tokens, selectively remasks uncertain positions, and regenerates responses conditioned on preserved tokens.It connects token-level redundancy with dLLM remasking and avoids generating a full response at every sampling action.
  • Remask sampling: Entropy-threshold parallel decoding commits token positions whose entropy is below α = 0.3 at each denoising step, and sampling stops when the voting answer converges.

5 Experiments

DVOTING improves reasoning performance across dLLM benchmarks while using fewer inference steps and adapting computation to problem difficulty. Its performance–efficiency advantage remains robust across models, settings, and ablations.

  • Main Results: DVOTING improves LLaDA performance by 6.22%–7.66% on GSM8K, 4.40%–7.20% on MATH500, 3.16%–14.84% on ARC-C, and 4.83%–5.74% on MMLU.
  • Main Results: DVOTING delivers 1.1–4.4× speedup over majority voting, 1.1–2.2× over RFG, and 5.5–22.1× over HEX on LLaDA.
  • Main Results: DVOTING also improves Dream performance across all benchmarks while delivering 1.0–2.7×, 1.0–1.8×, and 5.0–13.4× speedups over majority voting, RFG, and HEX, respectively.
  • Performance–Efficiency Trade-off: DVOTING achieves a superior performance–efficiency trade-off across settings, with larger benefits-per-cost improvements than other baselines as generation length increases on LLaDA.Benefits per cost measures performance gain normalized by the increase in denoising steps.
  • Visualization: On easy cases, DVOTING terminates early, while on harder cases it performs five sampling iterations and votes to recover the correct answer.
  • Ablation Study: Performance improves with larger sampling bounds until saturation, remains robust across block sizes 4–64, and stays strong across entropy thresholds 0.1–0.7.A noticeable drop occurs only at the excessively high threshold α=0.9, where DVOTING still outperforms the original results.

6 Conclusion

The conclusion presents DVOTING as a training-free voting strategy that uses dLLM remasking to refine uncertain tokens. Extensive evaluations support its effectiveness and its role as a foundation for efficient test-time scaling in dLLMs.

  • DVOTING identifies repeated token generation across samples and uses remask sampling to regenerate selected tokens and aggregate candidate generations by voting.
  • DVOTING provides a foundation for efficient test-time scaling in dLLMs and further unlocks their potential at test time.

Impact Statement

The impact statement frames DVOTING as an efficiency-oriented approach to test-time scaling in dLLMs. It focuses computation on uncertain tokens to improve reasoning under limited computational budgets.

  • DVOTING leverages remasking to focus test-time computation on uncertain tokens, improving reasoning performance while reducing inference cost.
  • The approach is intended to make test-time scaling more practical under limited computational budgets by reducing computational overhead and resource consumption.

A.1 The Detailed Algorithm of the Consistency-Guided Designs

Algorithm 1 iteratively samples generations, fixes consistent tokens, remasks uncertain tokens, and stops when no further remasking is needed before voting on candidate answers.

  • The algorithm takes a prompt, dLLM, maximum sample count n, and generation length L as inputs.
  • For each sample, it computes token consistency from previous generations and fixes positions that satisfy the remasking criterion.
  • Tokens failing the criterion are remasked and regenerated using entropy-threshold-based parallel decoding with a semi-autoregressive strategy.
  • Sampling stops when all tokens are fixed, after which the collected candidate generations are used for voting.

B Case Analysis

The case analyses show DVOTING adapting its sampling effort to problem difficulty: it stops quickly on easy cases and refines initially incorrect answers on harder ones.

  • Adaptive refinement: DVOTING quickly produces correct answers on simple ARC-C and MATH500 problems, stopping after consistent answers are obtained.The ARC-C example early-stops after two samplings.
  • Adaptive refinement: DVOTING refines initially incorrect answers on challenging problems through additional sampling and voting.In the MATH500 example, the base model is initially incorrect, while DVOTING stops after four sampling runs with the correct result.
  • ARC-Challenge case: The ARC-Challenge example asks whether snow, rain, hail, and fog are forms of gas, water, wind, or clouds.The accompanying reasoning identifies them as forms of water and gives B as the correct answer.
  • MATH500 case: The MATH500 example simplifies √(2.5^2−0.7^2)/(2.7−2.5) by substituting the squared values and denominator.The displayed work reaches √5.76/0.2, though the supplied calculation does not finish simplification.

C Future Work

The paper primarily studies language-modality applications, while broader modalities and tasks remain future directions. It also identifies resource-constrained and cost-sensitive deployment as practical settings for DVOTING.

  • Scope: The method and experiments primarily focus on the language modality.The paper identifies this focus as a scope boundary.
  • Future directions: Exploring parallel test-time scaling for broader modalities and tasks, including multimodal question answering, is proposed as future work.The passage notes that diffusion language models show promise for extension beyond language tasks.
  • Deployment: DVOTING is described as computationally lightweight and suitable for resource-constrained and cost-sensitive settings such as on-device reasoning and large-scale inference services.The passage also characterizes parallel test-time scaling as simple to implement and easy to run.
Loading 2602.12153v1…