Source-linked AI summary
Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Models
Jinsong Li, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Jiaqi Wang, Dahua Lin
TL;DR
DLLMs require a generation length to be fixed before denoising, creating a trade-off between insufficient space and the costs of excessive length. DAEDAL uses internal confidence signals to adjust length before and during denoising; it performs comparably to, and sometimes better than, meticulously tuned fixed-length baselines while improving computational efficiency.
Problem
DLLM inference requires a statically predefined generation length, although task requirements vary and too-short or excessive lengths pose performance or computational costs.
Source: §1 · p. 1Method
DAEDAL is a training-free strategy that uses EOS confidence to set an initial task-appropriate length and inserts masks during denoising to expand insufficient regions.
Source: §1 · p. 2, §3.2.2 · p. 5Results
DAEDAL achieves performance comparable to, and sometimes better than, meticulously tuned fixed-length baselines while improving computational efficiency.
Source: §1 · p. 3, §4.3 · p. 8Takeaways & Limitations
DAEDAL adaptively allocates generation length per task rather than requiring manually tuned, task-specific lengths.
Source: §4.3 · p. 8
Abstract
from arXiv · showhide
Diffusion Large Language Models (DLLMs) are emerging as a powerful alternative to the dominant Autoregressive Large Language Models, offering efficient parallel generation and capable global context modeling. However, the practical application of DLLMs is hindered by a critical architectural constraint: the need for a statically predefined generation length. This static length allocation leads to a problematic trade-off: insufficient lengths cripple performance on complex tasks, while excessive lengths incur significant computational overhead and sometimes result in performance degradation. While the inference framework is rigid, we observe that the model itself possesses internal signals that correlate with the optimal response length for a given task. To bridge this gap, we leverage these latent signals and introduce DAEDAL, a novel training-free denoising strategy that enables Dynamic Adaptive Length Expansion for Diffusion Large Language Models. DAEDAL operates in two phases: 1) Before the denoising process, DAEDAL starts from a short initial length and iteratively expands it to a coarse task-appropriate length, guided by a sequence completion metric. 2) During the denoising process, DAEDAL dynamically intervenes by pinpointing and expanding insufficient generation regions through mask token insertion, ensuring the final output is fully developed. Extensive experiments on DLLMs demonstrate that DAEDAL achieves performance comparable, and in some cases superior, to meticulously tuned fixed-length baselines, while simultaneously enhancing computational efficiency by achieving a higher effective token ratio. By resolving the static length constraint, DAEDAL unlocks new potential for DLLMs, bridging a critical gap with their Autoregressive counterparts and paving the way for more efficient and capable generation.
1 INTRODUCTION
DAEDAL uses internal confidence signals to adapt generation length rather than requiring a fixed length chosen in advance. It achieves performance comparable to, and sometimes better than, meticulously tuned fixed-length baselines while improving computational efficiency.
1 INTRODUCTION
- A fixed generation length forces DLLMs to fit the task to a predetermined token budget, unlike autoregressive models that adjust output length to the task.Short budgets can hinder complex tasks, while long ones add quadratic bidirectional-attention cost and may degrade performance.Source: §1 · pp. 1–2
- EOS prediction confidence serves as an internal signal of length sufficiency: it is higher when the available length is sufficient and lower when it is not.The comparison is made at the first denoising step from a fully masked sequence.Source: §1 · p. 2, Figure 2 · p. 3
- DAEDAL starts from a short unified length, expands it before denoising using an EOS-based completion metric, then inserts mask tokens into insufficient regions during denoising.This training-free strategy replaces manually tuned, task-specific generation lengths with adaptive expansion.Source: §1 · p. 2
- DAEDAL matches or sometimes surpasses the peak performance of meticulously tuned fixed-length baselines while significantly improving computational efficiency.Its per-problem length allocation is illustrated in Figure 1(b).Source: Figure 1 · p. 2, §1 · p. 3
2 RELATED WORKS
Prior DLLM research includes both models trained from scratch and models adapted from autoregressive LLMs. Existing inference strategies focus on speed, while dynamic adjustment of total generation length remains an unaddressed problem in the cited work.
2 RELATED WORKS
- DLLMs include native models such as LLaDA and models adapted from pretrained autoregressive LLMs, while this work focuses on native, from-scratch models.The passage identifies LLaDA as a large-scale diffusion model trained from scratch (Nie et al., 2025).Source: §2 · p. 3
- Prior DLLM inference methods target speed through caching and parallel decoding, but the cited work does not address adapting total generation length to task requirements.The paper presents dynamic inference-time length adjustment as an unexplored problem.Source: §2 · p. 3
- Figure 3 contrasts fixed-length iterative denoising with DAEDAL’s sequence of initial length adjustment followed by on-demand mask insertion.The second process adjusts length before denoising and can expand it during denoising.Source: Figure 3 · p. 4
3 METHODS
The methods section first outlines DLLM denoising, then introduces DAEDAL’s two-stage approach: estimate an initial task-appropriate length and expand locally during denoising when additional space is needed.
3.1 OVERVIEW OF DIFFUSION LARGE LANGUAGE MODELS
- LLaDA begins inference with a predefined-length sequence of masks, predicts tokens at masked positions, and iteratively remasks lower-confidence predictions until a fixed number of steps is complete.The required length is specified before inference and cannot adapt to task requirements.Source: §3.1 · p. 5
3.2 DAEDAL
- DAEDAL is a training-free, two-stage strategy that allocates sequence length per task and inserts additional space where needed.Its stages are Initial Length Adjustment and Iterative Mask Insertion.Source: §3.2 · p. 5, §3.2.2 · p. 6
3.2.1 INITIAL LENGTH ADJUSTMENT
- Initial Length Adjustment repeatedly evaluates average EOS confidence in a terminal window and appends masks while confidence remains below a threshold, stopping at the threshold or maximum length.The loop begins from a short initial length and runs before the main denoising process.Source: §3.2.1 · p. 5, §3.2.2 · p. 6
3.2.2 ITERATIVE MASK INSERTION
- During denoising, DAEDAL can replace a very low-confidence masked position with multiple masks, adding localized space for complex reasoning or detailed description.The method flags the lowest-confidence position below a threshold as an expansion point; expansion is subject to EOS-confidence and maximum-length conditions.Source: §3.2.2 · pp. 5–6
4 EXPERIMENTS
Across four benchmarks, DAEDAL approaches or exceeds the best tuned fixed-length baselines while using length adaptively and improving computational efficiency. Ablations examine its stages and the robustness of its length and threshold settings.
4.1 IMPLEMENTATION DETAILS
- Experiments use LLaDA-Instruct-8B and LLaDA-1.5-8B with official generation code and evaluate math reasoning and code generation.GSM8K and MATH500 use accuracy; MBPP and HumanEval use pass@1.Source: §4.1 · p. 6, §4.2 · p. 7
4.3 MAIN RESULTS
- DAEDAL matches or sometimes surpasses peak tuned fixed-length performance across benchmarks while using fewer total tokens and achieving a higher effective token ratio.It adapts response length per problem, while each baseline uses one fixed length per benchmark.Source: Table 1 · p. 7, §4.3 · p. 8, Table 2 · p. 9
4.2 BENCHMARKS AND METRICS
- The evaluation compares DAEDAL with fixed-length baselines from 64 to 2048 tokens using accuracy, effective tokens, total tokens, and effective token ratio.Effective tokens exclude trailing padding; the baseline’s total tokens equal its preset length.Source: §4.3 · p. 7, Table 1 · p. 7, Table 2 · p. 9
4.4 ANALYSIS ON DAEDAL
- Both DAEDAL stages improve over the baseline individually, but their combination performs best; iterative insertion alone remains sensitive to initial length.Starting Stage 2 at 256 tokens can surpass the baseline’s best result, whereas starting at 64 falls short; the authors link this to the need for an initial global plan.Source: §4.4 · p. 9, Table 3 · p. 10
- Performance remains stable across initial lengths of 32–512 tokens and expansion factors of 8–32, supporting a short unified starting length without meticulous tuning.HumanEval accuracy is identical across tested initial lengths, and GSM8K variation is negligible.Source: §4.4 · pp. 9–10, Table 4 · p. 10, Table 5 · p. 10
- A large EOS confidence window and varied threshold settings yield robust performance, while very small windows can misjudge intent and under-allocate length.Across 32 threshold configurations on GSM8K, results are comparable to the best baseline, with some outperforming it; a window of 8 degrades performance relative to larger windows.Source: §4.4 · pp. 10–11, Figure 5 · p. 11
5 CONCLUSION
DAEDAL addresses fixed generation lengths in diffusion LLMs with two-stage, training-free adaptive expansion. Experiments report performance comparable to or sometimes better than tuned fixed-length baselines, alongside improved computational efficiency.
5 CONCLUSION
- DAEDAL uses initial length adjustment followed by iterative mask insertion to adapt sequence length during denoising.The first stage sets a coarse task-appropriate budget; the second expands regions needing more detailed reasoning.Source: §5 · p. 11
- Starting from a short, unified initial length, DAEDAL achieves performance comparable to or sometimes better than meticulously tuned fixed-length baselines.The approach adapts length per problem and removes the need for manual length tuning.Source: §5 · p. 11
- The authors conclude that DAEDAL improves computational efficiency and enables more flexible non-autoregressive generation within the scope of their experiments.They frame the approach as bridging a capability gap between diffusion and autoregressive models.Source: §5 · p. 11