Source-linked AI summary
Energy Consumption Analysis of Discrete Diffusion and Autoregressive Language Models
Boris Ruf, Marcin Detyniecki
TL;DR
The paper examines how inference energy per token for discrete diffusion language models compares with autoregressive models, and benchmarks them across output lengths and cache settings. Its results show a length-dependent crossover, with diffusion more efficient at long sequences despite substantial KV-cache benefits for AR models.
Problem
The energy cost of multiple diffusion denoising steps versus single-pass autoregressive generation remains under-explored, despite diffusion’s parallel token updates.
Source: §1 · p. 1Method
The study benchmarks diffusion and autoregressive models on instruction prompts, comparing energy and latency across sequence lengths and AR cache settings.
Source: §1 · p. 2, §3.1 · p. 2Results
Takeaways & Limitations
Deployment choices should consider expected output length, since the observed efficiency advantage shifts from AR at short sequences to diffusion at long sequences.
Source: §8 · p. 5The study uses one fixed prompt and three sequence lengths, leaving the precise efficiency crossover uncertain, and does not rigorously evaluate output quality.
Source: §5 · p. 4
Abstract
from arXiv · showhide
As Large Language Models (LLMs) continue to scale, their energy consumption during inference has become a critical concern. While Autoregressive (AR) models are the industry standard, Diffusion Language Models (DLLMs) have emerged as a promising alternative capable of parallel token generation. In this paper, we perform a comparative energy benchmark of diffusion models (LLaDA-8B, Dream-7B) against AR counterparts (Llama-3.1-8B, Qwen-2.5-7B). We analyze the trade-offs between model architecture, KV-cache optimization, and Joules-per-token efficiency. Our results show that at longer sequences (200--600 tokens), diffusion models achieve 2.5--2.7 J/token, approximately 2$\times$ more efficient than AR models with KV-cache (5.3--5.7 J/token). KV-cache provides substantial benefits for AR models at longer sequences (50--70\% energy reduction).
1 Introduction
The paper asks whether discrete diffusion models’ parallel token generation can offset the energy cost of multiple denoising steps compared with autoregressive generation. It frames the comparison around Joules-per-token and latency across model families, sequence lengths, and KV-cache settings.
1 Introduction
- Autoregressive models generate sequentially, while text diffusion can refine multiple tokens simultaneously [9].The energy cost of multiple denoising steps versus a single-pass autoregressive generation remains under-explored.Source: §1 · p. 1
- A theoretical utilization model suggests diffusion’s shorter, more intense parallel-processing block may offer efficiency advantages, which the paper tests empirically.The autoregressive pattern instead has distinct sequential peaks for each token.Source: §1 · p. 1
- The central question is how end-to-end inference energy per token compares for discrete diffusion and autoregressive models on instruction-style generation under matched conditions.The study also asks how latency and sequence length affect the comparison, and whether KV-cache optimization narrows the gap.Source: §1 · pp. 1–2
2 Related Work
Prior work covers text diffusion methods, inference-time system bottlenecks, and energy measurement, but hardware-level Joules-per-token evidence for text diffusion LLMs remains scarce. This paper reports end-to-end energy and latency under controlled cache and backend settings.
2 Related Work
- Text diffusion spans masked and discrete formulations [12], large-scale diffusion LLMs, and AR-to-diffusion conversion pipelines [7] [17] [19].AR-to-block-diffusion and lightweight adaptation approaches [5] [16] align with AR pretraining to reduce adaptation cost relative to training diffusion models from scratch.Source: §2 · p. 2
- At inference, parallel token updates coexist with KV-cache reuse challenges and the extra compute of multiple denoising steps; proposed approaches target decoding, caching, and sampling [4] [10] [16].The cited work includes parallel decoding, adaptive KV strategies, and block or anchor decoding designs.Source: §2 · p. 2
- Hardware-level Joules-per-token measurements for text diffusion LLMs remain scarce, motivating this study’s controlled energy and latency measurements.The related work also covers Green AI and runtime energy logging [14], alongside image-diffusion profiling [2].Source: §2 · p. 2
3 Experimental Setup
The study benchmarks selected diffusion and autoregressive models on an NVIDIA A100 using a fixed instruction prompt, three target lengths, and repeated measured runs. It estimates GPU energy from power draw and reports energy and latency per generated token.
3 Experimental Setup
- The experimental setup is designed to make hardware, software, and measurement procedures reproducible.The model set is selected to illustrate algorithmic and system trade-offs.Source: §3 · p. 2, §3.1 · p. 2
3.1 Model selection
- The benchmark includes diffusion models LLaDA-8B and Dream-7B and autoregressive baselines Llama-3.1-8B-Instruct and Qwen-2.5-7B-Instruct.LLaDA represents native diffusion training, Dream-7B evaluates denoising-schedule and parallel-sampling trade-offs, and the AR models serve as instruction-generation baselines [19] [15].Source: §3.1 · p. 2
3.2 Hardware and system configuration
- Runs use an Azure instance with an NVIDIA A100, mixed precision where supported, consistent CUDA/cuDNN and PyTorch versions, and per-model Docker images.GPU power draw is captured through NVML and integrated over the measured duration.Source: §3.2 · p. 2
3.3 Evaluation protocol
- Each model configuration uses one fixed instruction prompt at 10, 128, and 1024 target tokens, with five warmups and ten measured runs.Autoregressive models are tested with and without KV-cache.Source: §3.3 · p. 2
3.4 Metrics
- Energy per token divides integrated GPU energy by generated tokens, while latency per token divides wall-clock time by generated tokens.Reported means cover ten measured runs; total GPU power during generation is included without subtracting an idle baseline.Source: §3.4 · p. 3
4 Results
Across the tested lengths, the results show an efficiency crossover: autoregressive models lead at short outputs, while diffusion models lead at long outputs. KV-cache substantially improves long-sequence AR energy, but does not erase the long-sequence diffusion advantage.
4.1 Per-model observations
- Diffusion energy per token falls from 32–36 J/token at 10 tokens to 2.01–2.40 J/token at 1024 tokens, as fixed initialization costs are amortized.Qwen-2.5 may terminate early at 564–587 tokens by generating an end-of-sequence token, so max_tokens is an upper bound rather than a guarantee.Source: §4.1 · p. 3
4.2 Impact of KV-cache on AR models
- At 1024 tokens, KV-cache reduces Llama from 18.21 to 5.26 J/token (71%) and Qwen from 12.51 to 5.72 J/token (54%); at 10 tokens, Llama drops only 3%.The measured benefit of caching is therefore much larger at the long tested length than at the short one.Source: §4.2 · p. 4
- Diffusion models cannot use KV-cache because of bidirectional attention, yet their 2.01–2.40 J/token at long sequences remains below cached AR’s 5.26–5.72 J/token.The cited execution traces compare cached Llama-3.1 and Dream-7B utilization and power over ten measured 128-token runs after a warmup period.Source: §4.2 · p. 4, Figure 2 · p. 4
4 Results
- At 10 tokens, cached Llama uses 8.10 versus Dream-7B’s 32.21 J/token; at 128, Dream-7B uses 4.45 versus Llama’s 5.07; at 1024, diffusion is approximately 2.5× more efficient.The overall results identify short-sequence AR efficiency and long-sequence diffusion efficiency, with a crossover around the medium-length setting.Source: §4 · p. 3, §4.3 · p. 4
4.3 Diffusion vs. Autoregressive
- At 1024 tokens, diffusion reaches 0.006–0.007 s/token versus 0.027 s/token for cached AR, a 4× latency advantage attributed to parallel token generation.The figure description characterizes diffusion’s compute pattern as shorter and more intense than AR’s sequential token generation.Source: §4.3 · p. 4
5 Limitations
The study’s limitations include sparse prompt and sequence-length coverage, no rigorous output-quality evaluation, and evaluation on a single hardware and software setup.
5 Limitations
- A single fixed prompt and three output lengths identify a crossover around 128 tokens but leave its precise threshold uncertain.The authors call for finer-grained output-length sweeps to locate efficiency cliffs more precisely.Source: §5 · p. 4
- Output quality was not rigorously evaluated, so the study does not verify whether efficiency gains come at a cost to generation quality.The authors propose incorporating benchmarks such as MMLU and HellaSwag in future work.Source: §5 · p. 4
- Results may differ on other hardware or optimized backends because experiments used one A100 with vanilla HuggingFace Transformers.Source: §5 · p. 4
6 Future Work
Future work proposes refining crossover estimates, jointly evaluating quality and energy, and testing adaptive serving across deployment settings.
6 Future Work
- Finer output-length sweeps and joint quality–energy evaluation are proposed to map the crossover more precisely.Source: §6 · p. 5
- A hybrid serving policy could route requests between AR and diffusion backends using expected output length, latency constraints, and current load.Source: §6 · p. 5
- The proposed policy should be validated across hardware platforms, optimized runtimes, and quantization settings for production-like robustness.Source: §6 · p. 5
7 Discussion
The discussion attributes the sequence-length crossover to architectural computation patterns, notes a workload mismatch for Qwen-2.5, and reports that KV-cache does not erase diffusion’s long-sequence advantage.
7 Discussion
- Diffusion’s fixed initialization overhead is amortized over longer sequences, while AR per-token costs remain relatively constant and become less competitive at scale.Source: §7 · p. 5
- Qwen-2.5 generated 564–587 tokens at a 1024-token maximum after producing EOS, complicating direct energy comparisons because workloads differ.Source: §7 · p. 5
- KV-cache reduces AR energy by 50–70% at long sequences, but diffusion retains an efficiency advantage there.Source: §7 · p. 5
8 Conclusion
The conclusion reports an efficiency crossover across sequence lengths and recommends choosing architectures according to expected response length.
8 Conclusion
- At long sequences (≥1024 tokens), diffusion models are 2.5× more efficient (2 J/token vs. 5 J/token) and achieve 4× lower latency (0.006 s/token vs. 0.027 s/token).At short sequences (≤10 tokens), AR models are 4× more efficient (8 J/token vs. 32–36 J/token); at medium lengths (≈128 tokens), efficiency converges and diffusion begins to lead.Source: §8 · p. 5
- KV-cache benefits AR models but does not close the efficiency gap at longer sequences.Source: §8 · p. 5
- For short responses, AR models remain preferable, while diffusion models offer advantages for longer generation tasks.Source: §8 · p. 5