PaperlayerView exact revision on arXiv

Source-linked AI summary

Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policies

Zhixuan Liang, Yizhuo Li, Tianshuo Yang, Chengyue Wu, Sitong Mao, Liuao Pei, Tian Nian, Shunbo Zhou, Xiaokang Yang, Jiangmiao Pang, Yao Mu, Ping Luo

arXiv:2508.20072v4cs.CVcs.LGcs.RO

TL;DR

Existing VLAs either decode actions autoregressively or use separate diffusion heads, limiting unified action modeling and multimodal capability retention. Discrete Diffusion VLA instead applies discrete diffusion to action chunks inside a unified transformer, achieving strong benchmark performance and improved retention under distribution shift. Its main scope boundary is that multi-step decoding is slower than single-pass decoding and requires fixed-length tokenization.

  • Problem

    Prevailing VLAs use fixed-order autoregressive decoding or separate diffusion heads, creating limitations for unified, scalable action modeling and pretrained vision-language capability retention.

  • Method

    The method discretizes fixed-length action chunks and performs iterative masked-token refinement with adaptive decoding and secondary re-masking inside the unified VLM transformer.

  • Results

    The method achieves state-of-the-art performance among discrete methods across benchmarks, including 96.4% average success on LIBERO and superior out-of-distribution vision-language retention.

  • Takeaways & Limitations

    Unified discrete diffusion preserves pretrained vision-language priors while enabling parallel, confidence-guided, revisable action decoding.

  • Takeaways & Limitations

    Multi-step iterative decoding is slower than single-pass decoding, and variable-length action tokenization schemes are incompatible with discrete diffusion.

Abstract

from arXiv · show

Vision-Language-Action (VLA) models adapt large vision-language backbones to map images and instructions into robot actions. However, prevailing VLAs either generate actions autoregressively in a fixed left-to-right order with poor performance or attach separate diffusion heads outside the backbone that fragments information pathways and hinders unified, scalable architectures. Instead, we present Discrete Diffusion VLA that discretizes action chunks and models them with discrete diffusion pattern retaining progressive refinement inside the unified transformer backbone. Our method achieves an adaptive decoding order that resolves high-confidence action elements before harder ones and employs secondary re-masking to revisit uncertain predictions, enabling robust error correction. This design preserves pretrained vision-language priors, supports parallel decoding, and improves the efficiency. Discrete Diffusion VLA achieves 96.4% avg. success on LIBERO, 71.2% visual matching on SimplerEnv-Fractal, and 54.2% overall on SimplerEnv-Bridge. On out-of-distribution tests of LIBERO-Goal, our method exhibits only 0.8% language degradation versus 8.0% of parallel decoding, and 20.4% vision degradation versus 29.0% for continuous diffusion, demonstrating well retention of pretrained vision-language capabilities. We also conduct two real-robot evaluations on AgileX Cobot Magic platform to show the method's effectiveness.

1. Introduction

Discrete Diffusion VLA unifies discrete action generation with vision-language modeling inside one transformer, using iterative refinement to improve decoding and retain pretrained multimodal capabilities. It achieves strong results across simulation benchmarks and real-robot evaluations.

  • Discrete Diffusion VLA introduces discrete diffusion for action generation within a single transformer, unifying vision, language, and action modeling.
  • Action dimensions are discretized into fixed-length chunks, masked during training, and iteratively predicted and re-masked during inference.
  • On LIBERO-Goal out-of-distribution tests, language degradation is 0.8% versus diffusion’s 2.4%, while vision degradation is 20.4% versus diffusion’s 29.0%.
  • 96.4% average success is achieved on LIBERO, best among discretized action methods and 0.7% behind the overall continuous state of the art.
  • Real-robot evaluations on AgileX Cobot Magic show the method outperforming other baselines on two tabletop manipulation tasks at 9.69 Hz.
  • 64.1% visual matching on SimplerEnv-Fractal and 54.2% overall on SimplerEnv-Bridge establish state-of-the-art performance across the reported SimplerEnv methods.

2. Related Works

Related work spans separate decoders, autoregressive action generation, and discrete diffusion for other modalities. Discrete Diffusion VLA extends token-based discrete diffusion to chunked robot actions inside the VLM backbone.

  • Earlier VLA systems used separate MLP decoders, while later systems adopted scaled autoregressive architectures for general manipulation.
  • The method operates inside the VLM backbone with consistent optimization gradients, enabling parallel, revisable decoding through iterative re-masking.
  • Discrete diffusion models have been applied to tokenized images, natural language, and unified multimodal generation with transformer-based iterative refinement.
  • Its architecture encodes multi-view RGB images and language, then progressively refines discrete action chunks using adaptive decoding and secondary re-masking.
  • Discrete Diffusion VLA performs discrete diffusion over tokenized action chunks, extending this modeling approach to robot action generation.

3. Discrete Diffusion VLA Model

Discrete Diffusion VLA extends a unified vision-language transformer to generate discretized action chunks through masked-token diffusion, adaptive refinement, and secondary re-masking.

  • Action representation: The model represents each action chunk as a length-L sequence of discretized tokens, augmented with a special [MASK] symbol.Each token corresponds to one of K action bins, and the vocabulary has V=K+1 symbols.
  • Discrete diffusion formulation: Discrete diffusion corrupts action tokens by independently masking them, then defines a multimodal reverse process that progressively recovers subsets of masked positions.The reverse process conditions on vision, language, and partially denoised action tokens while reducing the mask ratio toward the original action chunk.
  • Training: Training samples a mask ratio, replaces selected action positions with [MASK], and minimizes cross-entropy for the original tokens at masked indices.The transformer predicts masked action tokens from multimodal context; vision and language tokens are attended but excluded from the loss.
  • Unified architecture: The unified backbone uses bidirectional action-token attention so every action position can attend to vision, language, and other action tokens.A shared classification head projects action-position hidden states to the 256-way action vocabulary.
  • Adaptive decoding: Inference initializes all action positions as [MASK] and performs parallel refinement rounds, committing only the highest-confidence positions while leaving others masked.This produces an instance-adaptive decoding order rather than the fixed order used by autoregressive decoding.
  • Secondary re-masking: Secondary re-masking checks previously committed tokens and masks those whose confidence falls below a threshold, preventing early errors from persisting.The mechanism is intended to improve cross-iteration consistency while maintaining alignment with the reverse diffusion process.

4. Experiments

Experiments evaluate Discrete Diffusion VLA across multiple robot platforms, benchmarks, action-decoding strategies, and efficiency settings. The method achieves strong task performance, preserves vision-language capabilities under distribution shift, and improves refinement efficiency over autoregressive decoding.

  • Overall Performance: 96.4% average success on LIBERO establishes state-of-the-art among discrete tokenized methods, while remaining 0.7% below the overall continuous SOTA.The per-suite scores are 97.2% Spatial, 99.4% Object, 96.8% Goal, and 92.2% Long.
  • Vision-Language Retention: 0.8% language degradation and 20.4% vision degradation on LIBERO-Goal-OOD are lower than competing decoding and action-generation methods.The comparisons include 8.0% language degradation for parallel decoding and 29.0% vision degradation for continuous diffusion.
  • Extended Evaluation Across Robot Platforms: 71.2% Visual Matching and 64.1% overall average on SimplerEnv-Fractal surpass competing policies, while Variant Aggregation reaches 56.9%.Visual Matching exceeds π0, π0-FAST, and OpenVLA-OFT; Variant Aggregation remains competitive with RT-2-X and π0-FAST.
  • Extended Evaluation Across Robot Platforms: 54.2% overall on SimplerEnv-Bridge outperforms continuous diffusion, flow-matching, and discrete baselines.The reported margins are +14.1% over π0, +4.7% over GR00T-N1, and +5.9% over π0-FAST.
  • Ablation Study: 96.8% on LIBERO-Goal is achieved by max-confidence decoding, improving 1.2% over one-shot parallel decoding.The compared success rates are 95.6% for one-shot parallel, 95.8% for random order, 96.6% for confidence-gap, and 96.8% for max-confidence selection.
  • Inference Efficiency: 12 denoising steps reduce NFEs from 56 to 12, while end-to-end latency reaches 68.8 ms per chunk, twice as fast as autoregressive decoding.Secondary remasking adds less than 1 ms because it operates only on logits; one-shot parallel decoding is faster but sacrifices accuracy.
  • Visualization of Adaptive Decoding Order: High-frequency tokens predominantly resolve during early steps, and 6 of 8 tokens at t=4 belong to the top-three most frequent tokens per dimension.This visualization provides qualitative evidence for confidence-guided, instance-wise decoding order.

5. Conclusion

Discrete Diffusion VLA unifies perception, language grounding, and action generation in one transformer through discrete diffusion over action tokens. It achieves strong discrete-method performance, distribution-shift generalization, and inference efficiency, while iterative decoding remains slower than single-pass decoding and fixed-length tokenization limits compatibility.

  • Conclusion: Discrete Diffusion VLA uses one transformer and a shared token vocabulary and cross-entropy objective to integrate vision, language, and action generation.Bidirectional action-token attention supports full-chunk interaction and confidence-guided revisable decoding.
  • Conclusion: Experiments across simulation benchmarks and a real-robot platform report state-of-the-art performance among discrete methods, stronger distribution-shift generalization, and favorable efficiency over autoregressive decoding.These conclusions are stated across the paper’s benchmark and hardware evaluations.
  • Limitations and Future Work: Iterative decoding is slower than single-pass decoding, and variable-length action tokenization such as π0-FAST is incompatible with discrete diffusion.The authors identify more expressive fixed-length tokenization as a direction for narrowing the gap with continuous methods.

Impact Statement

The paper envisions assistive and industrial robotics applications but notes that safety-critical deployment requires additional formal safety evaluation. Reliance on large pretrained vision-language models may also transfer pretraining-data biases to deployment.

  • Impact Statement: Safety-critical deployment requires formal safety evaluation beyond the scope of this work.The impact statement places this requirement before deployment in safety-critical environments.
  • Impact Statement: Large pretrained vision-language models may inherit biases from their pretraining data, which should be considered before real-world deployment.This is presented as a deployment consideration rather than an evaluated failure mode.

A. Visualizations of Robot Task Executions

The visualization set identifies robot platforms paired with representative manipulation tasks across LIBERO, SimplerEnv, and AgileX Cobot Magic evaluations.

  • Robot Task Executions: The visualizations pair each listed robot platform with a representative task, including LIBERO arms, Google Robot, WidowX, and AgileX Cobot Magic.The supplied labels include tasks such as Click the Bell and Place Cup on Coaster for AgileX Cobot Magic.

B. Implementation Details

Experiments use environment-specific action chunks, consistent batch and compute settings, and selected training checkpoints. Secondary re-masking uses a step-dependent threshold.

  • Action chunk size H is 8 for LIBERO and SimplerEnv-Fractal, and 3 for SimplerEnv-Bridge.
  • All experiments use batch size 32 and typically run on 4 NVIDIA A800 Tensor Core GPUs.
  • Feature-wise Linear Modulation is applied in SimplerEnv-Bridge experiments to enhance language grounding on WidowX Arm tasks.
  • Training lasts 150k–300k steps on LIBERO and 100k steps on both SimplerEnv benchmarks, with highest-performing checkpoints reported.
  • Secondary re-masking uses η_abs^t = 0.5 × (1 − t/T) as the step-dependent threshold.

C. Reproducibility and Baseline Details

The evaluation controls baseline provenance, training conditions, and OOD perturbations while contrasting unified discrete diffusion with autoregressive and parallel decoding. The comparison emphasizes bidirectional action-chunk modeling, adaptive refinement, and inference efficiency.

  • Reproducibility: Reproduced baselines use identical input modalities, data splits, evaluation protocols, official checkpoints, fine-tuning conditions, and three random seeds.Published and reproduced baseline sources are documented in Table 10.
  • OOD evaluation: LIBERO-PRO settings are followed with corrections to published implementation inconsistencies, including object and language perturbations for OOD evaluation.Object variants differ in scale, material, and appearance, while instructions are semantically equivalent but lexically varied.
  • Decoding design: Discrete diffusion solves masked infilling tasks simultaneously, enabling bidirectional attention over full action chunks and arbitrary-order inference instead of causal left-to-right commitment.This structure supports confidence-guided refinement and secondary remasking for correcting inconsistent predictions.
  • Inference efficiency: Discrete Diffusion VLA decodes in T=12 parallel steps at 14.53 Hz, reducing NFEs 4.7× and doubling wall-clock speed relative to AR.AR requires L sequential forward passes, with NFE=56 for LIBERO.

E.2. Resolving the Limitations of Continuous Diffusion Heads

Continuous diffusion heads introduce a loss mismatch that can erode pretrained vision-language priors under distribution shift. Discrete Diffusion VLA avoids this competing-loss pathway while showing the smallest reported degradation across the two LIBERO OOD benchmarks and perturbation types.

  • Limitation of continuous diffusion: A diffusion loss over action outputs differs from the pretrained language cross-entropy objective, creating competing gradients that erode VLM priors under distribution shift.
  • OOD degradation: Continuous diffusion has the most severe vision degradation: 29.0% on LIBERO-Goal and 5.8% on LIBERO-Spatial.The passage links this pattern to diffusion-based action heads becoming overly vision-dependent.
  • Discrete diffusion design: Discrete action tokens share the language vocabulary and training objective, so the method introduces no competing diffusion loss alongside pretrained cross-entropy.This construction is presented as the reason for avoiding the continuous-head failure mode.
  • OOD robustness: Discrete Diffusion VLA exhibits the smallest degradation across both benchmarks and both perturbation types.Table 12 reports absolute performance drops from in-distribution results.
  • Overall comparison: The unified backbone combines prior preservation from architectural unification with structured parallel generation while avoiding AR compounding errors and diffusion gradient competition.

F. Action Head Ablation without Robot Pretraining

An action-head ablation with a pure vision-language backbone tests whether the observed gains arise from discrete diffusion rather than robot-specific pretraining. Discrete diffusion achieves the highest average LIBERO success under identical training conditions.

  • Experimental design: The ablation replaces OpenVLA with Qwen2.5-VL, a pure VLM without robot-specific pretraining, and compares four action-head variants under matched conditions.All four LIBERO suites are trained jointly for 30k steps with 256-bin discretization and learning rate 1e-4.
  • Results: 95.1% average success is achieved by discrete diffusion, exceeding AR by 13.9%, FAST by 4.0%, parallel decoding by 0.9%, and continuous diffusion by 0.2%.
  • Interpretation: The results support an action-head contribution intrinsic to the discrete diffusion paradigm rather than inherited from OpenVLA initialization.The comparison uses a backbone with no robot pretraining.
Loading 2508.20072v4…