PaperlayerView exact revision on arXiv

Source-linked AI summary

PD-VLA: Accelerating Vision-Language-Action Model Integrated with Action Chunking via Parallel Decoding

Wenxuan Song, Jiayi Chen, Pengxiang Ding, Han Zhao, Wei Zhao, Zhide Zhong, Zongyuan Ge, Zhijun Li, Donglin Wang, Jun Ma, Lujia Wang, Haoang Li

arXiv:2503.02310v2cs.ROcs.CV

TL;DR

Action chunking improves VLA control but makes autoregressive decoding inefficient as action sequences grow. PD-VLA parallelizes action-token decoding through fixed-point iterations, achieving 2.52× execution frequency versus the fundamental VLA model while maintaining competitive success rates.

  • Problem

    Action chunking increases action dimensions and lengthens single-inference time, creating an efficiency bottleneck for autoregressive VLA decoding.

  • Method

    PD-VLA reformulates autoregressive action decoding as a nonlinear system solved through parallel fixed-point iterations, accelerating inference without model redesign or training.

  • Results

    2.52× execution frequency versus the fundamental VLA model was achieved in simulation while maintaining competitive success rates, with 91.7% success on LIBERO-Long.

  • Takeaways & Limitations

    PD-VLA supports training-free deployment and balances performance with higher-frequency inference across simulation and real-world robotic manipulation tasks.

Abstract

from arXiv · show

Vision-Language-Action (VLA) models demonstrate remarkable potential for generalizable robotic manipulation. The performance of VLA models can be improved by integrating with action chunking, a critical technique for effective control. However, action chunking linearly scales up action dimensions in VLA models with increased chunking sizes. This reduces the inference efficiency. To tackle this problem, we propose PD-VLA, the first parallel decoding framework for VLA models integrated with action chunking. Our framework reformulates autoregressive decoding as a nonlinear system solved by parallel fixed-point iterations. This approach preserves model performance with mathematical guarantees while significantly improving decoding speed. In addition, it enables training-free acceleration without architectural changes, as well as seamless synergy with existing acceleration techniques. Extensive simulations validate that our PD-VLA maintains competitive success rates while achieving 2.52 times execution frequency on manipulators (with 7 degrees of freedom) compared with the fundamental VLA model. Furthermore, we experimentally identify the most effective settings for acceleration. Finally, real-world experiments validate its high applicability across different tasks.

I. INTRODUCTION

PD-VLA addresses the inference inefficiency created when action chunking lengthens autoregressive VLA decoding. It reformulates decoding for parallel fixed-point iteration, enabling deployment-friendly acceleration while preserving performance and achieving higher execution frequency.

  • Motivation: 7m-dimensional action sequences arise for 7-DoF manipulators with chunk size m, making autoregressive single-inference time grow linearly with chunking.Autoregressive decoding predicts tokens sequentially, so generation time is proportional to predicted token length.
  • Approach: PD-VLA reframes autoregressive action decoding as nonlinear equations solved through parallel fixed-point iterations.The framework accelerates only inference decoding and does not redesign or modify the model.
  • Deployment: Training-free, model-redesign-free acceleration enables deployment on existing VLA models and can synergize with other acceleration techniques.The paper characterizes these deployment properties through its acceleration-method comparison.
  • Results: 2.52× execution frequency is achieved by PD-VLA relative to the fundamental VLA model on 7-DoF manipulators.Simulation experiments report acceleration without compromising performance.
  • Results: Real-world experiments show strong applicability across tasks, including dexterous pouring-water manipulation.The study also identifies effective acceleration settings through empirical validation and ablation studies.
  • Contribution: The framework preserves action performance while eliminating autoregressive decoding bottlenecks.This is presented as the first parallel decoding framework for VLA models integrated with action chunking.

II. RELATED WORKS

Related work establishes VLA models as multimodal policies for robotic control and describes action chunking as a strategy for predicting multi-step actions. Evidence on action chunking is positive in laboratory settings but mixed in real-world applications.

  • Vision-Language-Action Models: VLA models combine visual feedback and natural-language instructions to generate executable robot commands.Large VLA models are commonly obtained by fine-tuning pretrained multimodal models on extensive robot datasets.
  • Action Chunking: Action chunking predicts multi-step action sequences and executes all or part of them without intermediate replanning.It can also model and sample action-chunk distributions for sequential decision-making.
  • Action Chunking: Action chunking has shown high-performing policies in laboratory settings, while other studies report contrasting real-world results.Bidirectional decoding has been proposed to balance consistency and reactivity under practical conditions.

C. Acceleration for Vision-Language-Action Models

This section describes VLA acceleration approaches and the PD-VLA pipeline. PD-VLA tokenizes multimodal inputs, uses parallel decoding in the LLM, and detokenizes generated action tokens into robot actions.

  • Acceleration Methods: Existing VLA acceleration methods include quantization, token pruning, dynamic inference depth, quantization-aware training, model replacement, token caching, and compressed tokenization.The paper positions PD-VLA as optimizing the decoding mechanism rather than redesigning the model architecture.
  • PD-VLA Pipeline: PD-VLA tokenizes images, proprioception, and language instructions, feeds them to the LLM in parallel-decoding form, then detokenizes action tokens for the mechanical arm.The figure describes the end-to-end inference flow from multimodal inputs to deployed action values.
  • Model Architecture: The VLA architecture uses a vision encoder to convert static and gripper images into visual tokens before combining them with instruction information.The resulting representations are processed by the language model to generate actions.
  • Action Tokenization: Continuous actions are discretized into 256 uniformly spaced bins and represented using 256 language-model vocabulary tokens.Action tokens across dimensions are concatenated into a textual training label.
  • Action Representation: A 7-dimensional action contains end-effector position coordinates, rotation angles, and gripper state.The dimensions are X, Y, Z, ϕ, θ, ψ, and G.

B. Action Chunking for VLA Models

Action chunking predicts multiple future actions for execution, but longer chunks increase single-inference time and can harm action continuity and effectiveness. This motivates a more efficient decoding method.

  • Action Chunking: Action chunking predicts action sequences over multiple time steps and executes all or part of the sequence.Prior work reports benefits in imitation learning, diffusion policies, and VLA models.
  • Efficiency Tradeoff: Longer action sequences consume more single-inference time, affecting action continuity and effectiveness.The section identifies efficient action decoding as necessary for addressing this tradeoff.

C. Parallel Decoding for VLA Models

The method reframes autoregressive action decoding as nonlinear equations and solves them through parallel fixed-point iterations. It uses decoding horizons and fixed tokens to reduce sequential work while preserving action-sequence modeling.

  • Parallel decoding: PD-VLA reframes autoregressive action decoding as a nonlinear equation system and solves it with Jacobi fixed-point iteration.The approach applies parallel decoding to VLA models with action chunking.
  • Parallel decoding: The model feeds a prompt and an initialized action-token sequence together, replacing causal attention with bidirectional attention to break sequential dependencies.All action tokens can be updated during each iteration.
  • Fixed-point iteration: Jacobi decoding predicts multiple tokens per forward pass, so its updating iterations can satisfy k ≤ n instead of requiring one sequential pass per token.Iterations terminate when consecutive token sequences become identical.
  • Decoding horizon: For action chunks of size m, the response length is l = 7m+2, including beginning and ending tokens.The decoding horizon n determines how many action tokens are handled in each parallel pattern.
  • Decoding horizon: Setting n = l enables inference in a single Jacobi decoding and is intended to preserve the original action distribution's modeling capabilities.The study evaluates horizons n of 7, 16, and 37.
  • Fixed tokens: PD-VLA identifies fixed tokens that are predicted correctly despite preceding incorrect tokens, enabling simultaneous extension of discontinuous correct tokens.These fixed tokens remain unchanged during subsequent updates.

IV. EXPERIMENTS

The experiments evaluate PD-VLA against baselines and acceleration methods, examine component coordination and decoding horizons, and test deployment in real-world robotic systems.

  • Experimental questions: The experiments compare PD-VLA with baselines and other acceleration methods while evaluating component coordination, decoding horizons, and real-world deployment.The study organizes these evaluations as four questions covering effectiveness, coordination, horizon effects, and applicability.

A. Experiments Setup

The evaluation uses simulated manipulation benchmarks, reports task success and completion metrics alongside inference speed, and measures training and implementation requirements.

  • Benchmarks: The CALVIN evaluation uses the ABCD→D setup, comprising 34 tasks across four PyBullet environments with a Franka Panda arm.Reported metrics are success rate and average completed sequential tasks.
  • Benchmarks: LIBERO contains Spatial, Object, Goal, and Long suites, each with 10 tasks evaluated over 50 rollouts.The evaluation reports suite-level success rates and the overall average.
  • Metrics: Inference speed is measured in tokens per second and execution frequency in Hertz to reflect robotic-task requirements.The CALVIN long-horizon challenge also reports five subtask success rates and average completed length.
  • Implementation: The implementation uses Vicuna-7B-v1.5 with CLIP ViT-L/14-336 to build LLaVA-7B-v1.5.The fundamental model is trained for one epoch on 8 NVIDIA H100 GPUs in approximately 10 hours.
  • Implementation: PD-VLA does not incur extra training costs, supporting its characterization as a training-free decoding acceleration method.Its acceleration is applied without additional model training.

B. Results on CALVIN Benchmark

On CALVIN, PD-VLA is reported to improve over the fundamental LLaVA-VLA model while maintaining competitive manipulation performance. Comparisons with FastV and SparseVLM show different speed and performance trade-offs.

  • Comparison with baselines: PD-VLA achieves significant improvements over the fundamental LLaVA-VLA model while delivering competitive manipulation performance.Table II compares manipulation baselines using success rate and average length.
  • Acceleration methods: FastV largely preserves manipulation performance but is slower because attention-token masking introduces additional overhead.This comparison concerns the traditional VLA model with action chunking.
  • Acceleration methods: SparseVLM decreases both success rates and inference speed because token pruning, merging, and recycling incur extra costs.The reported trade-off differs from FastV's performance-preserving but slower behavior.

C. Results on LIBERO Benchmark

PD-VLA improves performance and inference efficiency through complementary action chunking and parallel decoding. The combined approach achieves strong benchmark results while addressing autoregressive decoding bottlenecks.

  • PD-VLA achieves the best average performance on LIBERO-Long, attaining a 91.7% success rate.
  • 2.34 improvements in success rates and 2.52× execution frequency are achieved compared with the fundamental LLaVA-VLA model.
  • Action chunking improves consistency and stability, showing 2.42 improvements in average length, while reducing inference counts improves execution frequency 2.51×.
  • Parallel decoding increases average decoding speed by 1.28×, reducing single-inference time for high-frequency inference.
  • Action chunking and parallel decoding complement each other by balancing action consistency with high-frequency inference.

E. Decoding Horizon and Acceleration Phenomenon

Decoding-horizon experiments show that longer horizons can increase decoding speed, but horizon choice affects execution frequency and manipulation performance. Real-world experiments further demonstrate applicability across robotic tasks.

  • Decoding Horizon and Acceleration Phenomenon: The decoding horizon of 37 shows the strongest manipulation abilities with the highest decoding speed.This setting preserves the original action-distribution modeling by predicting whole action sequences together.
  • Decoding Horizon and Acceleration Phenomenon: Decoding speed improves from 41.48 to 52.84 tokens/second as the number of fixed tokens increases with decoding horizon.
  • Decoding Horizon and Acceleration Phenomenon: At n = 37, maximum speed reaches approximately twice that of n = 7 and autoregressive decoding because fewer iterations are required.
  • Real-world Experiments: PD-VLA produces higher real-world success rates than LLaVA-VLA across the evaluated robotic tasks.
  • Real-world Experiments: PD-VLA improves success rates by 20% and 30% on two tasks, while achieving a 50% higher success rate than LLaVA-VLA for pouring water.
  • Real-world Experiments: The real-world setup uses a 6-DOF Unitree Z1-Pro arm with a 1-DOF gripper and evaluates three tasks using 50 demonstrations and 10 episodes per task.

V. CONCLUSION

PD-VLA accelerates VLA inference with parallel decoding while retaining competitive success rates and supporting high-frequency robotic control. Real-world experiments validate its effectiveness, while future work targets faster fixed-point convergence.

  • PD-VLA predicts action tokens simultaneously over several iterations instead of sequentially, improving decoding efficiency.
  • Extensive experiments show improved inference speed and execution frequency while maintaining competitive success rates.
  • Future work will optimize the decoding algorithm and model to prevent redundant iterations during parallel decoding.The goal is faster convergence to a fixed point.
Loading 2503.02310v2…