PaperlayerView exact revision on arXiv

Source-linked AI summary

VQ-VLA: Improving Vision-Language-Action Models via Scaling Vector-Quantized Action Tokenizers

Yating Wang, Haoyi Zhu, Mingyu Liu, Jiange Yang, Hao-Shu Fang, Tong He

arXiv:2507.01016v1cs.ROcs.CV

TL;DR

VLA action tokenizers need scalable, accurate representations that support efficient modeling across short- and long-horizon tasks. The paper proposes a convolutional residual VQ-VAE action tokenizer trained on over 100 times more data than previous methods, and experiments show improved VLA performance, inference speed, and long-horizon capability, including 50% and 30% success rates on two long-horizon tasks.

  • Problem

    The paper investigates how scalable and accurate action tokenization can improve VLA performance, particularly for long-horizon planning, while enhancing training and inference efficiency.

  • Method

    The paper proposes a convolutional residual VQ-VAE action tokenizer trained on an action-trajectory dataset expanded by more than 100 times over previous approaches and integrated into VLA models for direct token prediction.

  • Results

    The tokenizer improves VLA performance, inference speed, and long-horizon capabilities; on two long-horizon tasks, VQO+L+M achieves 50% and 30% success rates, respectively, versus baselines as low as 15% or nearly 0.

  • Takeaways & Limitations

    Large-scale simulated action data can effectively scale the tokenizer, while the resulting action tokens support downstream simulated and real-world robotic manipulation tasks.

  • Takeaways & Limitations

    The tokenizer could be extended to larger simulated datasets, combined with VLM distillation and quantization, and improved through architecture changes such as conditioning on action-data frequency.

Abstract

from arXiv · show

In this paper, we introduce an innovative vector quantization based action tokenizer built upon the largest-scale action trajectory dataset to date, leveraging over 100 times more data than previous approaches. This extensive dataset enables our tokenizer to capture rich spatiotemporal dynamics, resulting in a model that not only accelerates inference but also generates smoother and more coherent action outputs. Once trained, the tokenizer can be seamlessly adapted to a wide range of downstream tasks in a zero-shot manner, from short-horizon reactive behaviors to long-horizon planning. A key finding of our work is that the domain gap between synthetic and real action trajectories is marginal, allowing us to effectively utilize a vast amount of synthetic data during training without compromising real-world performance. To validate our approach, we conducted extensive experiments in both simulated environments and on real robotic platforms. The results demonstrate that as the volume of synthetic trajectory data increases, the performance of our tokenizer on downstream tasks improves significantly-most notably, achieving up to a 30% higher success rate on two real-world tasks in long-horizon scenarios. These findings highlight the potential of our action tokenizer as a robust and scalable solution for real-time embodied intelligence systems, paving the way for more efficient and reliable robotic control in diverse application domains.Project website: https://xiaoxiao0406.github.io/vqvla.github.io

1. Introduction

VQ-VLA develops a scalable action tokenizer for VLA models, motivated by the need for precise, efficient representations that support long-horizon action modeling. It combines progressive training on real and synthetic trajectories with broad data scaling and evaluates the resulting system in simulated and real environments.

  • Motivation: Action tokenization compresses continuous action sequences into compact tokens, reducing the complexity of learning high-dimensional data.Action sequences are comparatively compressible because of their spatio-temporal continuity.
  • Motivation: More precise tokenization produces more pronounced improvements in long-horizon action modeling, motivating well-scaled trajectory training.The paper emphasizes scalability and accuracy as central goals for action tokenization.
  • Approach: The tokenizer uses progressive training: noisy real-world trajectories are combined gradually with cleaner, smoother synthetic trajectories.This strategy is designed to move the VQ model toward smoother and more stable representations.
  • Approach: The training dataset is expanded by more than 100 times relative to prior single-task approaches, covering a broad spectrum of downstream tasks.The enlarged dataset combines trajectories across multiple robotic datasets.
  • Findings: Experiments in simulated and real-world environments show improved success rates with more simulated data, faster and smoother inference, and reduced cumulative errors in long-horizon tasks.These findings are reported as the paper’s main experimental outcomes.
  • Contributions: The paper contributes a general convolutional residual VQ-VAE framework, demonstrates scaling with simulated action data, and reports gains in performance, inference speed, and long-horizon capability.The framework is intended for action tokenization in VLA models.

2. Releated Works

VLA models connect visual-language understanding to robot control by predicting discrete action tokens. Prior work uses several discretization and predictive-representation strategies to make continuous actions compatible with discrete generative models.

  • Vision-Language-Action Models: Vision-language-action models map multimodal inputs, including images and instructions, to discrete robot action sequences.These actions can include 6DoF motion and gripper control.
  • Vision-Language-Action Models: Discretizing continuous actions into bins enables VLA integration with vision-language models and supports zero-shot generalization from web-scale pretraining.RT-1 and RT-2 are cited as examples of this approach.
  • Action Tokenization: Existing approaches discretize low-level actions through per-dimension or per-timestep binning, VQ-VAE, and cosine-transform methods.These methods adapt continuous action distributions to discrete generative models.
  • Action Tokenization: Other methods extract predictive signals from action-less video as pixels, trajectories, or latent motion.These representations provide alternatives to directly discretizing low-level actions.

3. Methods

VQ-VLA replaces OpenVLA’s simple binning tokenizer with a frozen convolutional residual VQ-VAE that encodes, quantizes, and reconstructs action sequences. The resulting tokens are assigned to the VLM for direct prediction, while training and data choices target temporal structure, scalability, and efficiency.

  • Backbone: OpenVLA discretizes continuous robot actions into 256 bins per dimension, using the 1st and 99th training-data percentiles to reduce outlier effects.VQ-VLA uses OpenVLA as its backbone model.
  • Tokenizer Architecture: Two-dimensional temporal convolutions replace MLPs to capture local relationships and hierarchical temporal dependencies in action sequences.The design addresses scaling limitations attributed to MLP-based processing.
  • Pipeline: The VQ-VLA pipeline first trains a convolutional residual VQ-VAE, then freezes it as OpenVLA’s action tokenizer and fine-tunes OpenVLA with LoRA.The VQ-VAE is trained on Open X-Embodiment, LIBERO, and ManiSkill datasets.
  • Encoding and Quantization: The encoder maps an action sequence a_t:t+n into a latent embedding x, which residual vector quantization decomposes into quantized residuals across N_q stages.Each stage quantizes the current residual before the next residual is computed.
  • Decoding: A temporal deconvolutional decoder reconstructs the action sequence from the quantized embedding, preserving temporal structure.The reconstruction is produced as ˆa_t:t+n = ϕ_dec(q(x)).
  • Optimization: Training minimizes a weighted combination of reconstruction, codebook, and commitment losses, with λ balancing the components and set to 4 in experiments.The stop-gradient operation appears in the vector-quantization formulation.
  • Training Setup: The tokenizer is trained in three dataset configurations—Open X-Embodiment alone, plus LIBERO, and plus ManiSkill—to test scaling with simulated data.The model uses action sequences alone as input and is trained on a single A100 GPU.
  • Training Setup: Time and action-type embeddings improve processing of temporal patterns and the distinct roles of the seven action dimensions.The paper reports improved latent representations and overall tokenizer performance with these embeddings.

4. Experiments

The experiments evaluate the action tokenizer in simulation and on real robots, measuring VLA performance, inference speed, long-horizon capabilities, and key design choices.

  • Evaluation Setup: Experiments use the LIBERO simulator and subsequent real-world robotic tests to assess tokenizer scalability and downstream VLA effects.The evaluation includes performance, inference speed, long-horizon capability, and ablation studies.

4.1. Simulation Experiments

Simulation experiments evaluate VQ-VAE action tokenizers on LIBERO, showing that convolutional residual architectures and larger synthetic-data mixtures improve downstream performance. VQM+R reaches 80.98% on LIBERO-90, outperforming the OpenVLA baseline by 7.45%.

  • Evaluation setup: The evaluation uses the full LIBERO task suite, including LIBERO-90’s 90 short-horizon tasks and four additional 10-task suites.Each task has 50 demonstrations, and LIBERO-90 is used as the evaluation benchmark.
  • Tokenizer training: Two convolutional residual VQ-VAE variants are trained on ManiSkill alone or a ManiSkill–RLBench mixture before being used as frozen OpenVLA action tokenizers.Both tokenizer models are trained on a single A100 GPU with batch size 1024 for approximately one week.
  • Data scaling: Success rates consistently improve when tokenizer training expands from individual LIBERO tasks to the entire LIBERO dataset.This provides preliminary evidence that the action tokenizer scales with training-data volume.
  • Architecture ablation: Temporal convolutional encoders and decoders outperform MLP-based residual VQ-VAE architectures, particularly when trained on the full LIBERO dataset.The comparison indicates that temporal convolutions better capture action dependencies and improve success rates.

4.2. Real-Word Experiment

Real-world experiments evaluate VQ-VLA on six manipulation tasks spanning short- and long-horizon behavior. Adding synthetic trajectories improves success, with VQO+L+M outperforming the baseline and nearly tripling inference speed at compression ratio 5.

  • Experiment setup: The real-world benchmark contains six manipulation tasks—four short-horizon and two long-horizon—with 50 demonstrations per task and 20 evaluation trials.Experiments use a Franka Research3 arm with a fixed third-person RealSense D435 camera and 20 Hz control.
  • Short-horizon tasks: Adding synthetic trajectories raises average success from 23% to 46.25% across the evaluated short-horizon tasks.The co-trained VQO+L+M model incorporates 120k ManiSkill trajectories alongside Open X-Embodiment and LIBERO data.
  • Short-horizon tasks: 30%: synthetic-data training improves success on the “Flip the pot upright” task, while VQ models achieve at least 20% on “Pull out a tissue paper” versus 5% for baseline.The tissue task requires continuous, fine-grained grasping and pulling motions.
  • Short-horizon tasks: VQO+L improves average short-horizon success by only 0.5% over VQO, whereas the much larger ManiSkill dataset yields a significant improvement when used for training.The passage attributes the small LIBERO effect to its limited dataset size and notes that ManiSkill is 50 times larger.
  • Long-horizon tasks: VQO+L+M reaches 50% and 30% success on the two long-horizon tasks, where baseline success is as low as 15% or nearly 0.For the drawer task, VQO+L+M opens the drawer in all test cases, while the baseline proceeds beyond opening in most trials only rarely.
  • Long-horizon tasks: Predicting multiple actions per inference step reduces error accumulation over extended sequences and supports long-term planning and execution.The VQ-VAE tokenizer represents longer action sequences rather than requiring one action prediction at a time.
  • Sim&Real domain gap: VQL performs comparably to VQO+L and VQO on real-world tasks, indicating a minimal domain gap between synthetic and real-world data.VQL is trained exclusively on LIBERO data and tested on three real-world tasks.
  • Overall real-world results: 23.25%: VQO+L+M’s average success rate is higher than the baseline on both short-horizon and long-horizon real-world tasks.The ordering VQO+L+M > VQO+L > VQO indicates progressively better results as synthetic data is incorporated.

4.3. Ablation Studies

The ablations compare VQ-based action chunking with autoregressive chunking and test embedding integration, showing advantages for performance, speed, and action diversity.

  • Action Chunking via VQ-VAE and Autoregressive Output: Autoregressive action chunks contain remarkably similar action values, indicating limited diversity in their predictions.The approach also produces smaller spatial action magnitudes and slower execution than VQ-VAE chunking, even with the same chunk size.
  • Action Chunking via VQ-VAE and Autoregressive Output: VQ-VAE action chunking improves inference speed and VLA performance by generating more effective and diverse action sequences.The paper identifies this approach as better suited to real-world long-horizon tasks.
  • Action Chunking via VQ-VAE and Autoregressive Output: The action-chunking ablation evaluates one simulator task and two real-world tasks, with autoregressive OpenVLA output performing substantially worse than the VQ-based approach.The comparison indicates that VQ-based chunking is more effective across the evaluated settings.
  • Embedding Integration: Adding time and action-type embeddings significantly improves success rate compared with processing raw action sequences directly.The improvement is attributed to better representation of structured action sequences, especially for the evaluated embedding ablation.

5. Limitations and Future Works

The paper identifies larger simulation datasets, model distillation and quantization, and improved tokenizer conditioning as directions for future work.

  • Future Work: The tokenizer could be extended to larger-scale simulated datasets, including RLBench based on CoppeliaSim.This is presented as a future scaling direction for simulated action data.
  • Future Work: Inference-speed gains from decoding multi-step action sequences could be combined with VLM distillation and quantization techniques.The paper frames these combinations as opportunities for further improvement.
  • Future Work: The tokenizer architecture could be improved by conditioning on action-data frequency.The paper suggests this as an additional architectural refinement.

6. Conclusions

The paper presents a general convolutional residual VQ-VAE action-tokenizer framework that integrates with state-of-the-art VLA models and transfers across robotic manipulation settings.

  • Conclusion: The VQVAE is trained on 100 times more data than previous methods and transfers directly to real-world and simulated robotic manipulation tasks.Experiments in both environments validate improved VLA performance and accelerated inference.
  • Conclusion: The framework improves VLA policy performance and accelerates inference while scaling effectively with large-scale simulated data.These conclusions summarize the paper’s simulated and real-world experimental validation.
Loading 2507.01016v1…