Source-linked AI summary
VLA-0: Building State-of-the-Art VLAs with Zero Modification
Ankit Goyal, Hugo Hadfield, Xuning Yang, Valts Blukis, Fabio Ramos
TL;DR
VLAs raise an open question about how to adapt VLMs for robotic action prediction without adding unnecessary complexity. VLA-0 answers by generating numerical actions as text with an unchanged VLM, and reports state-of-the-art results across LIBERO comparisons, including against several large-scale pretrained models. The paper also identifies training and inference techniques needed for this simple design.
Problem
The paper asks whether VLAs can predict robotic actions without modifying a VLM’s vocabulary or adding architectural components.
Method
VLA-0 prompts a VLM to generate numerical robotic actions directly as text and uses action masking during training plus prediction ensembling during testing.
Results
On LIBERO, VLA-0 outperforms all compared VLAs trained without large-scale robotic data and surpasses multiple models pretrained on large-scale action data.
Takeaways & Limitations
A VLA can achieve strong benchmark performance while preserving the underlying VLM and avoiding vocabulary or architectural changes.
Takeaways & Limitations
The paper leaves VLA-0’s performance with large-scale action-data training and inference-speed improvements for future work.
Abstract
from arXiv · showhide
Vision-Language-Action models (VLAs) hold immense promise for enabling generalist robot manipulation. However, the best way to build them remains an open question. Current approaches often add complexity, such as modifying the existing vocabulary of a Vision-Language Model (VLM) with action tokens or introducing special action heads. Curiously, the simplest strategy of representing actions directly as text has remained largely unexplored. This work introduces VLA-0 to investigate this idea. We find that VLA-0 is not only effective; it is surprisingly powerful. With the right design, VLA-0 outperforms more involved models. On LIBERO, a popular benchmark for evaluating VLAs, VLA-0 outperforms all existing methods trained on the same robotic data, including $π_0.5$-KI, OpenVLA-OFT and SmolVLA. Furthermore, without large-scale robotics-specific training, it outperforms methods trained on large-scale robotic data, like $π_0.5$-KI, $π_0$, GR00T-N1 and MolmoAct. These findings also translate to the real world, where VLA-0 outperforms SmolVLA, a VLA model pre-trained on large-scale real data. This paper summarizes our unexpected findings and spells out the specific techniques required to unlock the high performance of this simple yet potent VLA design. Visual results, code, and trained models are provided here: https://vla0.github.io/.
I. INTRODUCTION
VLA-0 investigates whether a VLM can become a capable VLA by generating robotic actions as text without architectural changes. With a careful training and testing recipe, this simple design achieves strong benchmark performance against more involved alternatives.
- Approach: VLA-0 represents actions as text, avoiding changes to the VLM vocabulary, new tokens, and additional architectural components.Numerical actions such as coordinates or joint angles can be generated through the VLM’s native text-generation capability.
- Motivation: The paper frames the central design question as whether simpler VLA architectures can replace approaches that modify vocabularies or add action-generation components.
- Results: VLA-0 outperforms all compared VLAs trained without large-scale robotic data on LIBERO and also surpasses several models pretrained on large-scale action data.The introduction reports that VLA-0 also transfers this advantage to real-world evaluation against SmolVLA.
- Training and testing recipe: Random masking of action text during training and ensembling previous predictions during testing are identified as important recipe components for high performance.
- Contributions: The stated contributions are demonstrating state-of-the-art results without changing VLM architecture and devising the corresponding training and testing recipe.
II. RELATED WORK
Prior VLA methods modify tokenization, add action heads, or build specialized architectures. VLA-0 instead generates complete numerical actions directly as text, using a simpler end-to-end design that can outperform specialized policies.
- Existing VLA families: Existing VLA families modify the vocabulary with discrete action tokens, add auxiliary generative heads, or introduce specialized architectures and training pipelines.
- VLA-0: VLA-0 directly generates numerical action strings end-to-end, unlike LLARVA’s two-stage trajectory-then-action process.Its reported recipe includes action-token masking and prediction ensembling.
- VLA-0: Unlike auxiliary-head approaches, VLA-0 preserves the underlying VLM architecture while representing actions such as end-effector coordinates as text.
- Robot-learning policies: Diffusion Policy learns robotic policies with conditional diffusion, whereas VLA-0 follows the VLA paradigm by leveraging pretrained visual-language representations.
- Comparison with robot-learning policies: VLA-0 builds on pretrained VLM representations and reportedly outperforms Diffusion Policy on benchmark tasks using only in-domain action data.
A. Background
VLMs jointly process visual and textual inputs by projecting visual features into an LLM’s embedding space. VLA-0 is built on Qwen-VL-2.5-3B, selected for competitive performance, computational efficiency, and open weights.
- Vision-Language Models: A VLM combines a pretrained vision encoder with an LLM, projecting visual features into the LLM’s embedding space for multimodal text generation.
- VLA-0 backbone: VLA-0 uses the publicly available 3-billion-parameter Qwen-VL-2.5 model, although the method is described as applicable to other VLMs.
B. Method: VLA-0
VLA-0 represents robot actions as text while preserving the underlying VLM architecture. Its performance depends on a recipe combining integer action decoding, masked action augmentation, and ensemble prediction.
- VLA-0 preserves the VLM architecture by introducing no new tokens, vocabulary changes, or neural-network layers.
- Actions are normalized to integers and generated as text, allowing arbitrary action resolution without modifying the VLM vocabulary.
- Randomly masking characters in target action strings encourages visual- and instruction-based reasoning instead of numerical-sequence completion.
- The method uses a full fine-tuning procedure for the base VLM.
A. Setup
The real-world evaluation uses an SO100 robot and four manipulation tasks, with success rates reported for individual suites and overall performance.
- Four SO100 tasks evaluate block reorientation, apple pushing, banana placement, and cupcake placement.Each task uses 100 training demonstrations.
- VLA-0 is compared with SmolVLA on four real-world tasks.
- Performance is reported as success rates for each individual suite and as an overall average.
B. Baselines
The study compares VLA-0 with diffusion-policy and VLA baselines spanning discrete-token, generative-action, and custom-architecture families, including models with and without large-scale action pretraining.
- The baseline set includes Diffusion Policy plus discrete-token, generative-action, and custom-architecture VLA families.
- VLA-0 outperforms all LIBERO VLA baselines without large-scale robotic pretraining, beating the second-best method by 1.4 points on average.
- Without large-scale action pretraining, VLA-0 surpasses several pretrained models and achieves the second-best average rank of 2.8.OpenVLA-OFT leads with an average rank of 1.5.
- VLA-0’s simple strategy remains effective against models trained with large-scale action data.
D. Real-World Evaluation
Real-world evaluation on SO100 compares VLA-0 with the large-scale-data-pretrained SmolVLA across four tasks. VLA-0 achieves the stronger average result despite being trained from scratch.
- The evaluation uses the SO100 robot and LeRobot framework to compare VLA-0 with SmolVLA.SmolVLA was specifically trained on the large-scale SO100 dataset.
- Inference runs at 4 Hz on a desktop with a 5090 GPU using standard PyTorch.The real-world system does not ensemble actions.
- VLA-0 outperforms SmolVLA by 12.5 points across four real-world task success rates.SmolVLA was pretrained on large-scale SO100 data, whereas VLA-0 was not.
E. Ablations
Ablations on LIBERO identify action ensembling and masked action augmentation as beneficial, while resolution matters below 1000 and image tiling does not materially affect performance.
- Action ensembling improves the overall success rate by 2 points, making it a critical VLA-0 component.
- Removing masked action augmentation decreases the success rate by 1.2 points, indicating a modest but consistent benefit.
- Reducing action resolution from 1000 to 250 lowers success by 1.5 points, while increasing it to 4000 adds no gains.
- Tiling multiple image observations into one composite image has no discernible performance impact compared with separate inputs.
V. CONCLUSIONS AND LIMITATIONS
VLA-0 preserves the base VLM without changing tokenization or adding architectural components, yet with the right recipe it outperforms more involved strategies. The authors identify large-scale action-data training and faster inference as key directions for future work.
- VLA-0 preserves the base VLM without altering tokenization or adding new architectural components, yet outperforms more involved strategies with the right recipe.
- A key limitation is that VLA-0’s performance when trained with large-scale action data remains unexplored.
- Improving VLA-0 inference speed through quantization and distillation is an identified direction for future research.