Source-linked AI summary
VLANeXt: Recipes for Building Strong VLA Models
Xiao-Ming Wu, Bin Fan, Kang Liao, Jian-Jian Jiang, Runze Yang, Yihang Luo, Zhonghua Wu, Wei-Shi Zheng, Chen Change Loy
TL;DR
VLA research lacks a consistent basis for determining which design choices matter because training protocols and evaluation settings vary. This paper systematically studies the design space under a unified framework, producing the VLANeXt recipe and model, which achieves state-of-the-art performance on LIBERO and LIBERO-plus while supporting real-world manipulation.
Problem
Differences in VLA training protocols and evaluation setups make it difficult to identify which shared design choices truly matter.
Method
The study starts from an RT-2-like baseline and systematically examines foundational components, perception, and action modeling under a unified framework and evaluation protocol.
Results
VLANeXt achieves state-of-the-art performance on LIBERO and LIBERO-plus and adapts effectively to real-world manipulation tasks.
Takeaways & Limitations
The findings form a practical recipe centered on principled choices in VLM-policy interaction, multimodal fusion, and temporal action modeling.
Abstract
from arXiv · showhide
Following the rise of large foundation models, Vision-Language-Action models (VLAs) emerged, leveraging strong visual and language understanding from Vision-Language Models for general-purpose policy learning. Yet, the current VLA landscape remains fragmented and exploratory. Although many groups have proposed their own VLA models, inconsistencies in training protocols and evaluation settings make it difficult to identify which design choices truly matter. To bring structure to this evolving space, we reexamine the VLA design space under a unified framework and evaluation setup. Starting from a simple VLA baseline similar to RT-2, which is the origin of VLA, we systematically dissect design choices along three dimensions: foundational components, perception essentials, and action modelling perspectives. From this study, we distill 12 key findings that together form a practical recipe for building strong VLA models. The outcome of this exploration is a simple yet effective model, VLANeXt. It outperforms the state-of-the-art methods on the LIBERO and LIBERO-plus benchmarks and demonstrates strong performance in real-world experiments. We release a unified and easy-to-use codebase to reproduce our findings, explore the design space, and develop new VLA variants on top of a shared foundation. The codebase is available at https://github.com/DravenALG/VLANeXt.
1. Introduction
VLAs offer a scalable route to language-conditioned robot policies, but fragmented training and evaluation practices obscure which design choices matter. VLANeXt addresses this gap through unified analysis and achieves strong benchmark performance with a practical design recipe.
- Motivation: VLAs combine visual observations and language instructions to map directly to robot actions using pretrained vision-language models.This paradigm aims to support general-purpose, language-conditioned robot policies.
- Motivation: Inconsistent training protocols and evaluation setups make it difficult to identify which choices in the shared VLA design space truly matter.The paper characterizes the field as exploratory and fragmented despite rapid progress.
- Approach: The study reexamines VLA designs systematically under a unified framework and evaluation protocol, starting from an RT-2-like baseline.Variants are evaluated on LIBERO and LIBERO-plus while exploring foundational components, perception, and action modeling.
- Findings: The authors identify design findings including soft VLM-policy coupling, VLM-side proprioception conditioning, and frequency-domain action modeling.They also report that video inputs fail to distill useful information for action learning under their setup.
- Outcome: VLANeXt achieves state-of-the-art performance on LIBERO and LIBERO-plus and adapts effectively to real-world manipulation tasks.The model is derived from the design principles identified through the systematic exploration rather than aggressive scaling or task-specific engineering.
2. Recipes for Building Strong VLA Models
The study systematically evaluates VLA design choices from a unified RT-2/OpenVLA-style baseline across LIBERO and LIBERO-plus. The resulting recipe combines stronger components, multimodal perception choices, and continuous action modeling to form VLANeXt.
- Evaluation Setup: The roadmap evaluates VLA variants on LIBERO and LIBERO-plus, using LIBERO spatial as the primary testbed while findings generalize across other suites.LIBERO-plus tests robustness under controlled and unseen perturbations.
- Foundational Components: A deeper dedicated policy module, action chunking, and soft layer-wise VLM-policy connections improve the baseline design.The final policy uses multiple tokens and a 12-layer policy network, while action chunking adopts a chunk size of 8.
- Action Modelling Perspectives: Flow matching is selected for continuous action learning after regression performs best initially and diffusion objectives perform close behind.Classification-based approaches perform worst, while flow matching can surpass regression at higher performance levels.
- Foundational Components: Stronger VLM backbones improve VLA performance, with Qwen3-VL-2B chosen as an effective and efficient subsequent backbone.Qwen3-VL-4B outperforms Qwen3-VL-2B, which outperforms LLaMA-3.2-3B and PaliGemma-3B.
- Perception Essentials: Multi-view observations and VLM-side proprioception conditioning improve performance, whereas redundant temporal history slightly degrades action generation.Combining third-person and wrist views significantly improves performance, and VLM-side proprioception conditioning performs best among tested variants.
- Action Modelling Perspectives: World modeling improves action generation but nearly triples training time, while a lightweight frequency-domain auxiliary objective provides a similar or slightly stronger gain with negligible overhead.The frequency-domain strategy slightly surpasses world modeling and is retained in the final recipe; world modeling is excluded.
3. Benchmarks Evaluations
The evaluation uses the LIBERO ecosystem to measure standard task learning and robustness to unseen visual, physical, and semantic perturbations. VLANeXt achieves state-of-the-art performance on LIBERO and improves success rate by 13% over OpenVLA-OFT on LIBERO-plus.
- LIBERO evaluates Spatial, Object, Goal, and Long suites, each containing 500 expert demonstrations across 10 tasks.
- LIBERO-plus extends evaluation with unseen visual, physical, and semantic perturbations across the four LIBERO suites.
- All recipe variants are directly fine-tuned on LIBERO using 10,000 training steps and a batch size of 256 for fair comparisons.
- VLANeXt achieves state-of-the-art performance on the LIBERO benchmark under the reported success-rate comparisons.
- 13% improvement in success rate over OpenVLA-OFT is reported for VLANeXt on LIBERO-plus.
4. Real-World Evaluations
The real-world evaluation tests VLANeXt on single-arm and bimanual manipulation tasks using limited task demonstrations and matched baseline fine-tuning. The model performs well in deployment and adapts to bimanual tasks even without bimanual training.
- Four real-world tasks comprise two single-arm tasks and two bimanual tasks, including table cleaning, drawer manipulation, basket lifting, and bimanual table cleaning.
- Each task uses 50 training episodes and 20 evaluation trials, with performance reported as success rate.
- The model is pretrained on DROID for 100k steps and fine-tuned for 20k steps per task; bimanual adaptation reinitializes the proprioception projector and final action-generation layer.
- VLANeXt is compared with OpenVLA-OFT and π0 after fine-tuning their pretrained checkpoints on each task in the same manner.
- VLANeXt performs well in real-world experiments and achieves decent bimanual adaptation without bimanual training, indicating cross-embodiment adaptability.
5. Conclusion
The conclusion attributes VLA gains to principled choices in information integration and temporal action modeling rather than standalone architectural novelty. It also emphasizes efficiency and controlled exploration as priorities for future progress.
- VLANeXt revisits the VLA pipeline and shows that principled design choices within a unified framework can produce performance gains.
- VLM-policy interaction, proprioception fusion, and temporal action modeling are identified as central design factors.
- Soft VLM-policy coupling and VLM-side proprioception conditioning can meaningfully influence performance, making information placement important.
- Frequency-domain objectives transfer time-series learning ideas to robotics, while world modeling improves performance at substantially higher training cost.
- The released lightweight framework is intended to support controlled design-space exploration and shared progress, with broader embodiments and longer-horizon reasoning left for future work.
Impact Statement
The impact statement frames the work as advancing machine learning research on Vision-Language-Action models for robotic control and more capable embodied agents.
- The work advances Vision-Language-Action models for robotic control and contributes to developing more capable embodied agents.
A.1. Qualitative Experiments
The appendix provides qualitative demonstrations of the method across LIBERO, LIBERO-plus perturbations, and real-world tasks.
- Further demonstrations span LIBERO, LIBERO-plus, and real-world settings.
- The appendix also presents qualitative experiments on real-world tasks.
- Qualitative experiments cover all four suites of the LIBERO benchmark.
- Additional demonstrations evaluate the method across seven perturbation types in the same LIBERO-plus task.
A.2. More Ablation Experiments
Additional ablations show that the proposed recipe remains consistent across LIBERO suites, LIBERO-plus, and a real-world clean-table task.
- Ablations on the Object, Goal, and Long suites follow the main Spatial-suite experimental setup.
- Results remain consistent with the Spatial-suite experiments on LIBERO and LIBERO-plus, supporting the robustness of the recipe.
- The appendix also evaluates the recipe on a clean-table task in real-world settings.
A.3. Detailed Experimental Settings
The appendix documents shared training settings and table conventions for VLANeXt experiments across LIBERO and LIBERO-plus.
- The final model uses the same parameters across all four benchmark suites.
- Table 5 defines abbreviations for frequency-domain loss, proprioception input, multi-view inputs, and soft connections.
- Table 6 reports the ablation study on the real-world clean-table task.
- Table 7 lists VLANeXt training hyperparameters for LIBERO and LIBERO-plus across four suites.
B. Revisiting Robot Learning and VLA Models
Robot learning applies machine learning to robotic control, while VLA models integrate foundation models into robot learning for vision- and language-conditioned action generation.
- Robot learning uses machine learning techniques to enable robots to interact with the physical world and acquire diverse skills.
- Robot-learning tasks are commonly categorized as locomotion or manipulation according to the active components being controlled.
- VLA models integrate large foundation models into robot learning and were pioneered by RT-2.
- Subsequent VLA models address challenges including 3D spatial information and intermediate data such as subtasks or trajectory traces.