PaperlayerView exact revision on arXiv

Source-linked AI summary

Vlaser: Vision-Language-Action Model with Synergistic Embodied Reasoning

Ganlin Yang, Tianyi Zhang, Haoran Hao, Weiyun Wang, Yibin Liu, Dehui Wang, Guanzhou Chen, Zijian Cai, Junting Chen, Weijie Su, Wengang Zhou, Yu Qiao, Jifeng Dai, Jiangmiao Pang, Gen Luo, Wenhai Wang, Yao Mu, Zhi Hou

arXiv:2510.11027v2cs.CV

TL;DR

Vlaser addresses the gap between embodied VLM reasoning and downstream VLA policy learning by combining embodied-reasoning data with a robot-control action expert. It reports strong embodied-benchmark performance and finds that in-domain data is more effective for downstream policy learning than out-of-domain reasoning data.

  • Problem

    It remains poorly understood which multimodal data streams and abilities most improve downstream VLA models, despite work on embodied reasoning and extending VLMs to robot control.

    Source: §1 · pp. 1–2

  • Method

    Vlaser combines a VLM enhanced with the Vlaser-6M embodied dataset and a flow-matching action expert for predicting future actions from a single-frame observation.

    Source: §1 · p. 2, §2.3 · p. 5

  • Results

    Vlaser reports state-of-the-art performance across embodied reasoning benchmarks and state-of-the-art results on Bridge with competitive results on Google Robot for end-to-end control.

    Source: §1 · p. 3, §6 · p. 10

  • Takeaways & Limitations

    In-domain robot-interaction data is more effective than out-of-domain embodied-reasoning data for accelerating VLA convergence and increasing task success rates.

    Source: §1 · p. 3

    The study’s experiments were conducted in simulation; it lacks real-robot experiments.

    Source: §5 · p. 10

Abstract

from arXiv · show

While significant research has focused on developing embodied reasoning capabilities using Vision-Language Models (VLMs) or integrating advanced VLMs into Vision-Language-Action (VLA) models for end-to-end robot control, few studies directly address the critical gap between upstream VLM-based reasoning and downstream VLA policy learning. In this work, we take an initial step toward bridging embodied reasoning with VLA policy learning by introducing Vlaser - a Vision-Language-Action Model with synergistic embodied reasoning capability, which is a foundational vision-language model designed to integrate high-level reasoning with low-level control for embodied agents. Built upon the high-quality Vlaser-6M dataset, Vlaser achieves state-of-the-art performance across a range of embodied reasoning benchmarks - including spatial reasoning, embodied grounding, embodied QA, and task planning. Furthermore, we systematically examine how different VLM initializations affect supervised VLA fine-tuning, offering novel insights into mitigating the domain shift between internet-scale pre-training data and embodied-specific policy learning data. Based on these insights, our approach achieves state-of-the-art results on the WidowX benchmark and competitive performance on the Google Robot benchmark.

1 INTRODUCTION

The introduction identifies an underexplored gap between embodied VLM reasoning and downstream VLA policy learning, including uncertainty about which data streams best support transfer. It presents Vlaser and Vlaser-6M as a response, reporting broad embodied-benchmark performance and findings about in-domain data for policy learning.

1 INTRODUCTION

  • Vlaser-6M is constructed through a data engine that curates, reorganizes, and annotates public datasets across embodied QA, grounding, spatial reasoning, planning, and simulation data.The introduction presents the resulting dataset as the foundation for Vlaser’s embodied reasoning benchmarks.Source: §1 · p. 2
  • In-domain robot-interaction data is more effective than out-of-domain embodied-reasoning data at accelerating VLA fine-tuning convergence and increasing task success rates.The reported upstream reasoning gains from out-of-domain data may not translate directly or prominently to downstream VLA performance.Source: §1 · p. 3
  • The authors introduce Vlaser as an adaptable model extending InternVL with embodied reasoning and end-to-end robot control, with weights, data-generation pipeline, code, and dataset planned for public release.The contribution is framed as supporting reproducibility and future research.Source: §1 · p. 3
  • Vlaser achieves top-tier results across embodied reasoning benchmarks spanning visual grounding, task planning, spatial reasoning, and simulation-based robot evaluation.The introduction describes generalization across open-loop inference and closed-loop control settings.Source: §1 · p. 3

2 METHOD

Vlaser combines an InternVL3 vision-language backbone with a low-level action expert, supported by the Vlaser-6M data engine and a two-stage VLM-pretraining and VLA-finetuning recipe.

2.1 MODEL STRUCTURE

  • Vlaser’s 2B and 8B models use an InternVL3 backbone with InternViT vision encoders and Qwen2.5 language models.The models are described as optimized for robot computational constraints and emphasize embodied common-sense reasoning and end-to-end control.Source: §2.1 · p. 4

2.2 VLASER DATA ENGINE

  • Vlaser-6M combines embodied grounding, general and spatial reasoning, planning, and in-domain data for downstream VLA learning.The data engine includes 1.5 million grounding pairs plus 300k annotations, 1.2 million RoboVQA pairs, 500k spatial items, 400k planning examples, and 2 million in-domain pairs.Source: §2.2 · pp. 4–5

2.3 TRAINING RECIPE

  • VLM pretraining fine-tunes InternVL3 on embodied-related datasets with an autoregressive language-modeling loss.The recipe includes datasets focused on grounding, planning, and spatial intelligence.Source: §2.3 · p. 5
  • The action expert uses flow matching to predict future action chunks from a single-frame observation, with robot state and noisy actions as inputs.It shares self-attention with the language model and uses non-causal attention for the VLA stream.Source: §2.1 · p. 4, §2.3 · p. 5
  • The VLA fine-tuning stage trains an additional action expert on robot-specific datasets, using observations that include images, language, and robot state.The action expert predicts a horizon of future actions from a single-frame observation.Source: §2.3 · p. 5
  • At inference, actions are generated by integrating the learned vector field from τ = 0 to τ = 1, using H = 4 and δ = 0.1 for ten integration steps.The stated settings are intended to improve inference efficiency.Source: §2.3 · p. 6

3 EXPERIMENTS

Across embodied-reasoning benchmarks, Vlaser improves on its InternVL3 bases and scores highest overall against embodied-specific VLMs. For robot policy learning, in-domain data outperforms OOD-only initialization, and results also vary with model size and training hyperparameters.

3.1 PERFORMANCE ON EMBODIED REASONING CAPABILITY

  • The embodied-reasoning evaluation spans 12 benchmarks covering QA, planning, grounding, spatial intelligence, and closed-loop simulation.EmbodiedBench includes the ALFRED and Habitat simulation environments.Source: §3.1 · p. 6
  • 45.3 vs 15.2 is the average score for Vlaser-2B and InternVL3-2B, while 51.3 vs 22.3 is the average for Vlaser-8B and InternVL3-8B.Vlaser improves across embodied-reasoning capabilities, with notable gains in embodied grounding and simulation-based evaluation.Source: §3.1 · p. 7
  • Vlaser-2B performs better on simple point-grounding tasks, whereas Vlaser-8B is stronger on multi-step planning and closed-loop simulation.The paper links this pattern to selecting model size according to the target application.Source: §3.1 · p. 7
  • Vlaser achieves the highest overall score against embodied-specific VLMs, with a +10% overall margin and superior performance on most benchmarks.It remains highly competitive on the other benchmarks.Source: §3.1 · p. 7

3.2 PERFORMANCE ON DOWNSTREAM CLOSE-LOOP ROBOT TASKS

  • In-domain fine-tuning data improves closed-loop task success across the robot benchmarks, while Vlaser-OOD remains close to the InternVL3-2B baseline.The evaluations cover WidowX, Google Robot, and RoboTwin Aloha-AgileX tasks.Source: Table 2 · p. 7, Table 3 · p. 8, Table 4 · p. 8
  • Any of the three in-domain annotation types improves performance over the baseline, and combining all three yields further gains.The types are embodied QA, grounding, and spatial intelligence; the authors attribute improvements primarily to mitigating visual-observation domain shift.Source: §3.2 · p. 9

3.3 ABLATION STUDIES

  • Across the tested hyperparameter settings, Vlaser-OOD shows slight improvement over InternVL3-2B, while Vlaser-QA yields significant gains.The ablations vary predicted action length, executed action length, and flow-matching sampling steps.Source: §3.3 · p. 9

4 RELATED WORK

Prior work develops embodied reasoning models and advances VLA control, but the link between multimodal reasoning and low-level control remains underexplored. Vlaser combines benchmark evaluation with analysis of how VLM capabilities and training data relate to downstream VLA performance.

Vision-Language Model for Embodied Reasoning

  • Embodied reasoning work addresses grounding, spatial intelligence, and task planning, while data-driven methods curate reasoning datasets and use reinforcement fine-tuning with structured thought traces.Examples include Cosmos-Reason1, VeBrain, MolmoAct, EmbodiedOneVision, RoboBrain, Embodied R1, and Robix.Source: §4 · pp. 9–10
  • Vlaser aims to distinguish prior efforts by pairing competitive or superior embodied-reasoning benchmark performance with analysis of the relationship between pretrained VLMs and downstream VLAs.The authors present this analysis as a bridge between model capabilities and real-world deployment.Source: §4 · p. 10

Vision-Language-Action models

  • The relationship between high-level multimodal reasoning and low-level control performance remains largely unexplored, including which abilities and training data best enhance VLA control.The passage identifies spatial understanding, grounding, and planning as abilities whose effects remain unclear.Source: §4 · p. 10
  • VLA research incorporates diffusion, flow matching, and mixture-of-experts, and adopts larger VLM backbones to address more complex real-world manipulation tasks.Some studies also explore unified VLM–VLA training frameworks.Source: §4 · p. 10

5 LIMITATIONS

The study has no real-robot experiments; its experiments were conducted in simulation.

5 LIMITATIONS

  • The study has no real-robot experiments; all experiments were conducted in simulation.The authors plan future real-robot experiments to further evaluate and refine the methods.Source: §5 · p. 10

6 CONCLUSION AND DISCUSSION

Vlaser combines embodied reasoning with end-to-end robot control and reports strong results on reasoning benchmarks and robot-control tasks. The authors identify a viewpoint-related domain gap between embodied reasoning benchmarks and robots.

6 CONCLUSION AND DISCUSSION

  • Vlaser establishes state-of-the-art performance across embodied reasoning benchmarks and state-of-the-art Bridge and competitive Google Robot end-to-end control results.The model is powered by the Vlaser-6M dataset and its curated data pipeline.Source: §6 · p. 10
  • The authors attribute a domain gap between embodied reasoning benchmarks and robots to robots' different viewpoint from internet datasets and limited robot-perspective data.They argue for alignment techniques to bridge the representation gap between robot viewpoints and internet datasets.Source: §6 · p. 10
  • The work reports using publicly available datasets and no personally identifiable, sensitive, or harmful data, and states that no human subjects or vulnerable groups were involved.The authors considered potential societal impacts and foresee no harm from these contributions.Source: ETHICS STATEMENT · p. 11
  • The authors say they will release code, configuration files, and reproduction scripts, including for dataset construction, training, and evaluation.They plan to make these materials available on platforms such as GitHub and Hugging Face.Source: REPRODUCIBILITY STATEMENT · p. 11

A APPENDIX

The authors report limited use of ChatGPT for grammar correction and state that they reviewed the writing and retained responsibility for the final content.

THE USE OF LARGE LANGUAGE MODELS (LLMS)

A.1 TRAINING DETAILS

Vlaser’s VLM pretraining uses supervised fine-tuning of InternVL3-2B and InternVL3-8B, with all model parameters trainable. Downstream VLA fine-tuning also optimizes the VLM and Action Expert, with different base models compared.

VLM pretraining

  • VLM pretraining applies supervised fine-tuning to InternVL3-2B and InternVL3-8B, keeping the language model, projector, and visual encoder trainable.The pretraining hyperparameters are listed in Table 6.Source: §A.1 · p. 18, Table 6 · p. 18

VLA fine-tuning

  • Downstream VLA fine-tuning optimizes all parameters in both the VLM and Action Expert, and compares different base-model versions including InternVL3-2B.The VLA fine-tuning hyperparameters are listed in Table 7.Source: §A.1 · p. 18, Table 7 · p. 18

A.2 DATA GENERATION DETAILS

The data-generation pipeline combines embodied grounding, spatial reasoning, planning, and in-domain robot data, then filters generated QA pairs with an LLM judge and checks its ratings against human reassessment.

Embodied grounding data

  • The pipeline generates 300k grounding samples from SA-1B masks, using high-IoU selection, mask-derived boxes and points, and caption refinement.Masks below the IoU threshold of 0.9 are excluded or given reduced sampling weight; BLIP-2 generates coarse captions, which Qwen2.5-VL-7B filters and refines.Source: §A.2 · p. 19

Spatial reasoning data

  • A manually constructed set of 100k spatial-perception samples uses 3D point clouds and video to build scene graphs for questions about counts, distances, sizes, directions, and object relations.The scenes come from ScanNet, ScanNet++, and ARKitScenes, and question templates follow VSI-Bench.Source: §A.2 · p. 19

Planning data

  • Planning data retains successful Habitat rollouts with task instructions, action sequences, observations, and success signals for each action.An LLM agent based on gpt-4o performs rollouts for tasks initialized from LLaRP annotations.Source: §A.2 · p. 19

In-domain data

  • Two million in-domain multimodal samples cover general, grounding, and spatial-reasoning QA across WidowX, Google Robot, and Aloha-AgileX simulation data.Qwen2.5VL-7B generates the samples; the QA types address robot state and next actions, localization affordances, and scene geometry.Source: §A.2 · p. 19

Data filtering

  • Qwen2.5VL-32B scores generated QA pairs from 1 to 3 on relevance, informativeness, and clarity; samples scored 1, about 10% of the initial set, are removed.The supplied scoring rubrics assess general QA quality, spatial accuracy and detail, and grounding accuracy and coordinate validity.Source: §A.2 · pp. 19–21

Data filtering validation

  • Three human experts reassessed a random subset spanning all score levels, and their ratings aligned with the LLM judge in nearly 80% of cases.The authors report this as validation of the automated evaluation and filtering pipeline’s consistency and reliability.Source: §A.2 · p. 20

A.3 SIMULATION EVALUATION DETAILS

Simulation evaluation uses fixed-iteration checkpoints for WidowX and Google Robot tasks, with single-image observations and two-action chunks. The section also provides Embodied QA examples from ERQA.

Simulation evaluation

  • Evaluation uses checkpoints after 45,390 iterations for WidowX and 36,970 for Google Robot, with one image, action chunks of size 2, and 10 Euler flow-matching inference steps.The number of test samples for each task is listed in Table 8.Source: §A.3 · p. 21, Table 8 · p. 22

Embodied QA examples

  • The examples include an ERQA question about changes between two frames and another asking which action should come next in a video task.The first example asks whether a robot lifted a cup or poured nuts; the second concerns the next step in weighing dough.Source: §A.3 · p. 22

A.4 QUALITATIVE DEMONSTRATION

The qualitative demonstrations span embodied QA, grounding, spatial intelligence, planning, and SimplerEnv robot tasks. The SimplerEnv examples contrast baseline task failures from small positional errors with improved performance from Vlaser and data additions.

Embodied reasoning demonstrations

  • The paper presents qualitative samples for embodied QA, grounding, spatial intelligence, and simulation planning, and describes them as evidence of Vlaser’s capabilities across these tasks.Examples are identified in Tables 10–12 for grounding, spatial intelligence, and simulation planning.Source: §A.4 · pp. 22–23, Table 10 · p. 23, Table 11 · p. 27

SimplerEnv demonstrations

  • Under the same SimplerEnv test conditions, baselines often fail tasks after slight positional errors, while Vlaser shows improvement; adding in-domain QA, spatial, or grounding data also improves performance.Figures 6 and 7 show examples for WidowX and Google Robot tasks, respectively.Source: §A.4 · p. 23, Figure 6 · p. 27, Figure 7 · p. 29

Planning example

  • The Vlaser-8B planning example gives successive reasoning steps for putting a microwaved tomato slice on the counter, ending with an action to put down the tomato.The example includes locating a knife and tomato before describing later task steps.Source: §A.4 · p. 28

Related papers

Loading 2510.11027v2…