PaperlayerView exact revision on arXiv

Source-linked AI summary

VLA-R1: Enhancing Reasoning in Vision-Language-Action Models

Angen Ye, Zeyu Zhang, Boyuan Wang, Xiaofeng Wang, Dapeng Zhang, Zheng Zhu

arXiv:2510.01623v1cs.CVcs.RO

TL;DR

Existing VLA models often lack explicit step-by-step reasoning and systematic reinforcement of reasoning quality during post-training. VLA-R1 combines chain-of-thought supervision with RLVR-based optimization, and evaluations report strong performance and generalization across benchmark, simulation, and real-robot settings.

  • Problem

    Existing VLA models often lack step-by-step reasoning and systematic post-training reinforcement targeted at reasoning quality and execution.

    Source: §I · pp. 1–2

  • Method

    VLA-R1 combines chain-of-thought supervision from VLA-CoT-13K with GRPO-augmented RLVR post-training using rewards for region alignment, trajectory consistency, and output formatting.

    Source: CONSOLIDATED. IF FOLLOWING THE… · p. 2

  • Results

    VLA-R1 reports state-of-the-art performance and superior generalization across in-domain, out-of-domain, simulation, and real-robot evaluations, including 62.5% physical-hardware success for affordance perception.

    Source: CONSOLIDATED. IF FOLLOWING THE… · p. 2, §VI · p. 9

  • Takeaways & Limitations

    The work represents a promising step toward bridging reasoning quality and action execution in embodied AI.

    Source: §VI · p. 9

    VLA-R1 has not yet been developed or validated on bimanual robot arms or quadruped robot dogs.

    Source: §V · p. 9

Abstract

from arXiv · show

Vision-Language-Action (VLA) models aim to unify perception, language understanding, and action generation, offering strong cross-task and cross-scene generalization with broad impact on embodied AI. However, current VLA models often lack explicit step-by-step reasoning, instead emitting final actions without considering affordance constraints or geometric relations. Their post-training pipelines also rarely reinforce reasoning quality, relying primarily on supervised fine-tuning with weak reward design. To address these challenges, we present VLA-R1, a reasoning-enhanced VLA that integrates Reinforcement Learning from Verifiable Rewards (RLVR) with Group Relative Policy Optimization (GRPO) to systematically optimize both reasoning and execution. Specifically, we design an RLVR-based post-training strategy with verifiable rewards for region alignment, trajectory consistency, and output formatting, thereby strengthening reasoning robustness and execution accuracy. Moreover, we develop VLA-CoT-13K, a high-quality dataset that provides chain-of-thought supervision explicitly aligned with affordance and trajectory annotations. Furthermore, extensive evaluations on in-domain, out-of-domain, simulation, and real-robot platforms demonstrate that VLA-R1 achieves superior generalization and real-world performance compared to prior VLA methods. We plan to release the model, code, and dataset following the publication of this work. Code: https://github.com/GigaAI-research/VLA-R1. Website: https://gigaai-research.github.io/VLA-R1.

I. INTRODUCTION

VLA-R1 addresses VLA models’ lack of explicit reasoning and limited reasoning-focused post-training by combining chain-of-thought supervision with reward-aligned optimization. Its approach includes verifiable rewards for affordances, trajectories, and output format, plus a dataset aligned with affordance and trajectory annotations.

  • Existing VLA models often emit actions without reasoning over affordances or geometry, while post-training typically lacks rewards that jointly target region alignment and trajectory consistency.These limitations can impair instruction disambiguation and performance on out-of-distribution data and in the real world.Source: §I · pp. 1–2
  • VLA-R1 combines chain-of-thought supervision and reward alignment to explain actions and address color similarity, repeated instances, and multiple receptacle choices.Source: §I · p. 2
  • VLA-R1 uses GRPO [32] with GIoU-based affordance, improved-Fréchet-distance trajectory, and output-format rewards to optimize regions, trajectories, and specifications.The GIoU reward provides gradients for non-overlapping predicted and ground-truth regions; the distance reward targets reasonable trajectory curvature and segment length.Source: §I · p. 2
  • VLA-CoT-13K aligns explicit chain-of-thought steps with affordance and trajectory annotations to support task-consistent reasoning during supervised fine-tuning.The dataset is generated by the VLA-CoT data engine.Source: §I · p. 2
  • 36.51 IoU on the in-domain affordance benchmark represents a 17.78% improvement over the baseline.Source: §I · p. 2
  • Evaluations span in-domain and out-of-domain benchmarks, simulation, and real-robot settings.Source: §I · p. 2

CONSOLIDATED. IF FOLLOWING THE OFFICIAL UMD RELEASE,

VLA-R1 combines RLVR with GRPO and designed verifiable rewards, alongside VLA-CoT-13K supervision aligned with affordance and trajectory labels. Evaluations span in-domain and out-of-domain datasets, simulation, and real-robot platforms.

  • Distance 91.74 (lower is better) reduces the baseline by 17.25%; VLA-R1 is SOTA out-of-domain and achieves 62.5% affordance-perception and 75% trajectory-execution success on physical hardware.The results are reported as evidence of effectiveness under controlled conditions and robustness and practicality across domains and real-world scenarios.Source: CONSOLIDATED. IF FOLLOWING THE… · p. 2
  • VLA-R1 uses RLVR with GRPO and rewards for region alignment, trajectory consistency, and output formatting to strengthen reasoning and execution robustness while reducing reliance on manual annotation.Source: CONSOLIDATED. IF FOLLOWING THE… · p. 2
  • The VLA-CoT data engine produces VLA-CoT-13K aligned with affordance and trajectory labels, incorporating verifiable rewards to address the lack of step-wise reasoning in existing VLA models.Source: CONSOLIDATED. IF FOLLOWING THE… · p. 2
  • VLA-R1 is evaluated on in-domain and out-of-domain datasets, in simulation, and on real-robot platforms to assess effectiveness and cross-domain generalization.Source: CONSOLIDATED. IF FOLLOWING THE… · p. 2

II. RELATED WORK … A. Overview

Related work traces the shift from state-based manipulation methods to vision-centric VLA models and emerging RLVR approaches, while the method overview describes VLA-R1’s reasoning-to-action architecture.

A. VLA Models

  • Vision-centric VLA work spans zero-shot trajectory planning, language-conditioned manipulation, interpretable chain-of-thought, and long-horizon reasoning, but most approaches rely on large annotated datasets.VoxPoser generates 3D value maps; RoboFlamingo [21], ManipLLM, OpenVLA [18], RoboMamba [24], Embodied-Reasoner [42], Cosmos-Reason1 [2], and RoboBrain [17] represent other directions.Source: A. VLA Models · p. 3
  • ManipLVM-R1 [35] reduces supervision needs by combining small amounts of labeled data with RLVR-based self-improvement for robust generalization.Earlier manipulation research often relied on state-based reinforcement learning, which struggled with high-dimensional visual inputs, before vision-centric approaches became dominant,, [5].Source: A. VLA Models · pp. 2–3

B. RLHF for VLMs

  • RLVR has been applied to visual reasoning in non-embodied domains, while ManipLVM-R1 [35] adapts it to robotic manipulation for reasoning and action execution.Examples include Vision-R1 [15] for math reasoning, LMM-R1 [30] for staged textual and multimodal reasoning, and VLM-R1 [33] for visual grounding.Source: B. RLHF for VLMs · p. 3
  • The CoT data engine parses multimodal tasks by type, performs scene understanding and localization, checks feasibility, and generates structured reasoning traces for training.Task types include affordance and trajectory.Source: Figure 2 · p. 3

A. Overview

  • VLA-R1 maps visual observations and language instructions to explicit reasoning and discrete action tokens, which are detokenized into executable continuous 7D robot actions.Its vision and language branches feed a multimodal decoder that also reasons over temporal history.Source: A. Overview · p. 3

B. Data Synthesis · C. Supervised Fine-Tuning · D. Reinforcement Learning

Sections B–D describe a pipeline that synthesizes VLA-CoT-13K for reasoning supervision, then refines reasoning and action generation through SFT and GRPO with verifiable rewards.

B. Data Synthesis

  • Qwen2.5-VL-72B generates 13K CoT annotations for affordance and trajectory tasks, providing step-by-step, task-consistent reasoning supervision.The annotations are intended to bridge perception and action while enriching semantic interpretability.Source: B. Data Synthesis · pp. 3–4

C. Supervised Fine-Tuning

  • SFT pairs grounded visual evidence with action targets and supervises both <think> reasoning and <output> actions to encourage decomposition, grounding, and reliable long-horizon decoding.The intermediate supervision is presented as improving sample efficiency and preparing the model for post-training with verifiable rewards.Source: C. Supervised Fine-Tuning · p. 4
  • The model starts from Qwen2.5-VL-3B, adds an action decoder to its multimodal decoder, and uses a control stack to convert outputs into joint-level robot commands.Its vision pathway uses window attention and 2D RoPE, supports native resolution and dynamic video frame rates, and compresses visual tokens with an MLP merger.Source: C. Supervised Fine-Tuning · p. 4, Figure 3 · p. 4

D. Reinforcement Learning

  • After SFT, GRPO samples groups of outputs, normalizes their rewards into relative advantages, and uses clipping and a KL penalty to constrain policy updates.Step-wise process rewards are normalized similarly, with token-wise advantages accumulated and shared across outputs.Source: D. Reinforcement Learning · pp. 4–5
  • The trajectory reward uses Angle-Length Augmented Fréchet distance, combining order-preserving positional alignment with tangent-angle and neighboring-segment length penalties.The distance accounts for temporal ordering and local geometry; its normalized form defines a reward where larger values indicate better alignment.Source: D. Reinforcement Learning · p. 5
  • For affordance grounding, a GIoU reward compares predicted and ground-truth boxes and penalizes misalignment even when boxes do not overlap.Accounting for the smallest enclosing box is intended to improve spatial robustness in cluttered environments with partial overlaps.Source: D. Reinforcement Learning · pp. 5–6
  • A binary format reward scores outputs as 1 when they contain the required <think> and <output> structure and 0 otherwise, encouraging interpretable traces and preventing degenerate outputs.The required structure places the reasoning segment before the action segment.Source: D. Reinforcement Learning · p. 6

IV. EXPERIMENT

The experiments assess VLA-R1 across four settings and use baseline comparisons and ablations to evaluate its effectiveness and generalization.

IV. EXPERIMENT

  • Evaluation spans in-domain datasets, out-of-domain datasets, simulation environments, and real-robot platforms.These four settings are used to assess effectiveness and generalization capacity.Source: §IV · p. 6
  • The study compares VLA-R1 with strong baselines and ablates each component to assess its impact.Source: §IV · p. 6

A. Dataset and Metrics … 3) Metric Setting:

The evaluation uses annotated ShareRobot subsets for in-domain training, separate benchmarks for out-of-domain generalization, and localization, trajectory, and task-success metrics. The datasets and metrics assess affordance perception and trajectory prediction across real-world and simulated tasks.

1) In domain datasets:

  • ShareRobot draws from 23 Open X-Embodiment datasets[29] and covers 12 embodiments, 102 manipulation scenarios, and hundreds of primitive actions.All baseline models and VLA-R1 are trained on ShareRobot[17].Source: 1) In domain datasets · p. 6
  • The experiments use ShareRobot’s affordance- and trajectory-annotated subsets, with CoT rationales synthesized for training.The corpus includes over one million planning question–answer pairs, 6,522 affordance-annotated images, and 6,870 trajectory-annotated images; it is human-audited for image quality, task execution, annotations, and trajectory traces.Source: 1) In domain datasets · p. 6

2) Out of domain datasets:

  • The affordance OOD test set contains 1,200 randomly sampled UMD Part Affordance examples across grasp, cut, pound, and scoop categories[27].UMD spans 105 gardening, kitchen, and workshop tools.Source: 2) Out of domain datasets · p. 6
  • For trajectory OOD evaluation, the study uses 500 VAIT validation samples from LLARVA’s pretraining corpus[28], manually rectifying trajectories with excessive deviation.VAIT originates from the diverse Open X-Embodiment collection.Source: 2) Out of domain datasets · p. 6

3) Metric Setting:

  • Affordance perception is measured by IoU, which evaluates overlap between predicted and ground-truth regions, with higher values indicating more accurate localization and alignment.Source: 3) Metric Setting · p. 6
  • Trajectory prediction is evaluated using DFD, HD, and RMSE on ordered 2D waypoints normalized to [0, 1000), following prior work[17], [35].The metrics capture global shape and temporal alignment, maximum pointwise deviation, and average pointwise error, respectively.Source: 3) Metric Setting · pp. 6–7
  • Real-world and simulated evaluations report SR, the ratio of successful executions to total trials, using task-specific success criteria.Affordance success requires correct object localization and grasping, or no box when no object is present; trajectory success requires reaching the goal and reliably transporting the object there.Source: 3) Metric Setting · p. 7

B. Experiment on Benchmark · C. Experiment on Simulation

VLA-R1-3B outperforms baselines on in-domain and out-of-domain benchmark metrics, while simulation trials show success on both affordance perception and trajectory execution across Piper and UR5 robots. The experiments also compare it with supervised fine-tuning and RL baselines under varied evaluation conditions.

B. Experiment on Benchmark

  • Open-source multimodal instruction-following baselines have IoU below 10 and uniformly high trajectory errors, while SFT baselines improve but typically remain at IoU 5–25.RoboBrain-7B and NORA-3B achieve higher IoU and lower trajectory errors than the open-source baselines.Source: B. Experiment on Benchmark · p. 7
  • Benchmark comparisons include few-shot open-source models, SFT baselines, and ManipLVM-R1-3B, an RL post-trained model.The open-source set includes Gemma-3-12B-it, Gemma-3-27B-it, Phi-4multimodal-Instruct, and Qwen2.5-VL-3B/32B-Instruct; SFT baselines include InternVL2-2B, LLaVA-1.6-7B, RoboBrain-7B, and NORA-3B.Source: B. Experiment on Benchmark · p. 7

C. Experiment on Simulation

  • VLA-R1-3B achieves affordance success rates of 6/10 on Piper and 5/10 on UR5 (average SR = 55%) in simulation.Trials used randomized tabletop clutter and randomized initialization across ten trials per embodiment.Source: Table II · p. 7, C. Experiment on Simulation · p. 7
  • Trajectory execution reaches 8/10 successes on Piper and 6/10 on UR5 (average SR = 70%) in simulation.The paper reports that trajectory policy performance is highly stable once a reliable grasp is established.Source: Table II · p. 7, C. Experiment on Simulation · p. 7
  • In simulation, NORA reaches 50% and 30% SR on Piper and UR5 affordance tasks, and 1/10 and 0/10 on trajectory tasks, respectively.VLA-R1 was evaluated with Piper and UR5 in a RoboTwin tabletop environment that varied object categories, colors, poses, positions, and table color.Source: C. Experiment on Simulation · p. 7

B. Experiment on Benchmark

  • In-domain IoU = 36.51, DFD = 106.2, HD = 97.9, and RMSE = 71.12; OOD IoU = 33.96, DFD = 114.3, HD = 98.43, and RMSE = 68.97, surpassing ManipLVM-R1-3B.Relative to ManipLVM-R1, in-domain IoU improves by 17.78% and overall trajectory error falls by 17.25%.Source: B. Experiment on Benchmark · p. 7, Table III · p. 8

D. Experiment on the Real World · E. Ablation Study

Real-world tests report higher success rates for VLA-R1 than NORA-3B and spatially consistent predictions under clutter; ablations show gains from CoT and further improvements when combined with RL.

D. Experiment on the Real World

  • Across four real-world scenarios, VLA-R1 averaged ~62.5% affordance-perception SR and 75% trajectory-prediction SR, versus NORA-3B’s ~35% and 47.5%, respectively.The reported results are for affordance perception and trajectory prediction, respectively.Source: D. Experiment on the Real World · p. 8, Table IV · p. 8
  • The real-world evaluation used four tabletop scenarios—bowl picking, fruit picking, kitchen, and mixed objects—with ten trials each, randomized placements and poses, and shuffled scenario order.Source: D. Experiment on the Real World · p. 8
  • Under heavy clutter and attribute similarity, predictions stayed near targets rather than diverging, preserving local spatial consistency and indicating tolerance and self-correction.The authors identify color similarity and positional variation as primary error sources.Source: D. Experiment on the Real World · p. 8

E. Ablation Study

  • The ablation compared w/o CoT & RL, CoT only, and CoT+RL configurations trained under identical hyperparameters.Source: E. Ablation Study · p. 8
  • CoT alone raised IoU from 23.74 to 28.37 and reduced average distance from 128.38 to 124.6 versus the configuration without CoT or RL.The authors interpret the larger IoU improvement as evidence that CoT aids attribute disambiguation in affordance-centric tasks.Source: E. Ablation Study · p. 8, Table V · p. 9
  • CoT+RL achieved substantial gains across all metrics, with CoT structuring task decomposition and RLVR refining execution policies; higher IoU and lower DFD/HD/RMSE/AVG are better.The authors describe CoT and RLVR as complementary for trajectory prediction and end-to-end performance.Source: E. Ablation Study · p. 9

V. LIMITATION AND FUTURE WORK · VI. CONCLUSION

VLA-R1 combines chain-of-thought supervision, VLA-CoT-13K, and verifiable-reward reinforcement learning to strengthen reasoning and execution. Its reported evaluations span multiple settings, while validation on bimanual arms and quadruped robots remains future work.

V. LIMITATION AND FUTURE WORK

  • VLA-R1 has not yet been developed or validated on bimanual robot arms or quadruped robot dogs.The limitation concerns whether the model extends beyond the robotic platforms evaluated in the work.Source: §V · p. 9
  • Extending VLA-R1 to bimanual arms and quadruped robots is identified as future work to broaden applicability and test generalization in more diverse real-world scenarios.Source: §V · p. 9

VI. CONCLUSION

  • VLA-R1 achieves state-of-the-art performance and superior generalization across in-domain, out-of-domain, simulation, and real-robot experiments.The conclusion presents these experiments as evidence of performance across the named settings.Source: §VI · p. 9
  • VLA-R1 integrates chain-of-thought supervision with reinforcement learning from verifiable rewards in a reasoning-enhanced Vision–Language–Action model.Source: §VI · p. 9
  • VLA-CoT-13K and RLVR-based post-training explicitly strengthen step-by-step reasoning and execution robustness.Source: §VI · p. 9
  • The work frames VLA-R1 as a promising step toward bridging reasoning quality and action execution in embodied AI.Source: §VI · p. 9

Side View

The side-view examples pair task-specific reasoning about objects, affordances, and movement with spatial outputs. They show both trajectory waypoints for moving objects and image regions for grasp targets.

Side View

  • For moving the yellow bowl to the white basket, the examples describe obstacle-aware motion and provide four trajectory waypoints for each of two different starting configurations.Both responses characterize the waypoints as gradual, controlled movements toward the basket.Source: Side View · pp. 11–12
  • For picking up the yellow bowl, the reasoning identifies the bowl as the target and places the predicted affordance region over it.The output specifies a normalized region with x, y, width, and height.Source: Side View · p. 11
  • Other grasp-region examples target the ketchup bottle’s body, the left door’s handle, and a face of the white cube.The responses explain these regions as actionable locations for gripping or manipulating each object.Source: Side View · pp. 12–13
  • For reaching toward the pan, the response proposes a smooth route past nearby objects and outputs four waypoints ending near the pan handle.It notes that the sponge and fork-like tool require care during the approach.Source: Side View · p. 12

Related papers

Loading 2510.01623v1…