PaperlayerView exact revision on arXiv

Source-linked AI summary

VLA-R1: Enhancing Reasoning in Vision-Language-Action Models

Angen Ye, Zeyu Zhang, Boyuan Wang, Xiaofeng Wang, Dapeng Zhang, Zheng Zhu

arXiv:2510.01623v1cs.CVcs.RO

TL;DR

VLA models often lack explicit step-by-step reasoning and systematic reward optimization for execution-relevant qualities. VLA-R1 combines aligned CoT supervision with GRPO-based RLVR, and evaluations report stronger performance across domains and real-robot settings, while validation remains limited to certain robot embodiments.

  • Problem

    Existing VLA models often emit actions without explicit reasoning over affordances and geometry, while post-training provides limited systematic reinforcement for reasoning quality and execution.

  • Method

    VLA-R1 combines VLA-CoT-13K chain-of-thought supervision with GRPO-based RLVR using rewards for region alignment, trajectory consistency, and output formatting.

  • Results

    VLA-R1 achieves stronger generalization across in-domain, out-of-domain, simulation, and real-robot evaluations, including 91.74 trajectory distance and 62.5% and 75% physical-hardware success rates.

  • Takeaways & Limitations

    The results support combining explicit reasoning supervision with verifiable reward optimization to improve embodied reasoning and execution across evaluated settings.

  • Takeaways & Limitations

    VLA-R1 has not yet been developed or validated on bimanual robot arms or quadruped robot dogs.

Abstract

from arXiv · show

Vision-Language-Action (VLA) models aim to unify perception, language understanding, and action generation, offering strong cross-task and cross-scene generalization with broad impact on embodied AI. However, current VLA models often lack explicit step-by-step reasoning, instead emitting final actions without considering affordance constraints or geometric relations. Their post-training pipelines also rarely reinforce reasoning quality, relying primarily on supervised fine-tuning with weak reward design. To address these challenges, we present VLA-R1, a reasoning-enhanced VLA that integrates Reinforcement Learning from Verifiable Rewards (RLVR) with Group Relative Policy Optimization (GRPO) to systematically optimize both reasoning and execution. Specifically, we design an RLVR-based post-training strategy with verifiable rewards for region alignment, trajectory consistency, and output formatting, thereby strengthening reasoning robustness and execution accuracy. Moreover, we develop VLA-CoT-13K, a high-quality dataset that provides chain-of-thought supervision explicitly aligned with affordance and trajectory annotations. Furthermore, extensive evaluations on in-domain, out-of-domain, simulation, and real-robot platforms demonstrate that VLA-R1 achieves superior generalization and real-world performance compared to prior VLA methods. We plan to release the model, code, and dataset following the publication of this work. Code: https://github.com/GigaAI-research/VLA-R1. Website: https://gigaai-research.github.io/VLA-R1.

I. INTRODUCTION

VLA-R1 addresses missing step-by-step reasoning and weak reasoning-oriented post-training in VLA models by combining CoT supervision with verifiable-reward optimization. It targets affordance, trajectory, and formatting quality to improve reasoning and execution.

  • VLA models can generalize across tasks and scenes, but often emit actions without explicit reasoning over affordances, geometry, or receptacle selection.
  • VLA-R1 combines data-level Chain-of-Thought supervision with optimization-level reward alignment to bridge reasoning and execution.
  • Its RLVR strategy uses GRPO with verifiable rewards for affordance-region alignment, trajectory consistency, and output formatting.
  • The VLA-CoT data engine generates structured reasoning traces aligned with affordance and trajectory annotations for supervision.
  • VLA-R1 is evaluated across in-domain, out-of-domain, simulation, and real-robot settings, with reported gains over baselines.

CONSOLIDATED. IF FOLLOWING THE OFFICIAL UMD RELEASE,

The paper presents VLA-R1 as an RLVR- and CoT-enhanced VLA model, with broad evaluations spanning datasets, simulation, and physical robots. Its contributions center on reward design, aligned reasoning data, and cross-domain evaluation.

  • Distance 91.74 on the in-domain trajectory benchmark reduces the baseline by 17.25%, while physical hardware reaches 62.5% affordance success and 75% trajectory success.
  • VLA-R1 introduces RLVR with GRPO and rewards for region alignment, trajectory consistency, and output formatting.
  • The VLA-CoT data engine produces VLA-CoT-13K with reasoning aligned to affordance and trajectory labels.
  • VLA-R1 is evaluated on in-domain and out-of-domain datasets, in simulation, and on real-robot platforms.

II. RELATED WORK

Related work spans VLA and vision-centric manipulation models, emerging RLVR methods for visual reasoning, and data-engineering approaches for structured CoT supervision. VLA-R1 builds on these directions by combining multimodal encoding, CoT data, and embodied RLVR.

  • Vision-centric manipulation methods use multimodal models for value maps, language-conditioned control, object understanding, affordance perception, and pose prediction.
  • Recent LVLM research shows RLVR can enhance visual reasoning, while its earlier applications largely remained outside embodied domains.
  • The CoT Data Engine parses multimodal tasks, performs scene understanding and localization, validates feasibility, and generates structured traces.
  • VLA-R1 encodes an image and instruction through a vision-language backbone before producing low-level control signals with an action decoder.

B. Data Synthesis

VLA-R1’s data and training pipeline combines CoT supervision with structured affordance and trajectory outputs, then refines reasoning and actions using verifiable reinforcement rewards.

  • Data Synthesis: 13K CoT annotations provide explicit step-by-step supervision aligned with grounded visual evidence and action targets for affordance and trajectory tasks.The dataset is generated with Qwen2.5-VL-72B and is used to bridge perception and action during supervised fine-tuning.
  • Training Pipeline: The two-stage pipeline first applies SFT with CoT supervision, then uses GRPO-based reinforcement learning to refine reasoning and actions.Inference converts model outputs into joint-level robot commands through a control stack.
  • Trajectory Reward: ALAF rewards trajectory predictions using position, tangent-angle, and segment-length alignment under order-preserving sequence couplings.The trajectory reward normalizes ALAF distance so larger values indicate better alignment.
  • Affordance Reward: GIoU rewards affordance boxes by accounting for the smallest enclosing box, penalizing misalignment even when predictions and targets do not overlap.This design targets spatial robustness in cluttered environments with partial overlaps.
  • Format Reward: A binary format reward requires a structured <think> reasoning segment followed by a structured <output> action segment.The format constraint is intended to encourage interpretable reasoning traces and prevent degenerate outputs during post-training.

IV. EXPERIMENT

VLA-R1 is evaluated across in-domain, out-of-domain, simulation, and real-robot settings, with component ablations to assess effectiveness and generalization.

  • Evaluations span in-domain datasets, out-of-domain datasets, simulation environments, and real-robot platforms, while ablations isolate each component’s impact.

1) In domain datasets:

The experiments use affordance- and trajectory-annotated subsets of ShareRobot, assess out-of-domain transfer with UMD and VAIT, and apply complementary spatial, trajectory, and task-success metrics.

  • ShareRobot provides the training corpus, spanning 12 embodiments, 102 manipulation scenarios, and over one million planning question–answer pairs.Training is restricted to affordance- and trajectory-annotated image subsets, which are also used to synthesize CoT rationales.
  • OOD evaluation samples 1,200 UMD examples across grasp, cut, pound, and scoop categories, while trajectory transfer is measured on VAIT.
  • Affordance perception uses IoU, while trajectory prediction combines DFD, HD, and RMSE to assess localization, temporal alignment, maximum deviation, and average pointwise error.
  • Real-world and simulated task performance is reported with Success Rate, defined as successful executions divided by total trials.

B. Experiment on Benchmark

Benchmark experiments compare VLA-R1 with contemporary multimodal and supervised fine-tuning baselines, alongside simulation evaluations on different robot platforms.

  • The benchmark suite includes large multimodal models and supervised fine-tuning baselines, while simulation evaluation compares performance across robotic platforms.
  • VLA-R1-3B achieves the best reported in-domain results, with IoU = 36.51, DFD = 106.2, HD = 97.9, and RMSE = 71.12.Relative to ManipLVM-R1, IoU improves by 17.78% and overall trajectory error decreases by 17.25%.
  • Under distribution shift, VLA-R1-3B reaches IoU = 33.96, DFD = 114.3, HD = 98.43, and RMSE = 68.97, surpassing the strongest listed baseline on trajectory prediction.

D. Experiment on the Real World

Real-world experiments examine cluttered, attribute-similar tabletop tasks, while ablations show improvements from CoT and further gains when RL is added.

  • D. Experiment on the Real World: The real-world table reports canonical tabletop scenarios, including bowl and fruit picking with color and repeated-instance disambiguation.
  • D. Experiment on the Real World: In real scenes, predictions remain concentrated near targets and preserve local spatial consistency when targets cannot be fully locked.
  • E. Ablation Study: CoT alone raises IoU from 23.74 to 28.37 and reduces the average distance metric from 128.38 to 124.6 relative to the no-CoT, no-RL configuration.
  • E. Ablation Study: Adding RL to CoT produces substantial gains across metrics, indicating complementary contributions from structured reasoning and reward-based execution refinement.

V. LIMITATION AND FUTURE WORK

VLA-R1 shows strong performance across benchmarks, simulation, and real-robot settings, but its validation does not yet cover other robotic embodiments such as bimanual arms or quadruped robot dogs.

  • The model has not yet been developed or validated on bimanual robot arms or quadruped robot dogs.Extending evaluation to these platforms is identified as future work for broader applicability and more diverse real-world testing.
  • VLA-R1 achieves state-of-the-art performance and superior generalization across in-domain, out-of-domain, simulation, and real-robot evaluations.

{“x”:354“y”:330“w”:41“h”:39}

The examples illustrate chain-of-thought reasoning for identifying objects, selecting grasp regions, and planning trajectories for manipulation tasks.

  • The yellow-bowl examples connect task instructions to object identification, grasping, and movement toward a specified basket.The responses describe the bowl, basket, robot-arm position, and movement required for the task.
  • The yellow-bowl pickup example identifies the bowl’s affordance region as the gripper interaction area and links it to secure lifting.It also considers workspace sufficiency and avoiding interference with surrounding space.
  • Other examples apply similar reasoning to ketchup grasping, reaching for a pan, opening a door, and picking a cube.These examples consider object properties, grasp locations, spatial relations, and avoidance of other objects.
Loading 2510.01623v1…