Source-linked AI summary
Green-VLA: Staged Vision-Language-Action Model for Generalist Robots
I. Apanasevich, M. Artemyev, R. Babakyan, P. Fedotova, D. Grankin, E. Kupryashin, A. Misailidi, D. Nerus, A. Nutalapati, G. Sidorov, I. Efremov, M. Gerasyov, D. Pikurov, Y. Senchenko, S. Davidenko, D. Kulikov, M. Sultankin, K. Askarbek, O. Shamanin, D. Statovoy, E. Zalyaev, I. Zorin, A. Letkin, E. Rusakov, A. Silchenko, V. Vorobyov, S. Sobolnikov, A. Postnikov
TL;DR
Green-VLA tackles heterogeneous, low-quality robotics data and the limits of behavior cloning for long-horizon, cross-embodiment control. It combines quality-filtered, temporally aligned demonstrations with a unified action interface and five-stage training curriculum, then applies reinforcement-learning alignment. The framework achieves state-of-the-art results on Simpler BRIDGE WidowX and competitive near-state-of-the-art performance on CALVIN ABC→D, while its deployment scope remains bounded by retargeting fidelity, dataset bias, and dexterous-skill coverage.
Problem
Heterogeneous robotics data, inconsistent quality, behavior-cloning saturation, and inference-latency constraints limit robust real-world VLA deployment.
Method
Green-VLA combines DataQA filtering, temporal alignment, a unified embodiment-aware action space, staged L0→L1→R0→R1→R2 training, and planner-guided deployment.
Results
R2 alignment yields state-of-the-art results on Simpler BRIDGE WidowX and competitive, near-state-of-the-art performance on CALVIN ABC→D.
Takeaways & Limitations
A single staged policy supports diverse real-world robots while RL alignment improves difficult long-horizon tasks requiring recovery and dexterity.
Takeaways & Limitations
Performance depends on retargeting fidelity, residual dataset bias, and adequate coverage of dexterous skills.
Abstract
from arXiv · showhide
We introduce Green-VLA, a staged Vision-Language-Action (VLA) framework for real-world deployment on the Green humanoid robot while maintaining generalization across diverse embodiments. Green-VLA follows a five stage curriculum: (L0) foundational VLMs, (L1) multimodal grounding, (R0) multi-embodiment pretraining, (R1) embodiment-specific adaptation, and (R2) reinforcement-learning (RL) policy alignment. We couple a scalable data-processing pipeline (3,000 hours of demonstrations) with temporal alignment and quality filtering, and use a unified, embodiment-aware action interface enabling a single policy to control humanoids, mobile manipulators, and fixed-base arms. At inference, the VLA controller is enhanced with episode-progress prediction, out-of-distribution detection, and joint-prediction-based guidance to improve safety and precise target selection. Experiments on Simpler BRIDGE WidowX and CALVIN ABC-D, as well as real-robot evaluations, show strong generalization and performance gains from RL alignment in success rate, robustness, and long-horizon efficiency.
1 Introduction
Green-VLA addresses heterogeneous, noisy robotics data and the limitations of behavior cloning through quality alignment, unified actions, staged training, and reinforcement-learning refinement. The resulting policy targets real-world deployment across diverse embodiments, including a 32-DoF humanoid.
- Motivation: Heterogeneous observations, action spaces, sampling rates, noisy trajectories, and behavior-cloning saturation hinder robust long-horizon, cross-embodiment control.Behavior cloning fails to align policies with long-horizon objectives and task-level rewards, producing brittle generalization.
- Deployment: A high-level task planner decomposes user goals into atomic subtasks and conditions the VLA loop with auxiliary episode-end, OOD, and precise-target guidance signals.The architecture combines multimodal tokens with a flow-matching action expert and planner-level decomposition.
- Approach: Green-VLA filters and temporally aligns 3,000 hours of demonstrations while unifying heterogeneous robot actions into a normalized action space.DataQA scores jitter, sharpness, diversity, and state variance; frequency interpolation/extrapolation and control-type prompting support action unification.
- Deployment: The same policy controls the Green humanoid’s 32-DoF upper body and conventional manipulators without architectural changes.The evaluation spans single-arm, dual-arm, and humanoid platforms requiring coordinated bimanual and dexterous control.
- Approach: The five-stage curriculum progresses from foundational multimodal grounding through robotics pretraining and embodiment adaptation to reinforcement-learning policy alignment.The recipe transfers common-sense grounding into unified robot data, embodiment specialization, and long-horizon reward alignment.
2 Why a Staged VLA Pipeline Matters
Green-VLA argues that VLA scaling requires complementary multimodal and robotics data organized through a staged curriculum. Each stage addresses a distinct bottleneck, from semantic grounding and affordance learning to embodiment adaptation and reward-based robustness.
- Motivation: Staged training combines web-scale multimodal data for semantic and physical common sense with robotics data for grounded control.The paper argues that parameter count or dataset size alone does not address VLA scaling needs.
- L1 and R0: L1 builds semantic grounding, while R0 learns broad affordance priors and cross-embodiment invariants from multi-embodiment robotics data.These stages support reasoning about physics, object affordances, task structure, goals, objects, and kinematics.
- R1: R1 converts general pretrained capacity into target-embodiment competence through adaptation without requiring new large-scale data.Embodiment-aware heads, optimization, and inference-efficiency techniques are identified as useful adaptation mechanisms.
- R2: R2 addresses behavior-cloning limitations in long-horizon, contact-rich manipulation by incorporating task rewards, failure recovery, and preference-like feedback.The paper states that reinforcement learning improves success rate and average chain length.
- Stage outcomes: The robot-specific stages use VQA and robotics data to support new-embodiment adaptation, spatial reasoning, dexterous manipulation, and failure recovery.The staged pipeline is summarized as enriching grounding, capturing affordances, adapting embodiments, and injecting reward-based alignment.
3 Green-VLA Data Framework
Green-VLA builds a high-quality, embodiment-agnostic robotics corpus by combining temporally aligned web and robot data with structured augmentation, filtering, diversity assessment, and weighted sampling. This framework supports scalable unified pretraining across humanoids and manipulators.
- Data composition: Green-VLA combines 24M web-scale multimodal samples with 184M robotics-domain samples spanning more than 3,000 hours across humanoids and manipulators.RGB streams and proprioception are temporally normalized, while language instructions annotate each episode.
- Data composition: The L1 corpus covers general VQA, pointing, bounding-box prediction, pixel-wise trajectory prediction, multi-view VQA, captioning, and spatial reasoning.Sampling weights are customized across tasks for consistency during pretraining.
- Data composition: The R0 mixture combines open-source and internally collected datasets, including humanoid, dual-arm, mobile, and single-arm robot data.Internally collected data include 48 hours from the Green Humanoid dataset and 11.2 hours from ALOHA any_pick.
- Data augmentation: Structured mirroring and reversible time reversal expand Green Humanoid recordings from 48 real-world hours to 167 effective training hours.Mirroring swaps left/right views and joints, while time reversal is restricted to physically reversible task templates.
- Quality assurance: DataQA filters missing, abnormal, inactive, blurry, erratic, and task-inconsistent episodes using motion, sharpness, diversity, variance, and gripper-pattern checks.The pipeline includes tremble scoring, Laplacian-based sharpness estimation, and task-specific open-closed-open validation for pick-and-place.
- Quality assurance: Dataset sampling probabilities are weighted using visual diversity, state-space diversity, sharpness, tremble, dataset size, and trajectory characteristics.AgiBot twofinger, Galaxea, and DROID receive high weights for dataset size or diversity and smoother, more varied trajectories.
4 Green-VLA Model
Green-VLA combines staged training, a unified action space, and inference-time guidance to control diverse embodiments. Its design spans data alignment, task planning, target selection, safety correction, and RL-based trajectory validation.
- Staged training: Green-VLA trains through R0 robotics pretraining, R1 target-embodiment fine-tuning, and R2 reinforcement-learning fine-tuning.R0 uses available robotics data, R1 uses a high-quality target-embodiment dataset, and R2 aligns behavior through reinforcement learning.
- Task planner: A high-level GigaVision planner converts voice or text goals into atomic subtasks that condition the low-level Green-VLA policy.The planner also checks predicted subtask completion and either advances or replans the current subtask.
- Unified action space: Green-VLA maps heterogeneous native action spaces into a 64-dimensional unified semantic layout with embodiment masks, avoiding losses on unused slots.The masked objective removes spurious padding gradients while preserving semantics across joint, Cartesian, gripper, base, and other action modalities.
- Unified action space: The unified interface preserves shared-skill identifiability and lets inference specify which action modalities the policy should produce.Native actions are mapped into unified slots and downstream controllers apply the embodiment-specific inverse map to active slots.
- Temporal control: Speed-conditioned trajectory warping lets one policy represent fine manipulation and faster coarse motion, with inference speed controlled by hyperparameter v.The same model can operate as a careful local controller or an efficient long-horizon executor without retraining or separate planners.
- Inference guidance and refinement: A joint prediction module converts language-described targets into workspace points and uses IK plus flow-matching guidance to steer end-effector trajectories.The inferred target point is geometrically consistent with the scene and language query; RL trajectory optimization is validated in the environment before adding data.
- Inference guidance and refinement: The system uses controlled exploration and training-distribution guidance to keep RL-generated actions close to demonstrated behavior.Out-of-distribution states are corrected using a density-gradient procedure, while optimized trajectories are validated because heuristic RL updates may worsen outcomes.
5 Experiment Metrics Across Phases
Green-VLA is evaluated across staged training phases, heterogeneous embodiments, instruction-following tasks, and long-horizon benchmarks. Results emphasize progressive gains from filtering, embodiment adaptation, guidance, and RL alignment.
- Evaluation setup: The latest Green-VLA configuration is a mid-scale ~5B-parameter model evaluated across staged training phases and heterogeneous embodiments.R0 training used over 105 optimization steps on 64 H100 GPUs.
- Evaluation setup: Green-VLA uses approximately 3,000 hours of unified demonstrations to compare quality alignment and unified actions under constrained data.The reported comparison includes substantially more than 10,000 hours for π0.
- Cross-phase evaluation: On WidowX, success rate consistently increases from unfiltered R0 pretraining through filtered R1 supervised fine-tuning to R2 RL fine-tuning.The staged comparison uses the same architecture and a concrete target embodiment.
- Guided target selection: JPM guidance is evaluated for e-commerce shelf picking across in-domain coarse, in-domain SKU, and out-of-domain unseen-packaging regimes.The task tests precise item selection amid near-identical products, distractors, and packaging variation.
- Humanoid evaluation: Humanoid evaluation covers instruction-conditioned manipulation, including pick, place, handover, fruit sorting, and full table cleaning, with average success reported across in-domain and out-of-domain settings.Episodes vary object positions, distractors, and background clutter; success requires exact task following.
- RL alignment: R2 RL alignment markedly improves CALVIN long-horizon consistency and raises WidowX success rate by an absolute 24%.On CALVIN, R2 improves error recovery, compositional task success, and average chain length; the benchmark does not use the unified action space.
6 Conclusion
Green-VLA combines data quality alignment, unified actions, staged training, RL fine-tuning, and inference guidance to build responsive policies across robots. It reports strong benchmark and real-robot performance, while remaining dependent on retargeting fidelity, dataset bias, and dexterous-skill coverage.
- Conclusion: Green-VLA combines demonstration filtering and temporal alignment, a unified embodiment-aware action space, staged training, and RL fine-tuning.Inference efficiency and guidance support low-latency instruction-following control for novel language-specified items.
- Conclusion: Green-VLA shows strong pretrain-stage performance on Simpler and CALVIN, successful bimanual real-robot application, and reliable humanoid behavior under out-of-distribution layouts.The model outperforms prior foundation policies at comparable stages and approaches fine-tuned baselines.
- Conclusion: R2 RL alignment achieves state-of-the-art results on Simpler BRIDGE WidowX and competitive, near-state-of-the-art performance on CALVIN ABC→D.These results are attributed to the R2 alignment phase in the reported conclusion.
- Limitations and future work: Performance remains dependent on retargeting fidelity, residual dataset bias, and adequate coverage of dexterous skills.The paper identifies multilingual instruction following, real-time reasoning-control coupling, online data, and safety-aware RL as future work.
7 Contributors and Acknowledgments
The paper acknowledges contributors across VLA, RL fine-tuning, data pipeline, control, data collection, benchmarks, and project supervision. Authors are listed alphabetically.
- Contributors: VLA contributors are listed separately from the other project roles.The listed VLA contributors are I. Apanasevich, M. Artemyev, R. Babakyan, P. Fedotova, D. Grankin, E. Kupryashin, A. Misailidi, D. Nerus, A. Nutalapati, and G. Sidorov.
- Contributors: Named roles include RL fine-tuning, data pipeline, control, data collection, benchmarks, and project supervision.The listed contributors include I. Efremov, S. Davidenko, K. Askarbek, A. Letkin, S. Sobolnikov, and A. Postnikov, among others.
- Acknowledgments: Authors are listed in alphabetical order.This ordering note applies to the contributor list.