Source-linked AI summary
GigaBrain-0.5M*: a VLA That Learns From World Model-Based Reinforcement Learning
GigaBrain Team, Boyuan Wang, Bohan Li, Chaojun Ni, Guan Huang, Guosheng Zhao, Hao Li, Jie Li, Jindi Lv, Jingyu Liu, Lv Feng, Mingming Yu, Peng Li, Qiuping Deng, Tianze Liu, Xinyu Zhou, Xinze Chen, Xiaofeng Wang, Yang Wang, Yifan Li, Yifei Nie, Yilong Li, Yukun Zhou, Yun Ye, Zhichao Liu, Zheng Zhu
TL;DR
VLAs have limited long-horizon anticipation because they rely on current observations, motivating richer predictive conditioning. The paper introduces GigaBrain-0.5M* with RAMP, which uses world model predictions in iterative reinforcement learning; RAMP outperforms RECAP by approximately 30% points on Box Packing and Espresso Preparation and supports reliable long-horizon execution.
Problem
VLAs face limited temporally extended reasoning and weak future anticipation when directly acting from current observations.
Method
GigaBrain-0.5M* integrates RAMP, an iterative method that conditions policy learning on world model predictions of future states and value.
Results
RAMP achieves nearperfect success rates across Box Packing, Espresso Preparation, and Laundry Folding, surpassing RECAP by approximately 30% points on Box Packing and Espresso Preparation.
Takeaways & Limitations
GigaBrain-0.5M* exhibits robust and consistent long-horizon task execution in real-world deployment.
Abstract
from arXiv · showhide
Vision-language-action (VLA) models that directly predict multi-step action chunks from current observations face inherent limitations due to constrained scene understanding and weak future anticipation capabilities. In contrast, video world models pre-trained on web-scale video corpora exhibit robust spatiotemporal reasoning and accurate future prediction, making them a natural foundation for enhancing VLA learning. Therefore, we propose \textit{GigaBrain-0.5M*}, a VLA model trained via world model-based reinforcement learning. Built upon \textit{GigaBrain-0.5}, which is pre-trained on over 10,000 hours of robotic manipulation data, whose intermediate version currently ranks first on the international RoboChallenge benchmark. \textit{GigaBrain-0.5M*} further integrates world model-based reinforcement learning via \textit{RAMP} (Reinforcement leArning via world Model-conditioned Policy) to enable robust cross-task adaptation. Empirical results demonstrate that \textit{RAMP} achieves substantial performance gains over the RECAP baseline, yielding improvements of approximately 30\% on challenging tasks including \texttt{Laundry Folding}, \texttt{Box Packing}, and \texttt{Espresso Preparation}. Critically, \textit{GigaBrain-0.5M$^*$} exhibits reliable long-horizon execution, consistently accomplishing complex manipulation tasks without failure as validated by real-world deployment videos on our \href{https://gigabrain05m.github.io}{project page}.
1. Introduction
Mainstream VLAs remain limited in long-horizon planning because they rely on myopic observations, while world models offer predictive information for improving foresight. The paper introduces GigaBrain-0.5M* and RAMP, an iterative world model-based reinforcement-learning approach that improves challenging manipulation performance.
- Mainstream VLA architectures rely on myopic observations and reactive control, limiting long-horizon action planning.World models trained on large-scale video corpora forecast plausible future states, offering predictive priors for VLA foresight.
- GigaBrain-0.5M* extends GigaBrain-0.5M with RAMP, a world model-based reinforcement-learning method.GigaBrain-0.5M was pretrained on over 10K hours of real-world robotic interaction data.
- RAMP iteratively pretrains a world model, conditions policy actions on predicted values and future states, collects human-in-the-loop rollouts, and continually trains both models.The four-stage process is designed to support self-improvement through rollout data.
- RAMP uses future states predicted by a pretrained world model, whereas RECAP uses only sparse advantages and is theoretically established as a special case of RAMP.The authors characterize the predicted future states as providing greater information than RECAP’s binary advantage inputs.
- RAMP significantly outperforms AWR and RECAP, with improved multi-task generalization and sample efficiency during policy learning.The paper also reports state-of-the-art success rates across diverse manipulation tasks and robust execution of complex sequential tasks.
2. Related Works
Related work develops VLAs for cross-task and cross-embodiment generalization, but temporally extended reasoning remains limited. Reinforcement learning and advantage-conditioned policies address policy optimization challenges, motivating world model-based policy conditioning.
- Contemporary VLAs combine heterogeneous sensory inputs with executable motor commands, using autoregressive tokens or continuous flow-based action spaces.They scale model parameters and training corpora, including cross-embodiment datasets, to improve generalization.
- VLAs still face a fundamental limitation in temporally extended reasoning despite extensive cross-embodiment and proprietary training data.
- World models are increasingly used to generate data and bridge the simulation-to-reality gap in embodied AI systems.Applications include generating corner cases and constructing traffic situations for autonomous driving.
- Imitation-learning policies suffer compounding errors under distribution shift and remain dependent on demonstrations or continuous human supervision.DAgger-style methods mitigate distribution shift through online expert interventions but do not provide autonomous policy improvement.
- Scaling policy-gradient methods to large VLAs remains challenging because of training instability and sample inefficiency.Alternative approaches condition action generation on value or advantage signals, including RECAP’s use of on-robot data collection.
- RECAP motivates world model-based reinforcement learning by showing that advantage-conditioned policies can improve downstream VLA performance through on-robot data collection.The proposed direction uses a world model that jointly predicts value and future states as richer policy conditions.
3. GigaBrain-0.5M*
GigaBrain-0.5M* extends the GigaBrain-0.5 VLA with RAMP, a world model-based reinforcement-learning framework that iteratively refines policy behavior using predicted futures, values, and rollout feedback. RAMP combines world-model conditioning, human corrections, and continual training into a closed-loop policy-improvement process.
- 3. GigaBrain-0.5M*: GigaBrain-0.5M* integrates RAMP into the GigaBrain-0.5 foundation VLA to refine behavior through world model predictions and corrective feedback.RAMP is presented as world model-based reinforcement learning built upon the GigaBrain-0.5 policy.
- RAMP pipeline: RAMP uses four iterative stages: world model pre-training, policy pre-training with world model conditioning, human-in-the-loop rollout collection, and policy training with rollout data.The rollout stage combines autonomous execution with expert corrections, while continual training incorporates successful demonstrations and corrective signals.
- Policy objective: RAMP conditions action selection on world model latents and improvement signals through a weighted negative log-likelihood objective.The objective fits both the unconditional action distribution and the action distribution conditioned on the binary improvement indicator.
- Relation to RECAP: RECAP is theoretically characterized as a special case of RAMP that marginalizes over future latent states, whereas RAMP conditions explicitly on those predictions.The paper contrasts RECAP’s sparse binary advantage signal with RAMP’s richer future-state guidance.
- World model: The world model jointly predicts future visual states and value estimates, providing a unified representation for visual dynamics, task progress, and robot kinematics.Value and proprioceptive signals are spatially tiled and concatenated with visual latents before world-model processing.
4. Experiment
The experiments evaluate GigaBrain-0.5 on internal and public robotic benchmarks, then assess RAMP’s value prediction, policy conditioning, and reinforcement-learning performance. Results show strong foundation-model performance and substantial gains from joint world-model conditioning, especially on challenging manipulation tasks.
- Foundation Model Performance: GigaBrain-0.5 achieves the highest success rate across all evaluated internal tasks, including 100% on Juice Preparation versus 90% for GigaBrain-0.The model shows particularly notable gains on complex multi-step procedures.
- Foundation Model Performance: 10% and 20%: GigaBrain-0.5 improves success rates over π0.5 on Box Packing and Espresso Preparation, respectively.On Paper Towel Preparation, Laundry Folding, and Laundry Collection, success rates exceed 80%, with gains of 15%, 5%, and 10% over π0.5.
- Value Prediction Performance: The joint value-and-future-state predictor achieves the highest Kendall’s tau of 0.8018 and lowest MAE of 0.0621 while maintaining 0.25 s inference speed.Value-only prediction is faster at 0.11 s but has lower accuracy, while the VLM baseline has 0.32 s latency.
- World Model Conditioning for Policy Learning: World-model conditioning consistently outperforms the baseline in both single-task and multi-task training across four representative manipulation tasks.The comparison uses the same Stage-2 dataset and reports improvements throughout the training trajectory.
- RAMP Performance: RAMP achieves near-perfect success across Box Packing, Espresso Preparation, and Laundry Folding, surpassing RECAP by approximately 30% points on Box Packing and Espresso Preparation.GigaBrain-0.5M* also demonstrates reliable real-world task execution in supplementary deployment videos.
5. Conclusion and Future Work
GigaBrain-0.5M* builds on GigaBrain-0.5’s large-scale robotic pretraining by adding world model-conditioned reinforcement learning through RAMP. The resulting model targets cross-task generalization and reliable execution of complex long-horizon manipulation tasks.
- GigaBrain-0.5 was pretrained on over 10,000 hours of diverse robotic data and achieved a 51.67% average success rate across 30 RoboChallenge tasks.
- GigaBrain-0.5M* introduces future-state prediction to address conventional VLA models’ limited anticipation capabilities.
- RAMP integrates model-based reinforcement learning to improve robust cross-task generalization.
- GigaBrain-0.5M* reliably executes complex long-horizon tasks including sequential box packing and espresso preparation.
- Future work will improve rollout-data efficiency while reducing the computational overhead of synthetic trajectories.