Source-linked AI summary
GigaBrain-0.7: Scaling Embodied Foundation Models to Emergent Capabilities with a Three-System Architecture
GigaBrain Team, Angen Ye, Axiang Sun, Can Jin, Chenxi Cheng, Chong Shi, Dengke Shang, Dingqian Zhang, Guan Huang, Guangqiang Wang, Guangqing Ding, Guo Li, Hangcong Li, Hengyu Zhong, Hongtao Lu, Jianbo Qin, Jiming Mao, Jing Zhu, Jindi Lv, Jingzhi Cui, Junjie Xie, Junyi Bao, Kai Liu, Lei Yuan, Limin Long, Lv Feng, Mingming Yu, Peng Li, Pengfei Yi, Qi Li, Qianli Zhang, Qingfang Li, Qitang Hu, Rui Zhang, Shaoyan Sun, Shibo Sun, Shiying Duan, Tenghui Chen, Tianze Liu, Weijie Ke, Wenyao Xue, Xiaofeng Wang, Xiaoyu Tian, Xinyu Liu, Xinze Chen, Yang Wang, Yankai Wang, Yejun Zeng, Yifan Li, Yifei Nie, Yilong Li, Yilong Liu, Yongchao Feng, Yumeng Wang, Yun Ye, Zhichao Liu, Ziheng He, Zonghai Yang, Zheng Zhu
TL;DR
Scaling general-purpose robot control remains challenging given heterogeneous embodiments, action spaces, and execution conditions. GigaBrain-0.7 combines a three-system architecture with one-stage pretraining for multimodal understanding and multi-embodiment action generation. Larger-scale pretraining consistently reduces validation loss and improves challenging real-robot behavior reliability, with broader evaluations indicating signs of emergent embodied capabilities.
Problem
How to scale general-purpose robot control beyond larger datasets and policies remains challenging given heterogeneous embodiments, action spaces, and execution conditions.
Source: §1 · p. 2Method
GigaBrain-0.7 combines a three-system architecture for understanding, prediction, and action with one-stage, multi-embodiment pretraining jointly optimizing multimodal understanding and action generation.
Source: §7 · p. 44Results
Larger-scale pretraining consistently reduces validation loss and improves challenging real-robot behavior reliability, while scaling studies, ablations, and out-of-distribution evaluations indicate signs of emergent embodied capabilities.
Source: §1 · p. 3Takeaways & Limitations
Executable behavior across multiple out-of-distribution settings suggests that interaction patterns learned from heterogeneous experience can be reused and recombined as task conditions change.
Source: §6.6.2 · p. 33Foundation transfer remains limited when object identity, scene configuration, and task composition shift simultaneously, with out-of-distribution degradation on both AgileX PiPER and Maker H01.
Source: §6.6.1 · p. 31
Abstract
from arXiv · showhide
Vision-language-action (VLA) models have become a dominant paradigm for generalist embodied agents, demonstrating strong complex and long-horizon task completion in structured settings. Yet it remains an open question whether current VLA systems can benefit from more effective architectural design, scale to substantially larger and more heterogeneous data regimes, and achieve broader generalization across tasks and embodiments. To this end, we present GigaBrain-0.7, an embodied foundation model with substantially improved generalization across diverse robot embodiments. Specifically, GigaBrain-0.7 unifies understanding, prediction, and action through a three-system architecture, scales pretraining to over 37,000 hours of heterogeneous embodied data, and introduces one-stage alignment training that jointly optimizes vision-language understanding and multi-embodiment action generation. Compared with the preceding GigaBrain-0 series and prior state-of-the-art models including $π_{0.5}$, GigaBrain-0.7 achieves substantial improvements in foundation zero-shot capabilities, language-conditioned instruction following, and post-training task success rates. In particular, on our in-house Maker H01 platform and mainstream robot embodiments, GigaBrain-0.7 demonstrates strong task adaptability and completion ability across both home and industrial scenarios. All training code and pretrained model weights will be released.
1. Introduction
GigaBrain-0.7 coordinates understanding, prediction, and action through a three-system architecture and one-stage multi-embodiment pretraining. It reports improvements over the preceding GigaBrain-0 series in zero-shot capabilities, instruction following, and post-training task success.
1. Introduction
- Three specialized systems coordinate continuous action generation, temporal understanding and hierarchical planning, and world-model prediction and evaluation through semantic, visual, and value-based interfaces.System 1 uses flow matching [48] and an embodiment-aware action expert accommodating different degrees of freedom and control modes. System 2 tracks task progress and decomposes instructions into subtasks, while System 3 supplies predicted visual subgoals and state values to condition action.Source: §1 · p. 2
- One-stage pretraining jointly optimizes the VLM backbone, continuous-action expert, and multimodal understanding objectives, avoiding fragmented multi-stage optimization.Soft Knowledge Insulation, building on Knowledge Insulation [20], attenuates rather than blocks action gradients entering the VLM backbone, allowing controlled adaptation while preserving general vision-language capabilities.Source: §1 · pp. 2–3
- Over 37,000 hours of embodied trajectories across 16 robot morphologies and approximately 270 million vision-language samples support GigaBrain-0.7 pretraining.Trajectory processing includes LeRobot v3.0 [7] conversion, cross-embodiment state-action normalization, LLM-assisted instruction rewriting and subtask annotation, and multi-stage quality control.Source: §1 · pp. 2–3
- GigaBrain-0.7 improves foundation zero-shot capabilities, language-conditioned instruction following, and post-training task success rates over the preceding GigaBrain-0 series.Task-specific post-training evaluations compare against prior GigaBrain models [27] [28] and π0.5 [62].Source: §1 · p. 3
- Larger-scale pretraining consistently reduces validation loss and improves challenging real-robot behavior reliability, with evaluations suggesting emergent embodied capabilities.Scaling studies examine robot-data volume, source composition, VLM capacity, and VLA coupling architecture. The emergence interpretation also draws on System 3 ablations and out-of-distribution evaluations.Source: §1 · p. 3
- Evaluations cover multimodal understanding, tabletop, mobile, and household simulation, and AgileX PiPER/PiPER-X and Maker H01 execution, including multi-task, out-of-distribution, and long-horizon tasks.Controlled ablations isolate and combine future-state predictions and value signals; human-in-the-loop rollouts and corrective feedback assess experience-driven improvement. Training code and pretrained weights are planned for public release.Source: §1 · p. 3
2. Related Work
The section distinguishes autoregressive, serial, and parallel VLA architectures, contrasts staged and joint training, and reviews world models’ roles in robot learning. GigaBrain-0.7 follows parallel action generation and joint pretraining while adding a separately pretrained system for future-state prediction and task-progress estimation.
2.1. Vision-Language-Action Architectures
- VLA architectures fall into autoregressive VLM-as-Actor, serial VLM–action-expert, and parallel Mixture-of-Transformers families, distinguished by action representation and coupling to the pretrained VLM.Autoregressive models such as RT-2 [6] and OpenVLA [39] predict discrete action tokens, but sequential decoding becomes costly as control frequency, horizon, and dimensionality grow. FAST [60] uses frequency-domain compression, while Galaxea G0.5 learns cross-embodiment action tokenization.Source: §2.1 · p. 3
- Serial architectures use a pretrained VLM as a context encoder feeding a separate continuous-action decoder, making the exposed feature interface central to effectiveness.Examples include GR00T N1 [56], Gemini Robotics [25], and Qwen-RobotManip [87]. Qwen-RobotManip [87] uses a flow-matching DiT whose blocks cross-attend to visual and language representations from the final VLM layer.Source: §2.1 · p. 4
- Parallel Mixture-of-Transformers architectures couple dedicated continuous Action Experts deeply with vision-language backbones, as exemplified by 𝜋0 [5] and GigaBrain-0 [27].GigaBrain-0.7 follows this direction while separating interfaces for understanding and planning from future-state prediction and evaluation. HyVLA-0.5 [89] and 𝜋0.7 [63] also incorporate visual history.Source: §2.1 · p. 4
2.2. VLA Training Paradigms
- Staged training separates representation learning from continuous-action optimization, while task-specific post-training specializes pretrained capabilities to target embodiments, tasks, and deployment conditions.𝜋0.5 [62] introduces its flow-matching Action Expert during post-training. Xiaomi-Robotics-0 [9] first jointly trains its VLM on vision-language and robot-trajectory supervision, then freezes it while training the action DiT.Source: §2.2 · pp. 4–5
- GigaBrain-0.7 jointly optimizes vision-language supervision, hierarchical task prediction, discrete action representations, and continuous action generation during heterogeneous multi-embodiment pretraining.This follows joint-training approaches including GigaBrain-0 [27] and Wall-OSS-0.5 [86], which combine multimodal and action-generation objectives.Source: §2.2 · p. 5
- Reinforcement-learning approaches extend static imitation through policy-generated experience, advantage conditioning, world-model guidance, and corrective feedback.VLA-RL [52] and SimpleVLA-RL [46] directly update pretrained VLAs using environment feedback, while RECAP combines demonstrations, on-policy experience, and expert corrections through advantage conditioning. RAMP [28] uses predicted future states and values with human-in-the-loop experience; FlowPRO [89] learns from paired failure and corrective trajectories without critics or rewards.Source: §2.2 · p. 5
2.3. World Model-Powered VLA
- GigaBrain-0.7 separately pretrains System 3 for robot-centric future-state prediction and task-progress estimation, then uses its outputs to condition task-specific VLA post-training.Predicted subgoal images and value-derived progress signals provide additional conditions. At inference, positive-progress conditioning guides continuous action generation without explicit candidate-action scoring or selection.Source: §2.3 · p. 6
- World models support robot learning through synthetic data, unified world-action modeling, predictive policy conditioning, value estimation, and iterative policy improvement.DreamGen [38] generates synthetic interaction trajectories, Unified World Models [95] couples video and action diffusion, and Feat2Go [69] derives task-progress targets for value-based reward shaping. World-VLA-Loop [50] alternates improving a world model with policy failure rollouts and improving the policy through reinforcement learning in the updated learned environment.Source: §2.3 · p. 6
3. Data Curation
The training corpus combines heterogeneous embodied trajectories with vision-language supervision to support robot control while preserving visual-semantic capabilities. Source-specific processing standardizes trajectories and VLM annotations, while simulation and world-model generation complement physically collected data.
3.1. Dataset Composition
- 37,256.98 hours of embodied trajectories complement 271,976,674 VLM image–text/question-answering samples, combining robot-control supervision with visual-semantic, spatial, and task understanding.Trajectories span real robots, UMI demonstrations, egocentric human demonstrations, simulation, and world-model-generated data. The VLM corpus includes 15,234,327 self-collected samples and helps mitigate vision-language degradation from action-only training.Source: §3.1 · pp. 7, 10, §3 · p. 7
- 20,535.65 real-robot hours span 16 robot types, complemented by 8,251.83 UMI hours and 2,862.36 EGO hours.The real-robot corpus contains 1,810,101 episodes and 2,077,837,071 frames across heterogeneous hardware. UMI supplies near-manipulation viewpoints and hand–eye coordination supervision, while EGO adds broader manipulation scenes and first-person human-interaction priors.Source: §3.1 · pp. 7, 9
3.2. Embodied Trajectory Data Processing Pipeline
- UMI and EGO processing aligns human demonstrations with robot-compatible training representations while retaining near-manipulation viewpoints and first-person interaction information.UMI processing synchronizes cameras, poses, gripper states, and timestamps, filters poor trajectories, and retargets to Maker H01 for inverse-kinematics, simulation, and sampled robot-replay validation. EGO processing filters unsuitable interaction videos, normalizes temporal and coordinate fields, and supplements missing hand–object information to pair observations and language with future actions or hand trajectories.Source: §3.1 · pp. 7, 9, §3.2 · pp. 11–13
3.3. Simulation and Generated Data
- 5,607.14 hours of simulation and generated data broaden long-tail coverage through controllable physics environments and context-preserving visual diversification of real interactions.Simulation contributes 1453.92 hours, while world-model-generated data contribute 4,153.22 hours. Prompt-conditioned generation varies real interaction scenes; configurable simulation assets provide repeatable experience in configurations difficult or unsafe to collect physically.Source: §3.1 · p. 9, §3.3 · p. 13
3.2. Embodied Trajectory Data Processing Pipeline
- Trajectory preprocessing standardizes storage, robot representations, camera definitions, and instructions to enable joint training across heterogeneous embodiments.LeRobot datasets use v3.0 storage with filtering metadata; unified representations retain joint-space and end-effector fields, mask missing components, and preserve camera traceability. GLM-5.1 rewrites instructions with human spot checks, while multimodal temporal segmentation and human correction produce subtask annotations.Source: §3.2 · pp. 10–11
- Multi-stage trajectory cleaning addresses numerical outliers, prolonged inactivity, kinematic inconsistencies, and samples with anomalous training losses.Per-robot, per-action-dimension q01/q99 filtering controls outliers; joint-range-normalized state changes identify inactivity, and URDF-based end-effector alignment follows Qwen-RobotManip [87]. Persistent abnormal losses trigger inspection for temporal misalignment, action discontinuities, incorrect instructions, or video corruption before confirmed anomalies are filtered.Source: §3.2 · p. 11
3.4. VLM Data Processing and Annotation
- Qwen3.6-27B converts cleaned robot, UMI, and EGO observations into structured states supporting self-collected embodied VLM supervision.Following [9], annotation extracts objects, interactable parts, spatial relations, viewpoints, task stages, and action intent from trajectory-derived annotation units. Public and self-collected sources enter a common multimodal dialogue format with image and structural validation; generated annotations undergo consistency filtering, human spot checks, and provenance tracking.Source: §3.4 · pp. 13–14
- Embodied VLM annotations target affordance prediction, spatial understanding, and high-level task planning through localization, feasibility, progress, and next-action supervision.Affordance labels identify usable object parts and placement regions; spatial labels prioritize available geometry, otherwise using visual reasoning with rule-based consistency checks. Temporal annotations capture task stages, subtasks, action pre/post relationships, and completion states to supervise next actions, consequences, progress, and goal consistency.Source: §3.4 · p. 14
4. Model Architecture
GigaBrain-0.7 couples three systems for understanding and planning, prediction and evaluation, and action and control. Its action system integrates temporal context and embodiment-specific interfaces, while experience reinforcement connects the same policy to offline and online learning.
4. Model Architecture
- GigaBrain-0.7 separates understanding and planning, prediction and evaluation, and action and control into three interacting systems.System 2 uses PaliGemma2 (3B) [71]; System 1 combines a PaliGemma2 (3B) backbone with a dedicated Action Expert (0.5B). System 3 is an independent GigaWorld-1 [75] based world value model (5B) providing predictive signals to System 1.Source: §4 · p. 15
4.1. World Simulation: Understanding, Prediction, and Evaluation
- System 2 supplies scene reasoning and immediate subtasks, while System 3 conditions action generation on predicted visual outcomes and progress evaluations.System 2 processes the observation and instruction, describes relevant objects and spatial arrangements, and passes subtask context to both other systems. Given observations, proprioception, and subtask context, System 3 predicts a video aligned with System 1’s subtask horizon, supplies its final frame as a subgoal image, and converts a scalar progress value into a binary prompt condition indicating whether predicted progress is increasing.Source: §4.1 · p. 16
4.2. Action Alignment: Multi-Embodiment Action Generation
- System 1 fuses observations, visual memory, task reasoning, robot state, Robot ID, and predictive signals into action chunks for closed-loop control.Its Mixture-of-Transformers design couples vision-language and Action Expert streams through joint attention while retaining separate feed-forward parameters. The vision-language stream attends causally only to its own tokens, while the Action Expert attends bidirectionally across both streams without feeding action-specific representations into VLM attention.Source: §4.2 · pp. 16–17
- Temporal-Spatial Blocks incorporate short-term visual memory while keeping the VLA’s visual token count comparable to single-frame processing.Blocks in the visual encoder first aggregate recent frames causally, then model spatial interactions within the current observation. Historical information is progressively fused into current-frame tokens, and past-frame tokens are discarded before entering the VLA backbone.Source: §4.2 · p. 17
- System 1 uses discrete next-token action prediction during training and continuous conditional flow matching for executable deployment actions.The discrete pathway generates action tokens autoregressively through the vision-language head, bridging language understanding and motor control. The continuous pathway uses the Action Expert to predict trajectories from noise-corrupted action representations; both pathways share visual-language context through MoT joint attention.Source: §4.2 · p. 17
- Soft Knowledge Insulation allows controlled adaptation between vision-language and continuous-action pathways while preserving general-purpose vision-language capabilities.The interface allows the backbone to develop embodied-aware visual representations rather than completely isolating semantic representations from action learning [5].Source: §4.2 · pp. 17–18
- Embodiment-specific state and action projections let heterogeneous robots share the Action Expert while retaining their native control interfaces.A per-embodiment validity mask distinguishes absent state dimensions from zero-valued states before proprioception and Robot ID enter a dedicated projection module. Core Action Expert parameters are shared, input/output projections are embodiment-specific, and rotations use the continuous 6D format [94].Source: §4.2 · p. 18
4.3. Experience Reinforcement
- Experience reinforcement connects the same System 1 policy to offline intervention learning and online actor–critic improvement without introducing a separate controller.Human-in-the-loop intervention data support offline learning of an error-aware progress discriminator and a policy with some error-recovery capability. Online interaction combines subtask success labels with that discriminator to construct dense rewards for a lightweight actor–critic loop, retaining the policy across pretraining, post-training, and deployment.Source: §4.3 · p. 18
5. Model Training
GigaBrain-0.7 uses four training stages: joint VLA pre-training, world-model pre-training, task-specific VLA post-training, and experience-driven reinforcement learning. The world model remains frozen during post-training, while subsequent policy refinement progresses from offline experience to online learning with human corrections.
5.3. World-Model-Powered VLA Post-Training
- Task-specific post-training optimizes Systems 1 and 2 on demonstrations while frozen System 3 conditions System 1 with predicted subgoal images and progress signals.Flow-matching training uses encoded subgoal images and a binary progress token distinguishing improving from nonimproving or regressing segments; inference selects positive progress. Subgoal-image and value-condition dropout probabilities are 0.50 and 0.15, respectively, exposing the policy to neither signal, either signal alone, or both.Source: §5 · p. 19, §5.3 · p. 21
5.1. VLA Pre-Training
- Systems 1 and 2 jointly learn autoregressive language, subtask, and discrete-action prediction alongside flow-matching continuous-action generation throughout heterogeneous-data pre-training.The corpus combines real-robot, UMI, EGO, simulation, world-model-generated trajectories, and vision-language data to jointly optimize semantic understanding, hierarchical task prediction, and continuous action generation.Source: §5.1 · p. 19
- Soft Knowledge Insulation [20] attenuates flow-matching gradients entering the vision-language backbone while preserving full autoregressive supervision and control-oriented representation adaptation.Flow-matching gradients propagate fully within the Action Expert but are scaled by a coefficient before entering the backbone.Source: §5.1 · p. 19
- Hierarchical instruction supervision couples task reasoning with action generation, while randomly dropping historical observations trains robustness to varying temporal context.Training alternates between supplying task and subtask instructions and requiring System 2 to infer the subtask before System 1 acts; causal masks prevent target leakage. History dropout includes single-frame inputs, reducing dependence on a fixed history length.Source: §5.1 · pp. 19–20
- Multi-embodiment optimization aligns heterogeneous states and actions semantically, sharing Action Expert Transformer parameters while retaining embodiment-specific interfaces and validity information.Embodiment-specific input/output projections accommodate differing action dimensions and control conventions. Validity masks distinguish unavailable dimensions from valid zeros and exclude invalid action dimensions from flow-matching supervision and noise injection.Source: §5.1 · p. 20
5.2. World Model Pre-Training
- System 3 extends GigaWorld-1 [75] through robot-centric video pre-training followed by task-progress value learning, supplying future observations and progress estimates.Real-robot and simulation videos adapt manipulation dynamics; predicted videos match System 1’s action-chunk horizon, with their final frames serving as subgoal images. The Mixture-of-Transformers World Value Model learns progress from trajectory-level completion annotations using current and historical observations, then remains frozen.Source: §5.2 · p. 20
5.4. Experience-Driven Reinforcement Learning
- Offline advantage-weighted refinement strengthens online initialization and reduces costly exploration by emphasizing successful, higher-progress deployment segments.An Advantage-Weighted Regression (AWR) [59] style objective refines the policy using collected successful and failed rollouts without additional interaction during optimization. Rewards come from a progress discriminator and predefined rules, while failed or lower-progress segments receive less weight.Source: §5.4 · pp. 21–22
- Online actor–critic learning incorporates human corrections at failure states and repeatedly updates System 1 using deployment rollouts.System 1 remains the deployment actor, a separate critic estimates value, and rewards combine chunk-level progress differences with terminal sparse signals. The rollout–correction–update cycle can continue with new data; the full intervention protocol and online RL formulation are deferred to a future report.Source: §5.4 · p. 22
6. Experiments … 6.3. Data Scaling
The experiments select a PaliGemma2-based dual-stream System 1 and show that increasing pretraining data scale and diversity improves validation loss and real-robot performance.
6. Experiments
- The experiments examine architecture, data scaling, temporal context, System 3 conditioning, out-of-box generalization, task-specific post-training, and experience-driven reinforcement learning.System 3’s future-state and value-derived conditioning signals are assessed individually and jointly.Source: §6 · p. 22
6.1. Experimental Setup
- Real-world evaluations span AgileX PiPER/PiPER-X and Maker H01, comparing GigaBrain-0.7 against π0.5 [62], GigaBrain-0.1, Xiaomi-Robotics-1 [79], and Galaxea G0.5.Success rate measures full-task completion over typically 10–20 trials per configuration; language-following trials require the correct target, and ID/OOD evaluations distinguish represented configurations from unseen objects, layouts, or instructions. Policy baselines share task definitions and protocols, while post-training uses task-specific demonstrations from the target platform.Source: §6.1 · p. 23
6.2. Backbone and Architecture Comparison
- PaliGemma2 [71] is selected because it alone achieves nonzero shirt-folding success while remaining competitive on fruit manipulation.Larger backbones do not uniformly improve performance, although Gemma 4 [74] performs strongly on structured tasks. The comparison with Qwen3.5 [65] and Gemma 4 [74] varies model size and image resolution, so it is not a controlled model-scaling study.Source: §6.2 · p. 23, Table 4 · p. 24
- Dual Stream achieves the strongest performance across all evaluated tasks, including nonzero shirt-folding success, and becomes the default with PaliGemma2 despite higher inference cost.Dual Stream interleaves VLM–expert attention at every layer, while cross-attention alternatives use corresponding-layer or final-layer VLM states. Final-layer coupling has the lowest inference latency; multi-layer cross-attention adds computational cost without improving task success.Source: §6.2 · pp. 23–24, Table 5 · p. 24
6.3. Data Scaling
- Combining robot, UMI, and EGO data yields the strongest evaluated performance, improving over robot-only pretraining with gains that persist after task-specific post-training.Starting from 3,000 hours of robot data, experiments add equal amounts of UMI or EGO demonstrations and evaluate their combination. Robot trajectories supply embodiment-specific control supervision, UMI supplies near-hand manipulation experience, and EGO broadens first-person human interaction diversity.Source: Figure 8 · p. 25, §6.3 · pp. 25–26
- Additional robot experience improves fruit manipulation and chaotic clothes folding, with clothes folding showing a much sharper dependence on data scale.The authors interpret increasingly reliable manipulation as capability emergence and suggest that long-horizon deformable-object skills demand broader coverage of intermediate states.Source: Figure 8 · p. 25, §6.3 · p. 25
- Larger pretraining datasets consistently achieve lower validation loss under a fixed model configuration, with differences between data scales persisting throughout optimization.Improvement continues throughout the evaluated data range.Source: §6.3 · p. 24, Figure 8 · p. 25
6.4. Temporal Context Analysis · 6.5. System 3 Prediction and Evaluation Analysis · 6.6. Foundation Model Evaluation
GigaBrain-0.7 performs language-conditioned and longer-horizon manipulation without task-specific adaptation, although distribution shifts remain challenging. Temporal context helps resolve recurring-state ambiguity, while System 3’s future-image and value guidance improves task success, scores, and execution efficiency.
6.4. Temporal Context Analysis
- Temporal visual context helps a policy escape repeated-action cycles by distinguishing visually similar states occurring at different stages of task execution.In the qualitative matched-policy comparison, the context-enabled policy progresses to the next task stage, whereas the single-frame policy continues repeating similar actions.Source: §6.4 · pp. 26–27
6.5.1. Prediction and Evaluation
- System 3 jointly predicts future observations and state values, using imagined outcomes to estimate candidate-trajectory advantages that decrease with failure and increase with recovery.Dense world-model evaluation replaces direct reliance on human annotations or VLM-based reasoning for action-quality assessment. Compared with GigaBrain-0.5M*, prediction extends beyond the action-chunk horizon, improving complex-task success and robustness.Source: §6.5.1 · p. 27, Figure 10 · p. 27
6.5.2. Ablation in System 3
- 80% gift-wrapping success is achieved by +SubImage+Value on AgileX PiPER-X, while Base fails to complete the task.+SubImage and +Value achieve 20% and 60% success; task scores are 61.1% (Base), 71.4% (+SubImage), 93.3% (+Value), and 96.7% (+SubImage+Value). Among successful episodes, completion times are 65 s (+SubImage), 60 s (+Value), and 55 s (+SubImage+Value), illustrating gains in execution efficiency as well as success.Source: Figure 11 · p. 28, §6.5.2 · p. 28
- 100% clothes-folding success across all four configurations masks improvements in average task score and completion time from System 3 conditioning.Average scores are 68.3% (Base), 81.7% (+SubImage), 85.0% (+Value), and 88.3% (+SubImage+Value). Average completion times are 107 s (Base), 83 s (+SubImage), 79 s (+Value), and 75 s (+SubImage+Value).Source: §6.5.2 · p. 28, Figure 11 · p. 28
- 55% cube-sorting success with +SubImage+Value exceeds Base’s 40%, while the combined configuration also achieves the highest average task score.+SubImage achieves 50% success and +Value achieves 45%; average task score increases from 55.0% for Base to 87.5% for the combined configuration. Value conditioning alone produces the shortest observed completion time, 80 s.Source: §6.5.2 · p. 28, Figure 11 · p. 28
6.6. Foundation Model Evaluation
- The same pretrained GigaBrain-0.7 policy follows multiple language instructions on AgileX PiPER and Maker H01 without task-specific adaptation.Evaluation covers language-conditioned multi-task execution and longer-horizon manipulation under ID and OOD settings. The results demonstrate executable behaviors before downstream adaptation, rather than merely a stronger initialization for post-training.Source: Figure 12 · p. 29, §6.6.1 · p. 29, §6.6 · p. 29
6.6.1. Out-of-the-Box Language Following
- Qualitative OOD executions demonstrate target understanding, spatial grounding, and compositional generalization beyond training configurations, but both embodiments exhibit performance degradation.PiPER handles an unseen pepper with previously unpaired, randomly positioned containers; H01 handles grapes, a scene, and a task composition absent from H01 training. Generalization becomes more challenging when object identity, scene configuration, and task composition shift simultaneously.Source: §6.6.1 · pp. 30–31, Figure 13 · p. 30
6.6.2. Out-of-the-Box Complex Manipulation
- The pretrained policy sustains garment folding on AgileX PiPER and Maker H01 despite unseen garments and highly unstructured initial configurations.Execution requires repeatedly establishing contact and responding to evolving cloth geometry across successive action chunks, rather than replaying a fixed trajectory. OOD complex manipulation remains harder when interaction patterns shift, although executable behavior persists across multiple settings.Source: Figure 15 · p. 32, §6.6.2 · p. 33
6.7. Post-Training Evaluation
Task-specific post-training improves GigaBrain-0.7’s language following and complex manipulation over the preceding GigaBrain model across AgileX PiPER and Maker H01. Representative rollouts preserve fine-grained instruction grounding, although several contact-rich tasks remain far from saturation.
6.7.1. Multi-Task Language Following
- 84.2% average success versus GigaBrain-0.1’s 69.6% marks GigaBrain-0.7’s post-trained language-following performance on Maker H01.On AgileX PiPER, GigaBrain-0.7 consistently improves over the preceding GigaBrain model and matches or exceeds π0.5 across reported tasks. The six-task evaluation covers color, target, and direction discrimination.Source: §6.7.1 · p. 33, Table 6 · p. 36
- Post-trained policies preserve fine-grained instruction grounding across both embodiments, using color, object identity, and relative spatial relations to guide target selection and execution.Representative rollouts show instruction-dependent behavior rather than reproduction of a fixed manipulation trajectory.Source: §6.7.1 · p. 36
6.7.2. Complex Manipulation
- GigaBrain-0.7 consistently improves complex manipulation over the preceding GigaBrain model across both embodiments, remaining competitive with or stronger than π0.5 in completed evaluations.PiPER gains are modest on several structured tasks but clearer for sustained-contact or multi-stage interactions. Maker H01 shows substantially larger separation across broader skills, including deformable-object manipulation, food-related interaction, and multi-stage cleanup.Source: §6.7.2 · pp. 36, 39, Table 7 · p. 39
- Several contact-rich tasks remain far from saturation despite gains across manipulation categories, motivating additional data scaling and experience-based post-training.Representative complex-manipulation executions span deformable-object interaction, tool-mediated manipulation, household-item organization, and sequential sorting, with differing demands on contact, temporal coordination, and task progress.Source: §6.7.2 · p. 39
6.8. Benchmark Evaluation · 6.9. Experience-Driven Reinforcement Learning
GigaBrain-0.7 leads the reported simulation comparisons and achieves the strongest overall and spatial-understanding averages before MiMo-enhanced training. Experience-driven reinforcement learning improves all four evaluated real-robot tasks, with final online-RL policies reaching 100% success.
6.8. Benchmark Evaluation
- Benchmark evaluation pairs spatial understanding and affordance reasoning with bimanual manipulation, mobile and long-horizon interaction, and reconstruction-based Real2Sim2Real policy evaluation.RoboColiseum complements physics-based simulation by aligning reconstructed simulated observations more closely with real environments. Integration into XPolicyLab [17] is planned to support a unified training, inference, and deployment workflow.Source: §6.8.1 · p. 39, §6.8.2 · pp. 40–42
6.8.1. Embodied Vision-Language Evaluation
- Before MiMo data enter pretraining, GigaBrain-0.7 achieves the strongest overall and Spatial Understanding averages among models compared on MiMo-Embodied [34].Overall averages 13 benchmarks: eight Spatial Understanding and five Affordance benchmarks. VSI-Bench is excluded because it requires video, whereas the evaluated interface accepts image observations.Source: §6.8.1 · pp. 39–40, Table 8 · p. 40
- 0.5704 is the final checkpoint’s overall MiMo score after MiMo-enhanced continued pretraining, reported as a diagnostic rather than a held-out benchmark result.MiMo supervision enters joint vision-language/action optimization without a separate VLM adaptation stage. The released checkpoint uses this final stage, whose training mixture is no longer disjoint from the evaluation set.Source: §6.8.1 · p. 40
6.8.2. Simulation Benchmark Evaluation
- GigaBrain-0.7 leads RoboTwin 2.0 [15] overall/Hard, EBench [24] SR/Score among compared publicly available VLAs, and all four RoboColiseum [1] dimensions among evaluated open-source models.RoboTwin uses one policy jointly post-trained on 50 tasks with 50 clean demonstrations per task; several models perform better under Easy conditions. RoboColiseum evaluates instruction following, spatial reasoning, robustness, and general manipulation.Source: §6.8.2 · pp. 40–41, Table 9 · p. 41, Table 10 · p. 41
6.9. Experience-Driven Reinforcement Learning
- Experience reinforcement refines the same System 1 policy through advantage-weighted offline learning followed by on-policy actor–critic updates with human corrections.Offline optimization learns from successful, failed, and corrective rollouts without new interaction; online updates combine progress feedback with terminal outcomes. Evaluation spans long-horizon and precision-critical tasks across PiPER-family manipulators and Maker H01.Source: §6.9 · pp. 42–43
- 100% average success after online RL compares with 57.5% after offline RL and 30.0% for the SFT baseline, with every evaluated task reaching 100%.Online RL raises Link Installation and Cable Tie Insertion and Tying from 40% to 100%, Bearing Installation from 60% to 100%, and Gift Box Packing from 90% to 100%. The largest online gains occur on precision-critical alignment, insertion, and threading tasks.Source: Table 12 · p. 43, §6.9 · p. 43
- 40-percentage-point offline-RL gains occur in Bearing Installation (20% to 60%) and Cable Tie Insertion and Tying (0% to 40%).Gift Box Packing improves from 80% to 90%, while Link Installation improves from 20% to 40%. Offline gains depend on starting-policy capability, with the largest gains occurring where SFT encounters substantial failures.Source: §6.9 · p. 43
7. Conclusion and Future Work
GigaBrain-0.7 combines large-scale heterogeneous embodied pretraining with a three-system architecture, demonstrating strong generalization, post-training performance, and robustness across diverse evaluations. Future work targets data scaling and alignment, stronger predictive modeling, and scalable closed-loop learning.
Conclusion
- GigaBrain-0.7 demonstrates strong out-of-the-box generalization, post-training performance, and robustness across real-robot, embodied vision-language, and simulation evaluations.These evaluations span diverse tasks and embodiments.Source: §7 · p. 44
- Over 37,000 hours of embodied trajectories spanning 16 robot morphologies, alongside large-scale vision-language supervision, underpin GigaBrain-0.7’s pretraining.Its three-system architecture coordinates understanding, prediction, and action. One-stage VLA pretraining jointly optimizes multimodal understanding, hierarchical task prediction, discrete action supervision, and continuous action generation across heterogeneous embodiments.Source: §7 · p. 44
- Separately pretrained System 3 predicts future states and estimates task progress, providing conditioning during task-specific post-training and positive-progress guidance at inference.Source: §7 · p. 44
- Experience-driven offline and online reinforcement learning further improves the policy using rollout experience and corrective feedback.Source: §7 · p. 44
Future Work
- Future work targets heterogeneous data scaling and cross-source alignment, stronger long-horizon prediction and task-progress estimation, and scalable closed-loop learning from autonomous experience and human correction.Source: §7 · p. 44
A. Simulation Benchmark Leaderboard Snapshots
The appendix preserves complete simulation benchmark leaderboard snapshots at manuscript preparation for completeness and reproducibility, including submissions omitted from the main-text tables.
- The appendix records complete leaderboard snapshots for the simulation comparisons in Section 6.8.2 as of manuscript preparation.These snapshots support completeness and reproducibility.Source: A. Simulation Benchmark Leaderboard… · p. 45
- The main-text tables focus on representative publicly available methods for concise, reproducible comparison, while the appendix includes additional submissions.Source: A. Simulation Benchmark Leaderboard… · p. 45
- Each benchmark includes a source URL and snapshot access date because online leaderboards may change as new submissions arrive.Source: A. Simulation Benchmark Leaderboard… · p. 45
A.1. RoboTwin 2.0
The official RoboTwin 2.0 leaderboard reports performance under clean Easy and domain-randomized Hard settings. GigaBrain-0.7 uses the official Co-Train evaluation protocol.
A.1. RoboTwin 2.0
- GigaBrain-0.7 is evaluated under the official Co-Train protocol used for the comparison in Table 9.The supplied caption does not provide performance values or establish a ranking.Source: Figure 28 · p. 45
- The leaderboard reports performance under clean Easy and domain-randomized Hard settings.Source: Figure 28 · p. 45
- Figure 28 presents a snapshot of the official RoboTwin 2.0 leaderboard, including its evaluation protocol.Source: Figure 28 · p. 45
A.2. EBench
The complete official EBench leaderboard includes submissions beyond the main text’s representative publicly available model comparison. The appendix also clarifies the separate source of the InternVLA-A1 result.
- Qwen-RobotManip is retained in the complete official leaderboard record, beyond the representative publicly available model comparison in the main text.Source: §A.2 · p. 45
- The complete official EBench leaderboard reports overall task success rate and aggregate task score for all listed submissions at manuscript preparation.Source: Figure 29 · p. 46
- The InternVLA-A1 result in Table 10 comes from the EBench evaluation reported by Qwen-RobotManip [87], not the online leaderboard snapshot.A superscript asterisk identifies this result in the main-text table.Source: §A.2 · pp. 45–46
A.3. RoboColiseum
RoboColiseum reports separate leaderboards for instruction following, spatial reasoning, robustness, and general manipulation. This section provides snapshots associated with the comparison in Table 11.
A.3. RoboColiseum
- Four complementary capability leaderboards underpin the RoboColiseum comparison in Table 11, with separate snapshots provided in Figures 30–33.The dimensions are instruction following, spatial reasoning, robustness, and general manipulation.Source: §A.3 · p. 46
- Robustness and general manipulation have separate leaderboards, evaluating policy robustness under the corresponding RoboColiseum protocol and general manipulation capability, respectively.Their snapshots also reflect the leaderboards at manuscript preparation.Source: §A.3 · p. 46, Figure 32 · p. 48, Figure 33 · p. 49
- The instruction-following leaderboard evaluates language-conditioned instruction following by robot policies, while the spatial-reasoning leaderboard evaluates spatial reasoning.Both snapshots reflect the leaderboards at manuscript preparation.Source: Figure 30 · p. 47, Figure 31 · p. 48
Related papers
This paper cites
- Microsoft COCO: Common Objects in Context2014
- OpenVLA: An Open-Source Vision-Language-Action Model2024
- $π_0$: A Vision-Language-Action Flow Model for General Robot Control2024
- RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control2023
- GLM-5: from Vibe Coding to Agentic Engineering2026
- Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs2024
Cited by
- WorldBridge: Bridging Video and Action for Task Generalization in World Action Models2026
- REKI: Registering Knowledge in VLAs2026
- GreenWAM: A World Foundation Model for Robotics2026
- PerturBot: Breaking Shortcut Priors in Vision-Language-Action Models with Perturbative Training2026
- OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining2026