GigaWorld-0: World Models as Data Engine to Empower Embodied AI

GigaWorld Team; Angen Ye; Boyuan Wang; Chaojun Ni; Guan Huang; Guosheng Zhao; Haoyun Li; Jiagang Zhu; Kerui Li; Mengyuan Xu; Qiuping Deng; Siting Wang; Wenkang Qin; Xinze Chen; Xiaofeng Wang; Yankai Wang; Yu Cao; Yifan Chang; Yuan Xu; Yun Ye; Yang Wang; Yukun Zhou; Zhengyuan Zhang; Zhehao Dong; Zheng Zhu

Source-linked AI summary

GigaWorld-0: World Models as Data Engine to Empower Embodied AI

GigaWorld Team, Angen Ye, Boyuan Wang, Chaojun Ni, Guan Huang, Guosheng Zhao, Haoyun Li, Jiagang Zhu, Kerui Li, Mengyuan Xu, Qiuping Deng, Siting Wang, Wenkang Qin, Xinze Chen, Xiaofeng Wang, Yankai Wang, Yu Cao, Yifan Chang, Yuan Xu, Yun Ye, Yang Wang, Yukun Zhou, Zhengyuan Zhang, Zhehao Dong, Zheng Zhu

arXiv:2511.19861v2Nov 25, 2025cs.CVcs.RO

TL;DR

Embodied-AI training needs scalable, controllable data beyond costly real-world collection. GigaWorld-0 addresses this with coordinated video and 3D world-model components, and its generated data improves VLA performance in real robotic environments.

Problem
Embodied-AI agents rely heavily on expensive real-world data collection, motivating scalable and controllable synthetic training data.
Method
GigaWorld-0 combines controllable photorealistic video generation, physically grounded 3D scene modeling, and efficient GigaTrain training as a VLA data engine.
Results
Training VLA policies on GigaWorld-0-generated data improves task success, robustness, and zero-shot generalization in real-world robotic environments.
Takeaways & Limitations
GigaWorld-0 supports scalable synthesis of diverse, instruction-conditioned interaction data that transfers to real-world robotic tasks.

Abstract

from arXiv · show

World models are emerging as a foundational paradigm for scalable, data-efficient embodied AI. In this work, we present GigaWorld-0, a unified world model framework designed explicitly as a data engine for Vision-Language-Action (VLA) learning. GigaWorld-0 integrates two synergistic components: GigaWorld-0-Video, which leverages large-scale video generation to produce diverse, texture-rich, and temporally coherent embodied sequences under fine-grained control of appearance, camera viewpoint, and action semantics; and GigaWorld-0-3D, which combines 3D generative modeling, 3D Gaussian Splatting reconstruction, physically differentiable system identification, and executable motion planning to ensure geometric consistency and physical realism. Their joint optimization enables the scalable synthesis of embodied interaction data that is visually compelling, spatially coherent, physically plausible, and instruction-aligned. Training at scale is made feasible through our efficient GigaTrain framework, which exploits FP8-precision and sparse attention to drastically reduce memory and compute requirements. We conduct comprehensive evaluations showing that GigaWorld-0 generates high-quality, diverse, and controllable data across multiple dimensions. Critically, VLA model (e.g., GigaBrain-0) trained on GigaWorld-0-generated data achieve strong real-world performance, significantly improving generalization and task success on physical robots without any real-world interaction during training.

1. Introduction

GigaWorld-0 is a unified world-model data engine that combines controllable video generation with physically grounded 3D modeling for scalable embodied-AI data synthesis. Its evaluations report strong generation quality and improved real-world VLA performance.

1. Introduction: The authors position GigaWorld-0 as a scalable, controllable, photorealistic training-data source and plan to open-source its models and generation pipeline.The paper also identifies policy environments and policy generation as future world-model directions.
1. Introduction: GigaWorld-0 combines GigaWorld-0-Video and GigaWorld-0-3D to generate controllable embodied interaction data across visual, geometric, and physical conditions.The video stream controls appearance, object placement, and viewpoints, while the 3D stream targets geometric consistency and physical plausibility.
1. Introduction: GigaWorld-0-Video includes a foundation video model and post-training adaptations for appearance, viewpoint, and action control.The model suite includes Video-Dreamer, AppearanceTransfer, ViewTransfer, and MimicTransfer.
1. Introduction: Evaluations report state-of-the-art performance across physical plausibility, geometric consistency, text-to-video alignment, multi-view coherence, and visual fidelity.The paper also reports improved VLA task success, robustness, and generalization in real-world robotic environments.
1. Introduction: GigaTrain uses FP8 precision and sparse attention to make large-scale GigaWorld-0-Video training more efficient.The framework is presented as an efficiency mechanism for reducing training memory and compute requirements.

2. Related Work

Related work frames generated data and world models as increasingly important tools for embodied-AI learning. Prior approaches span multimodal scene generation, environmental variation, and video translation for richer training data.

2. Related Work: Recent world-model research increasingly uses generated data as a training source for embodied-AI learning.The related work describes world models as supporting scalable generation of training experiences.
2. Related Work: Unified multimodal pipelines synthesize aligned RGB, depth, surface normals, and 3D point clouds for coherent 4D scene reconstruction.The cited work reports improved policy learning over RGB-only training.
2. Related Work: Background inpainting and video-to-video translation increase environmental diversity by modifying scene textures and visual content.These approaches are presented as complementary strategies for expanding embodied-AI data.

3. GigaWorld-0 Models

The GigaWorld-0 model family combines photorealistic video generation with 3D representations that enforce spatial coherence and physical plausibility. Together, these components support large-scale data generation under diverse conditions.

GigaWorld-0-Video: GigaWorld-0-Video synthesizes photorealistic sequences with control over appearance, object placement, and camera viewpoints.This enables large-scale, high-quality data generation under diverse real-world conditions.
GigaWorld-0-3D: GigaWorld-0-3D uses 3D representations to enforce geometric consistency, spatial coherence, and physical plausibility.Its modeling includes object rigidity or deformability and physical constraints such as contact dynamics.

3.1. GigaWorld-0-Video

GigaWorld-0-Video is a controllable embodied-video foundation and data engine, combining efficient generation with appearance, viewpoint, and action-transfer capabilities for VLA training.

GigaWorld-0-Video-Dreamer: GigaWorld-0-Video-Dreamer uses flow matching and compressed video latents to support embodied image-text-to-video generation.The model uses a 3D-VAE with spatial-temporal compression and a Mixture-of-Experts design with routed experts.
GigaWorld-0-Video-Dreamer: GigaWorld-0-IDM infers robotic joint-angle trajectories from generated videos, using masked arm regions to improve robustness under cluttered visual conditions.Qualitative trajectories closely align with ground truth across 12 arm joints and 2 gripper degrees of freedom.
GigaWorld-0-Video-AppearanceTransfer: AppearanceTransfer synthesizes photorealistic variants with user-specified textures, colors, and lighting while preserving geometry and motion.The resulting data supports real2real and sim2real augmentation, and VLA models trained on it show improved robustness to appearance variation.
GigaWorld-0-Video-ViewTransfer: ViewTransfer generates novel viewpoints from single-view robot videos while transforming actions to maintain geometric task consistency.This expands datasets without additional real-world collection and improves VLA robustness to viewpoint shifts.
GigaWorld-0-Video-MimicTransfer: MimicTransfer converts first-person human-hand demonstrations into realistic robotic-arm videos using scene conditioning and inverse-kinematics-derived arm motions.The method addresses the human-hand–robotic-arm appearance gap and enables scalable augmentation for robot learning.

3.2. GigaWorld-0-3D

GigaWorld-0-3D builds geometrically consistent and physically grounded embodied scenes by combining generative asset reconstruction, view restoration, differentiable physics, and action generation. Its integrated pipeline produces high-quality training data for VLA models while supporting both simple and complex manipulation scenarios.

GigaWorld-0-3D-FG: Foreground asset generation filters reconstructed objects through segmentation and four-view geometric inspection, retrying failed stages before URDF export.The pipeline uses ImageSegChecker and MeshGeoChecker quality gates, with modified parameters and random seeds for resynthesis.
GigaWorld-0-3D-BG: Sparse-view background reconstruction uses view restoration to synthesize consistent intermediate views before dense 3DGS reconstruction.The refined views reduce artifacts and augment inputs for the second-stage reconstruction.
GigaWorld-0-3D-Phys: GigaWorld-0-3D-Phys assigns physical properties to robots and objects using differentiable parameter estimation and multimodal visual inference.The system estimates robotic parameters such as friction and controller gains, while predicting object scale, mass, and friction from orthographic views.
GigaWorld-0-3D-Act: GigaWorld-0-3D-Act uses MimicGen for simple scenarios and reinforcement learning for complex contact-rich tasks to generate scalable manipulation trajectories.Seed demonstrations support pose and layout augmentation, while converged policies generate physically plausible and diverse trajectories.
GigaWorld-0-3D: GigaWorld-0-3D combines 3DGS and meshes to provide photorealistic rendering alongside collision detection, dynamics simulation, and physical interaction.Foreground and background assets use complementary representations within the integrated pipeline.

4. GigaWorld-0 Training

GigaWorld-0 training combines public and proprietary robotic data with distributed, mixed-precision infrastructure for scalable video foundation-model training. The reported configuration study shows that FP8 reduces memory and training time, while stronger distributed memory optimization increases communication and latency.

4. GigaWorld-0 Training: GigaWorld-0 training combines public manipulation and locomotion datasets with thousands of hours of proprietary robotic data collected across 3,100 m2.The proprietary collection uses Agilex Cobot Magic and AgiBot G1 platforms across broad environment categories.
4. GigaWorld-0 Training: Video-Dreamer is trained at 480×768 resolution for 61-frame sequences to balance visual fidelity with training efficiency.The choice matches the 480p inputs used by many contemporary VLA models.
4. GigaWorld-0 Training: GigaTrain supports scalable multi-GPU and multi-node training with distributed execution, mixed precision, checkpointing, and configurable optimization modules.The framework is intended for both large-scale pretraining and resource-constrained post-training.
4. GigaWorld-0 Training: FP8 precision consistently reduces memory consumption and training time across distributed training frameworks.This result is reported for GigaWorld-0-Video-Dreamer training under the configurations summarized in Table 2.
4. GigaWorld-0 Training: FSDP-2 provides the best memory efficiency, followed by DeepSpeed ZeRO-2 and ZeRO-0, with stronger optimization increasing communication overhead and per-step latency.The comparison uses 8×H20 GPUs with batch size 32.

5. Experiments

GigaWorld-0 is evaluated for embodied-video quality, controllability, visual fidelity, geometric consistency, and downstream usefulness for VLA training. Across benchmarks, visual analyses, and robot deployments, it demonstrates strong generation quality and practical utility.

5.1. Benchmark Results: GigaWorld-0-Video-Dreamer achieves the highest overall score on PBench Robot Set despite having the smallest activated parameter count.It is compared with Cosmos-Predict2-14B, Cosmos-Predict2.5-2B, Wan2.2-5B, and Wan2.2-14B.
5.1. Benchmark Results: GigaWorld-0-Video-Dreamer consistently outperforms Cosmos-Predict2.5-2B across GR1-Env, GR1-Obj, and GR1-Behavior in instruction-following fidelity.The comparison uses DreamGen Bench evaluation after fine-tuning on the GR1 robot dataset.
5.2. Visualization Results: GigaWorld-0-Video supports photorealistic appearance editing and arbitrary-viewpoint synthesis while preserving scene structure, motion dynamics, or action consistency.AppearanceTransfer changes texture, material, and lighting; ViewTransfer expands single-view data with multi-perspective analogs.
5.2. Visualization Results: GigaWorld-0-Video-MimicTransfer converts first-person human demonstrations into robot-executable trajectories with spatial alignment, natural dynamics, and physical plausibility.This makes egocentric human videos usable as scalable supervision for robotic policy learning.
5.2. Visualization Results: GigaWorld-0-3D combines generated foregrounds, reconstructed backgrounds, and physically plausible dynamics to create geometrically consistent, simulation-ready manipulation environments.The resulting scenes integrate 3DGS and mesh-based modeling with articulated dynamics.
5.3. Downstream Task Results: Policies trained on GigaWorld-0 data demonstrate real-world performance across dexterous, long-horizon, and dynamic mobile manipulation tasks.Reported deployments include laundry folding, paper towel preparation, juice preparation, table bussing, boxes moving, and laundry baskets moving.

6. Conclusion

GigaWorld-0 is presented as a scalable, controllable data engine that unifies photorealistic video generation with geometrically consistent and physically grounded 3D simulation. The conclusion reports improved VLA task success, robustness, and zero-shot generalization, while identifying interactive policy environments as a future direction.

6. Conclusion: GigaWorld-0 unifies photorealistic video generation with geometrically consistent and physically grounded 3D scene simulation for embodied-AI data synthesis.It targets diverse, instruction-conditioned interaction data across textures, object configurations, and viewpoints.
6. Conclusion: Training VLA policies on GigaWorld-0-generated data improves task success, robustness, and zero-shot generalization in real-world robotic environments.The conclusion frames these improvements as evidence for the system’s utility as an embodied-AI data engine.
6. Conclusion: A proposed future direction is to use GigaWorld-0 as an interactive policy environment for model-based reinforcement learning.The stated goal is to let agents explore, plan, and refine behaviors in simulation before real-world execution.
6. Conclusion: GigaWorld-0-Video-MimicTransfer extends the system to cross-embodiment supervision by converting human demonstrations into robot-executable manipulation trajectories.This supports scalable use of human video data for VLA training.
6. Conclusion: GigaWorld-0-3D provides geometrically consistent renderings and physically realistic robot actions for embodied simulation.The 3D component is illustrated as supporting physically grounded scene and action generation.
6. Conclusion: The reported downstream deployments include manipulation and mobile tasks on G1 humanoid and PiPER platforms.Examples include laundry folding, paper towel preparation, table bussing, juice preparation, and laundry-basket moving.