GigaWorld-Policy: An Efficient Action-Centered World--Action Model

Angen Ye; Boyuan Wang; Chaojun Ni; Guan Huang; Guosheng Zhao; Hao Li; Hengtao Li; Jie Li; Jindi Lv; Jingyu Liu; Min Cao; Peng Li; Qiuping Deng; Wenjun Mei; Xiaofeng Wang; Xinze Chen; Xinyu Zhou; Yang Wang; Yifan Chang; Yifan Li; Yukun Zhou; Yun Ye; Zhichao Liu; Zheng Zhu

Source-linked AI summary

GigaWorld-Policy: An Efficient Action-Centered World--Action Model

Angen Ye, Boyuan Wang, Chaojun Ni, Guan Huang, Guosheng Zhao, Hao Li, Hengtao Li, Jie Li, Jindi Lv, Jingyu Liu, Min Cao, Peng Li, Qiuping Deng, Wenjun Mei, Xiaofeng Wang, Xinze Chen, Xinyu Zhou, Yang Wang, Yifan Chang, Yifan Li, Yukun Zhou, Yun Ye, Zhichao Liu, Zheng Zhu

arXiv:2603.17240v2Mar 18, 2026cs.CV

TL;DR

Robot policies receive sparse action supervision, while existing World–Action Models can incur high inference cost and couple action quality to future-video prediction. GigaWorld-Policy couples action prediction with action-conditioned video supervision during training, uses causal action-centered decoding, and makes video generation optional at inference. On real robots, it reports 9× faster inference and 7% higher task success than Motus, while improving performance by 95% over π0.5 on RoboTwin 2.0.

Problem
VLA action supervision is sparse and low-diversity, while existing joint world–action approaches can require costly video generation and entangle action prediction with visual forecasting.
Method
GigaWorld-Policy jointly predicts future actions and action-conditioned future videos during training, with causal masking and optional video decoding at inference.
Results
9× faster inference and 7% higher task success rates are reported over prior baselines, while performance improves by 95% over π0.5 on RoboTwin 2.0.
Takeaways & Limitations
The action-centered design provides dense visual-dynamics supervision while supporting action-only, low-latency control during deployment.
Takeaways & Limitations
Existing video-prediction approaches remain vulnerable to stochastic pixel forecasting and compounded long-horizon visual errors, motivating reduced reliance on explicit video prediction.

Abstract

from arXiv · show

World-Action Models (WAM) initialized from pre-trained video generation backbones have demonstrated remarkable potential for robot policy learning. However, existing approaches face two critical bottlenecks that hinder performance and deployment. First, jointly reasoning over future visual dynamics and corresponding actions incurs substantial inference overhead. Second, joint modeling often entangles visual and motion representations, making motion prediction accuracy heavily dependent on the quality of future video forecasts. To address these issues, we introduce GigaWorld-Policy, an action-centered WAM that learns 2D pixel-action dynamics while enabling efficient action decoding, with optional video generation. Specifically, we formulate policy training into two coupled components: the model predicts future action sequences conditioned on the current observation, and simultaneously generates future videos conditioned on the predicted actions and the same observation. The policy is supervised by both action prediction and video generation, providing richer learning signals and encouraging physically plausible actions through visual-dynamics constraints. With a causal design that prevents future-video tokens from influencing action tokens, explicit future-video generation is optional at inference time, allowing faster action prediction during deployment. To support this paradigm, we curate a diverse, large-scale robot dataset to pre-train an action-centered video generation model, which is then adapted as the backbone for robot policy learning. Experimental results on real-world robotic platforms show that GigaWorld-Policy runs 9x faster than the leading WAM baseline, Motus, while improving task success rates by 7%. Moreover, compared with pi-0.5, GigaWorld-Policy improves performance by 95% on RoboTwin 2.0.

1. INTRODUCTION

GigaWorld-Policy addresses sparse action supervision by coupling action prediction with future visual dynamics, while making video generation optional for efficient deployment. It reports faster inference and higher task success than strong baselines.

VLA models face supervision sparsity because high-dimensional observations receive sparse, low-diversity action labels.This can encourage shallow contextual behavior instead of physically grounded action modeling.
Future-state supervision can densify learning, but VLM-based VLAs are optimized for discrimination rather than high-fidelity generation.This makes continuity and physical consistency in predicted actions difficult to enforce.
GigaWorld-Policy jointly predicts future action sequences and action-conditioned future visual observations from the current context.The coupled objectives provide dense visual supervision and regularize action plausibility through 2D pixel-level dynamics.
Future-video prediction is optional at inference, allowing action-only decoding without rolling out long video-token sequences.This reduces compute and memory overhead and supports low-latency closed-loop control.
GigaWorld-Policy uses a pre-training paradigm that converts a generic video generation model into an initialization for robot policy learning.The approach leverages complementary data sources across training stages.
9× faster inference and 7% higher task success rates are reported over baseline methods on real robotic platforms.Compared with π0.5, the model matches inference speed and improves performance by 95% on RoboTwin 2.0.

2. Related Work

Prior world-model methods improve video generation and use visual dynamics for robot control, but often require costly future-video sampling. GigaWorld-Policy instead adapts video generation into an action-centered policy initialization with optional video prediction at inference.

2.1. World Models for Robotic Video Generation: World models aim to capture environmental temporal evolution by predicting future visual sequences.
2.1. World Models for Robotic Video Generation: Existing video world models emphasize fidelity, consistency, and controllability more than adapting generic video generators for low-latency policy learning.GigaWorld-Policy treats the video generator as policy initialization and aligns it with robotic observations and action-conditioned dynamics.
2.2. World–Action Models for Robotic Control: World–Action Models combine robot-action prediction with future visual dynamics to provide dense temporal supervision and predictive priors for policy learning.
2.2. World–Action Models for Robotic Control: VideoVLA jointly models video, language, and action modalities to predict actions and future visual outcomes.
2.2. World–Action Models for Robotic Control: Mimic-video first predicts future visual observations and then maps video latents to low-level actions with an inverse-dynamics decoder.
2.2. World–Action Models for Robotic Control: Existing methods typically require iterative diffusion sampling of future videos during inference, introducing latency that limits real-time deployment.Long-horizon pixel prediction can also compound small visual errors and weaken robust action generation.
2.2. World–Action Models for Robotic Control: GigaWorld-Policy jointly predicts action chunks and future videos during policy learning, while making the future-video branch optional at inference.This design enables faster inference while retaining visual dynamics as auxiliary supervision.

3. Method

GigaWorld-Policy jointly learns action prediction and action-conditioned future visual dynamics in one causal model, using visual forecasting as dense supervision while keeping action decoding efficient. Its architecture combines multi-view inputs, shared Transformer processing, causal token dependencies, and flow-matching objectives with optional video generation during inference.

Policy formulation: GigaWorld-Policy predicts action chunks from observations, robot state, and language while learning future visual observations conditioned on the same context and predicted actions.The unified model couples action learning with explicit 2D pixel-level state evolution.
Input representation: The model merges three camera views into one composite image, preserving shared spatial structure for cross-view consistency while forecasting sparse future frames at a fixed stride.Sparse prediction reduces redundancy from adjacent observations while retaining scene evolution.
Shared Transformer blocks: A single shared Transformer stack processes visual, state, action, and language-conditioned inputs, coupling action tokens with visual evidence without using an MoE design.Different token types receive modality-specific positional encodings, while language is supplied through cross-attention rather than the causal sequence.
Causal attention: The blockwise causal mask lets action tokens attend to states and current observations but prevents them from attending to future-video tokens.Future-video tokens can attend to states, observations, and actions, enabling feedforward dynamics prediction without information leakage into action generation.
Training objective: Training uses flow matching for both action prediction and future visual dynamics, combining the video and action objectives during post-training with scalar weights.Pre-training optimizes only the video objective, while post-training balances video and action losses.
Inference: At inference, the policy samples only action tokens from the learned action flow model, omitting future-video tokens for lower-latency control while retaining optional video decoding.Video tokens can later be denoised jointly with actions or using a cached context when future prediction is needed.

4. Experiment

Experiments evaluate GigaWorld-Policy across simulation, real-world manipulation, inference efficiency, data efficiency, pre-training, future-frame prediction, and causal masking. The results show faster inference with strong task performance, while ablations identify complementary pre-training benefits and gains from moderate future modeling.

Simulation Benchmark Experiments: GigaWorld-Policy achieves performance comparable to Motus in RoboTwin 2.0 while providing a 9× inference speedup.The simulation evaluation uses 50 manipulation tasks under domain randomization.
Simulation Benchmark Experiments: Over 44 percentage points higher average success rate than 𝜋0.5 is achieved in the RoboTwin 2.0 multi-task setting.The comparison is reported alongside performance comparable to Motus despite the 9× inference speedup.
Real-world Evaluation: 9× faster inference than Motus accompanies a 7% higher success rate in real-world robotic experiments.The real-world evaluation uses four tasks on an AgileX PiPER 6-DoF arm.
Data Efficiency: Real-world success rises from 57% without embodied pre-training to 83% with the full dataset as embodied pre-training data increases.The study reports a clear and consistent improvement across increasing data fractions.
Causal Self-Attention: Causal self-attention preserves similar SR while enabling optional video prediction and improving held-out video reconstruction quality over self-attention.The method achieves higher PSNR and SSIM, with more accurate fine-grained object-state and appearance changes in qualitative comparisons.

5. Conclusion

GigaWorld-Policy is an action-centered World–Action Model built on a robot-pretrained, action-conditioned video-generation backbone. It combines future action prediction with action-conditioned video generation during training, achieving faster inference and higher task success than prior baselines.

GigaWorld-Policy uses an action-conditioned video-generation backbone pre-trained on a curated, multi-level large-scale robot dataset.
The model jointly learns future action sequences and action-conditioned video dynamics, providing richer supervision for 2D pixel–action dynamics.
GigaWorld-Policy reduces inference overhead by 9× to 0.36 s per inference while improving task success rates by up to 7% over prior baselines.

A. Implementation Details.

The implementation comparison covers VLA and World-Action Model baselines, alongside pretraining configuration for GigaWorld-Policy.

Baselines: The VLA baselines include π0.5, GigaBrain-0, and X-VLA, while World-Action Model baselines include Cosmos-Policy and Motus.These baselines represent two dominant paradigms for robot policy learning.
VLA baselines: π0.5 combines heterogeneous robot, semantic, web, and low-level action supervision for open-world robotic generalization.Its final policy uses Flow Matching to predict continuous action chunks for real-time control.
VLA baselines: X-VLA uses embodiment-specific soft prompts with a shared Transformer backbone to adapt across heterogeneous robotic data sources.Its policy also uses Flow Matching for continuous action prediction.
VLA baselines: GigaBrain-0 uses world-model-generated data, RGBD input modeling, and embodied Chain-of-Thought supervision to improve cross-task generalization and robustness.The model targets dexterous, long-horizon, and mobile manipulation tasks.
World-Action Model baselines: Motus integrates understanding, video generation, and action modeling through a unified latent action world model and Mixture-of-Transformer architecture.It supports large-scale heterogeneous-data pretraining and mode switching through a UniDiffuser-style scheduler.
World-Action Model baselines: Cosmos-Policy fine-tunes a pretrained video model to jointly generate actions, future-state images, and value estimates as latent frames.It performs policy learning and test-time planning without architectural changes.
Pretraining configuration: Pretraining uses public datasets for 6000 GPU hours with a global batch size of 256 and a cosine-decayed learning rate from 1 × 10^-4 to 1 × 10^-6.The AdamW optimizer uses β1 = 0.85 and β2 = 0.9.

B. More details about the simulation experiments.

The simulation evaluation uses RoboTwin 2.0 across all 50 tasks in clean and randomized scenes, while deployment examples include bowl stacking and desk cleaning on PiPER arms.

Simulation benchmark: RoboTwin 2.0 evaluates Motus and other baselines on all 50 tasks under both clean and randomized scenes.The setup assesses standard effectiveness and robustness to scene-configuration changes.
Deployment examples: Figure 10 depicts GigaWorld-Policy deployed on PiPER arms for stacking bowls and cleaning a desk.

C. More details about the real world experiments.

The real-world experiments cover four manipulation tasks using demonstration trajectories for post-training, and the simulation comparison reports performance comparable to Motus with a 9× speedup.

Experimental setup: The real-world evaluation covers four manipulation tasks, with 50 demonstration trajectories collected per task for post-training.The scenarios and tasks are illustrated in Figures 5, 6, and 10.
Task descriptions: Clean the desk requires sequentially relocating tableware into a basket while placing plates beneath bowls.Objects have stochastic color attributes and obstacles are randomly distributed.
Task descriptions: Stack bowls requires manipulating two arbitrarily posed bowls into a securely nested configuration.
Task descriptions: Scanning a QR code evaluates multi-step planning, tool use, and precise visual alignment through scanner and object manipulation.The robot scans the object and returns it to its original location.
Task descriptions: Sweeping up trash requires tool use, sustained contact-force control, and adaptive trajectory adjustment to gather scattered objects.The robot uses a brush-like tool and dustpan.
Evaluation outcome: 9× inference speedup is achieved by GigaWorld-Policy while maintaining performance comparable to Motus on RoboTwin 2.0.The table marks the best results in bold and the second-best results with underlining.