Source-linked AI summary
OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining
Yuran Wang, Siqiao Huang, Mingleyang Li, Chenhao Zhang, Jiaqi Liang, Weiyang Jin, Yue Chen, Xuemin Chi, Donghao Zhou, Qize Yu, Yu-Kai Wang, Yuhan Rui, Shenzhe Yao, Zhen Yuan, Zhenhao Shen, Kefei Zhu, Zijie Zhu, Ning Gao, Xiaowei Chi, Guanqi He, Shanghang Zhang, Hao Dong, Lin Shao, Hang Zhao
TL;DR
Existing World–Action Models tightly couple major design choices, obscuring which components and interactions matter. OpenWAM factorizes the design space into modular components and uses controlled studies to derive principles for world knowledge transfer, world–action synergy, and cross-domain scaling, then validates them in OpenWAM-α across simulation and real robots. The stack, models, protocols, and data recipes are released for further research.
Problem
Existing World–Action Model systems tightly couple generative backbones, representations, architectures, information flow, inference, and training data, obscuring which design choices matter and why.
Method
OpenWAM factorizes WAM design into composable modules with unified training, inference, deployment, and evaluation, then applies controlled studies and composes their principles into OpenWAM-α.
Results
OpenWAM-α delivers consistently strong results across eight simulation benchmarks and real-robot experiments spanning single-arm, bimanual, and dexterous-hand embodiments.
Takeaways & Limitations
The study provides a practical recipe for inheriting world knowledge, enabling world–action synergy, and scaling it across domains, with a released stack for reproducible research.
Takeaways & Limitations
The evaluation uses synchronous inference, pausing execution whenever the action buffer empties until a fresh action chunk is predicted.
Abstract
from arXiv · showhide
World-Action Models inherit world knowledge from video-generative priors, and channel it into executable control signals through embodied experience. Existing systems, however, are monolithic: the generative backbone, visual representation, architecture, information flow, inference procedure, and training data are tightly coupled, obscuring which design choices matter and why. We introduce OpenWAM, an open research stack that turns world-action pretraining into a controlled experimental program. OpenWAM-Infra factorizes the WAM design space into composable modules with unified training, inference, deployment, and evaluation. On this substrate, OpenWAM-Study examines three questions through controlled experiments: what to inherit, how world and action learning interact, and how their synergy scales; and distills three principles: upstream knowledge transfers through a sufficiently capable generative backbone and a compact, information-rich latent space; world-action synergy requires dedicated action capacity, explicit world-to-action information flow, and synchronized joint denoising; and embodied pretraining principally improves out-of-domain generalization, with one-stage co-training over egocentric and robot data integrating world coverage and action grounding. Composing these principles, we build OpenWAM-α, an open WAM pretrained on roughly 6,400 hours of egocentric human and robot data and evaluated across simulation and real-world benchmarks. Across the eight simulation benchmarks and the real-robot experiments, which together span embodiments from single-arm and bimanual manipulation to dexterous hands, OpenWAM-α delivers consistently excellent performance, sustaining its top-tier standing from simulation to the physical world. We release the full stack, including infrastructure, evaluation protocols, pretrained models, and data recipes, to facilitate future research.
1 Introduction
OpenWAM frames World–Action Model research as a controlled study of how world knowledge, action learning, and their interaction should be designed. It distills principles for inheritance, synergy, scaling, and an open pretrained model validated across simulation and real robots.
- Embodied agents must model both how the world evolves and which actions can cause desired changes.
- OpenWAM addresses monolithic systems by exposing backbone, representation, architecture, information flow, inference, and data composition as controlled design choices.
- Upstream world knowledge transfers most effectively through a capable generative backbone and a compact, information-rich visual latent space.
- World–action synergy requires action-specific capacity, explicit world-to-action information flow, and a joint test-time denoising schedule rather than parameter count or joint prediction alone.
- Embodied pretraining primarily improves out-of-domain generalization: egocentric video broadens world coverage, robot trajectories provide executable action knowledge, and joint training integrates both.
- OpenWAM-α scales these principles to 518M frames, approximately 6,400 hours, of egocentric human and robot data.
- OpenWAM-α delivers consistently strong results across eight simulation benchmarks and preserves this standing in real-robot experiments spanning single-arm, bimanual, and dexterous-hand platforms.
- The released stack includes modular infrastructure, evaluation protocols, pretrained weights, and data recipes for reproducible comparison and extension.
2 Related Work
Related work spans predictive world models, executable World–Action Models, open robot-learning ecosystems, and empirical approaches to studying model design. OpenWAM extends these directions by applying controlled, modular experimentation specifically to WAMs.
- World–Action Models: World models learn predictive structure for planning, reinforcement learning, and policy evaluation, whereas WAMs connect predictive capacity more directly to executable behavior as policy models.
- World–Action Models: Existing WAM implementations couple architecture and infrastructure choices, leaving it unclear which components transfer world knowledge, create synergy, or generalize across domains.
- Open Research Ecosystems for Generalist Robot Policy Learning: Open robot-learning ecosystems provide open weights, modular platforms, pretrained models, and unified standards for generalist policy research.
- Towards a Scientific Understanding of Model Design: Empirical model-design research decomposes systems into controlled variables, tests mechanisms behind gains, derives recipes, and validates whether they survive scale.
- Towards a Scientific Understanding of Model Design: OpenWAM brings controlled-comparison methodology to WAMs, covering multitask transfer, heterogeneous supervision, action design, and embodiment interfaces.
3 OpenWAM-Infra: A Modular Infrastructure for World–Action Modeling
OpenWAM-Infra factorizes World–Action Models into interchangeable visual, stream, and attention modules, enabling controlled architectural comparisons without tightly coupled implementations. It provides unified training around joint video-action flow matching and supports multiple communication patterns between world and action streams.
- Motivation: OpenWAM-Infra addresses monolithic, idiosyncratic WAM systems by separating model, representation, training, inference, deployment, and evaluation choices.This modularization is intended to make systems easier to extend and comparisons less confounded by component coupling.
- Composable Model: A concrete architecture is formed as C(E, S, M), combining a visual encoder, stream backbones, and a visibility attention mask.The visual encoder maps observations to world-stream latents, stream backbones process world, action, and optional understanding streams, and the mask controls within- and cross-stream visibility.
- Composable Model: The composition rule sequences backbone stages and can split attention blocks to enable cross-stream interaction without modifying backbone internals or adding parameters.All learned capacity remains in the stream backbones, while composition determines where and how information crosses streams.
- Architecture Families: OpenWAM-Infra supports Single-System, Dual-System, and Tri-System architecture families with variants that differ in modality-specific capacity and world-action communication.Examples include shared processing, dedicated action experts, joint self-attention, joint cross-attention, and inverse-dynamics conditioning.
- Architecture Families: Dual-System variants use independent video and action backbones connected by joint self-attention, joint cross-attention, or a two-stage inverse-dynamics pathway.Joint cross-attention can propagate action-learning gradients into the video stream or detach them, while IDM generates the video trajectory before predicting actions.
- Training Runtime: Every architecture is trained with one sample contract and one joint flow-matching objective that predicts video and action velocities in a single forward pass.Independent video and action noise timesteps cover the full noise plane, keeping synchronous and asynchronous inference schedules in-distribution.
4 OpenWAM-Study: Design Principles for World–Action Models
OpenWAM-Study uses controlled experiments to identify how WAMs should inherit world knowledge, build world–action synergy, and scale embodied pretraining. It finds that capable generative backbones, compact information-rich latents, explicit training-time information flow, synchronized denoising, and mixed embodied data are central design choices.
- Study scope: The study evaluates upstream priors, world–action interaction, and cross-domain embodied pretraining using RoboTwin2.0 in-domain and out-of-domain settings.The evaluation uses a widely adopted bimanual manipulation benchmark spanning over 50 tasks.
- Generative world priors: Increasing video-backbone capacity consistently improves average WAM success, while the 5B model offers a performance-efficiency trade-off for controlled studies.Wan2.1-I2V-14B performs best, but Wan2.2-TI2V-5B trails by only 1.40 points despite nearly 3x fewer parameters.
- Visual representation priors: WAM latent quality depends on compactness and rich world information rather than whether the encoder is reconstructive or representational.S-VAE dimension contraction makes DINOv3 and V-JEPA 2.1 variants outperform FLUX.2-VAE.
- World–action synergy: World–action synergy improves with greater action-specific capacity, explicit world-to-action information flow, and synchronized joint denoising.Tri-system architecture performs best overall, while dual-system joint self-attention is retained to balance performance and complexity; asynchronous denoising provides no gain.
- Embodied pretraining: Embodied pretraining primarily improves out-of-domain generalization, with robot data strengthening action grounding and egocentric data broadening transfer.Robot-only pretraining is strongest in-domain, whereas mixed strategies generalize better out-of-domain; one-stage co-training is adopted as the practical default.
- Pretraining and information flow: After embodied pretraining, mutual world–action visibility becomes preferable, supporting bidirectional exchange between predicted future frames and actions.The preference reversal appears across both RoboTwin2.0 evaluation protocols and is not attributed to domain shift or evaluation protocol.
5 OpenWAM-α: From Principles to a Pretrained Model
OpenWAM-α scales the study’s design principles into an open pretrained model and performs strongly across diverse simulation and real-robot settings. Its main boundary is that WAMs fit in-distribution data well but remain less robust to distribution shifts and visual disturbances than leading VLAs.
- From Principles to a Pretrained Model: OpenWAM-α instantiates the study’s principles at scale as an open foundation world–action model for diverse robotic tasks.The model is guided by the empirical insights established in OpenWAM-Study.
- Simulation Performance: OpenWAM-α sits in the top tier across LIBERO, VLABench, RoboTwin2.0-Full, RoboCasa365, and RoboCasa-GR1.It remains within a marginal gap of the best model across these single-arm, bimanual, mobile, and dexterous-hand benchmarks.
- Simulation Performance: OpenWAM-α leads EBench by roughly 4 points in both SR and Score over Qwen-RobotManip.EBench is the mobile bimanual benchmark.
- Simulation Performance: OpenWAM-α is the strongest WAM on RoboTwin2.0-Clean2Random and RoboDojo, although VLAs remain ahead on those bimanual leaderboards.The model retains a clear advantage over other WAMs while trailing the best models on these tasks.
- Limitations and Robustness: LIBERO-Plus is the main exception, with marked losses concentrated under camera and noise perturbations.These deficits are associated with the sensitivity of pixel-level prediction architectures to visual disturbances.
- VLA versus WAM: WAMs fit in-distribution tasks better, whereas VLAs generalize better out of distribution; sufficiently rich pretraining data can compensate for either deficit.Video-latent supervision supports in-distribution fitting, while long-horizon prediction can accumulate error under distribution shift.
- Real-Robot Evaluation: OpenWAM-α leads most single-arm real-robot tasks and reaches the state of the art across the great majority of RoboDojo-Real bimanual tasks.The results support general and fine-grained manipulation capability across multiple real-world embodiments.
6 Conclusions
OpenWAM turns world–action modeling into a modular, controlled research program and applies its resulting recipe in OpenWAM-α. The released infrastructure, protocols, weights, and data recipes are intended as a reproducible foundation for further research.
- Contributions: OpenWAM-Infra factorizes WAM design choices into composable modules with shared training, serving, and evaluation across simulation and real robots.The infrastructure supports controlled experiments across eight simulation benchmarks and real-robot platforms.
- Contributions: OpenWAM-α applies the distilled recipe to egocentric human and robot data through a unified action space and performs strongly across simulation and real-robot experiments.The evaluations span single-arm, bimanual, and dexterous-hand platforms.
- Release and Future Work: The authors release infrastructure, evaluation protocols, pretrained weights, and data recipes as a shared, reproducible foundation for world–action research.OpenWAM-α is positioned as a baseline for future development and comparison.
- Release and Future Work: Future directions include larger and more diverse embodied data, compact robust visual representations, and end-to-end designs combining WAM and VLA strengths.The paper points to these directions while referring readers to Section A for a fuller limitations discussion.
A Limitations and Future Work
The work identifies limitations in training-phase coverage, supported architectures, and pretraining data mixtures, and points to corresponding directions for future research.
- Training phases: The study mostly focuses on embodied pretraining, leaving post-training and adaptation methods and their underlying mechanisms open.
- Architecture: The supported architecture variants mainly use cross-modality attention or hard-routed MoE for modality fusion.Future work is encouraged to explore tokenization-phase early fusion and soft-routed MoE.
- Pretraining data mixture: The pretraining mixture excludes UMI-style data, which may provide task and scene diversity relevant to out-of-domain generalization.Future co-training could use robot-executable data with broader scene and task diversity.
B Training Details
The training details organize OpenWAM-α optimization around multi-domain pretraining, with configurations summarized in Table 9 and a large distributed setup.
- Pretraining setup: Multi-domain pretraining uses 16 nodes with eight NVIDIA H200 GPUs per node, totaling 128 GPUs, for approximately seven days.
- Pretraining setup: The global batch size is 3,072 clips per optimizer step, using 24 clips per GPU without gradient accumulation.
- Pretraining setup: Table 9 summarizes the key hyperparameters used for OpenWAM-α pretraining.
B.2 Dataset-Specific SFT Configuration
Dataset-specific supervised fine-tuning keeps the pretraining configuration fixed while varying three benchmark-dependent settings.
- SFT configuration: Supervised fine-tuning changes only the global batch size, training length, and image-augmentation setting across benchmarks.
- SFT configuration: All downstream models are initialized from the same pretrained OpenWAM-α checkpoint.
- SFT configuration: Table 10 reports the dataset-specific SFT configurations, with unlisted settings inherited from pretraining and “–” indicating no image augmentation.
C Real-World Evaluation Protocols
The real-world evaluation documents the task setup and evaluation protocol separately for each embodiment.
- Evaluation protocol: The real-world evaluation specifies the task setup and corresponding evaluation protocol for each embodiment.
C.1 Single-Arm Real-Robot Experiments
The single-arm real-robot evaluation uses six Franka-Research-3 tabletop tasks spanning stacking, hanging, and drawer manipulation, with task-specific fine-tuning and repeated trials.
- Six real-world tabletop tasks cover stacking, hanging, and drawer manipulation on the Franka-Research-3 platform.The setup uses a parallel gripper and RGB observation cameras.
- Execution sequences visualize six uniformly sampled frames from each task's rollout video.
- Each task is fine-tuned with 100 task-specific real-robot demonstrations before evaluation.
- Success rates are reported from 20 independent real-world trials per task.A stacking, hanging, or drawer trial succeeds only when its complete task criterion is met.
C.2 Dexterous-Hand Real-Robot Experiments
The dexterous-hand evaluation covers four instructed manipulation tasks on a physical platform and measures both complete-task success and partial progress under ID and OOD conditions.
- Four tasks evaluate stacking, shuttlecock collection, bottle-cap twisting, and clothing placement with a dexterous hand.Each task is specified by a natural-language instruction defining the manipulation objective.
- OOD evaluation changes object identity, layout, illumination, or background while preserving the instruction and overall manipulation objective.
- The protocol reports progress score and final success rate across independent ID and OOD trials.
- Final success requires complete task achievement, whereas progress score measures the fraction of required manipulation elements completed.The progress score is defined per trial using successfully completed elements relative to total elements present.
- Task-specific success criteria require all required objects or actions to be completed, including three-disc ordered stacking and full clothing placement.The bottle-cap progress score counts twisting off the cap regardless of final stable grasp, while final success also requires a stable grasp.
C.3 Bimanual Real-Robot Experiments
The bimanual real-robot evaluation covers all 18 RoboDojo-Real tasks across the ARX X5, Piper, and Piper X embodiments using the official unified protocol.
- All 18 RoboDojo-Real tasks are evaluated on the official real-robot platform.
- The evaluation spans three bimanual embodiments: ARX X5, Piper, and Piper X.
- The experiments follow the unified protocol defined by the official evaluation team.
D Per-Benchmark Simulation Results
The per-benchmark results report full scores across eight simulation benchmarks, grouping baselines as VLA or WAM and marking the best and second-best values.
- The tables group baselines into VLA and WAM categories, with bold marking the best value and underlining marking the second best.
- LIBERO scores are reported in the main text rather than repeated in the summarized tables.
- Fast-WAM is listed with reported values of 4.7 and 7.6 in the summarized benchmark results.