PaperlayerView exact revision on arXiv

Source-linked AI summary

World Action Models: The Next Frontier in Embodied AI

Siyin Wang, Junhao Shi, Zhaoyang Fu, Xinzhe He, Feihong Liu, Chenchen Yang, Yikang Zhou, Zhaoye Fei, Jingjing Gong, Jinlan Fu, Mike Zheng Shou, Xuanjing Huang, Xipeng Qiu, Yu-Gang Jiang

arXiv:2605.12090v1cs.ROcs.CLcs.CV

TL;DR

VLA models map observations directly to actions without explicitly predicting environmental change, while WAM research lacks a unified framework. This survey formally defines WAMs, organizes their architectures and data, and synthesizes multidimensional evaluations, establishing a systematic account of the field and its open challenges.

  • Problem

    Standard VLA models lack explicit world-dynamics prediction, motivating predictive physical reasoning for embodied policy learning.

  • Method

    The survey formally defines WAMs as models unifying predictive state modeling with action generation and categorizes Cascaded and Joint architectures.

  • Results

    The survey establishes a systematic account of WAM architectures, scalable training datasets, and evaluation protocols spanning visual fidelity, physical commonsense, and action plausibility.

  • Takeaways & Limitations

    Action plausibility should be evaluated separately from visual appearance and physical realism because visually convincing videos may not yield executable robot behavior.

  • Takeaways & Limitations

    Principles for designing optimal mixtures of human video, other non-robot data, and robot demonstrations remain poorly understood.

Abstract

from arXiv · show

Vision-Language-Action (VLA) models have achieved strong semantic generalization for embodied policy learning, yet they learn reactive observation-to-action mappings without explicitly modeling how the physical world evolves under intervention. A growing body of work addresses this limitation by integrating world models, predictive models of environment dynamics, into the action generation pipeline. We term this emerging paradigm World Action Models (WAMs): embodied foundation models that unify predictive state modeling with action generation, targeting a joint distribution over future states and actions rather than actions alone. However, the literature remains fragmented across architectures, learning objectives, and application scenarios, lacking a unified conceptual framework. We formally define WAMs and disambiguate them from related concepts, and trace the foundations and early integration of VLA and world model research that gave rise to this paradigm. We organize existing methods into a structured taxonomy of Cascaded and Joint WAMs, with further subdivision by generation modality, conditioning mechanism, and action decoding strategy. We systematically analyze the data ecosystem fueling WAMs development, spanning robot teleoperation, portable human demonstrations, simulation, and internet-scale egocentric video, and synthesize emerging evaluation protocols organized around visual fidelity, physical commonsense, and action plausibility. Overall, this survey provides the first systematic account of the WAMs landscape, clarifies key architectural paradigms and their trade-offs, and identifies open challenges and future opportunities for this rapidly evolving field.

1 Introduction

VLA models have become the dominant paradigm for generalist embodied policy learning, but their reactive observation-to-action mappings do not explicitly predict environmental change. This survey defines World Action Models (WAMs) as models unifying predictive state modeling and action generation, and organizes their architectures, data, evaluation, and challenges.

  • Motivation: VLA models ground internet-scale vision-language representations into motor behavior and transfer across robot embodiments with minimal finetuning.This marks a qualitative advance over earlier task-specific controllers and establishes VLA models as the dominant paradigm for generalist embodied policy learning.
  • Motivation: Standard VLA models learn direct observation-to-action mappings without explicitly modeling how environments change under intervention.The lack of predictive physical reasoning limits generalization where anticipating future states is essential, motivating integration of world models.
  • Definition: WAMs unify predictive state modeling with action generation by targeting the joint distribution p(o′, a | o, l) over future states and actions.They are formally distinguished from models targeting actions alone.
  • Architecture: Existing WAMs comprise Cascaded architectures, which first synthesize anticipated future-state representations before deriving actions, and Joint architectures, which model state prediction and action generation together.The Cascaded objective factorizes as p(a | o′, o, l)p(o′ | o, l), whereas Joint WAMs directly model p(o′, a | o, l).
  • Survey scope: The survey categorizes WAM methods by Cascaded or Joint paradigm, generation modality, conditioning mechanism, and action decoding strategy.It also formalizes WAMs, disambiguates related concepts, and traces the development of world modeling and action generation.
  • Survey scope: The survey analyzes four data sources and evaluates WAMs through visual fidelity, physical commonsense, and action plausibility while identifying open challenges and future directions.The data sources are robot teleoperation, portable human demonstrations, simulation, and internet-scale egocentric video.

2 Definitions and Formalism

This section formalizes embodied intelligence probabilistically and defines World Action Models as embodied foundation models that jointly model future environmental states and actions. It distinguishes WAMs from VLAs, world models, and related predictive-agent paradigms through their predictive targets and action coupling.

  • 2 Definitions and Formalism: Embodied agents receive observations o ∈ O and language instructions l ∈ L, then produce actions a ∈ A while transitioning to a subsequent observation o′.Observations may include visual inputs, proprioceptive signals, and other sensory modalities.
  • 2.1 Foundational Paradigms: VLAs frame robot control as multimodal sequence modeling, mapping current observations and language instructions directly to action-token sequences.Their architectures typically leverage pretrained semantic latent spaces from LLMs or VLMs.
  • 2.1 Foundational Paradigms: World models are predictive transition functions that simulate future environmental observations o′ from preceding observations o and hypothetical intervention sequences a.They act as probabilistic state propagators describing how the environment changes in response to actions.
  • 2.1 Foundational Paradigms: WAMs unify environmental dynamics modeling with motor control, extending beyond VLAs’ direct observation-to-action mappings by predicting physical-environment evolution.Their defining objective is a joint or conditional distribution over future states and actions.
  • 2.1 Foundational Paradigms: WAMs require forward predictive modeling of quantifiable future states and coupled action generation aligned with those anticipated states.Future states may be explicit visual predictions or implicit physics-grounded latent representations, while coupling may be joint or cascaded.
  • 2.1 Foundational Paradigms: By jointly predicting states and actions rather than mapping observations directly to actions, WAMs leverage spatiotemporal priors for deeper physical understanding and stronger zero-shot generalization.The framework characterizes future states and actions within a unified model.
  • 2.2 Disambiguation: WAM vs. Related Concepts: WAMs are a modality-independent superset of Video Action Models: video is one predictive proxy, whereas WAMs can use alternative targets such as other world representations.VAMs specifically align actions with synthesized video frames.
  • 2.2 Disambiguation: WAM vs. Related Concepts: WAM differs from Video Policies in both structural heritage and predictive commitment, because WAMs are not conceptually tied to video-generation backbones.Video Policies are associated with generative video architectures such as Video Diffusion Transformers.

3 VLAs and World Models: Foundations and Early Integration · 3.1 Vision-Language-Action Models

The section traces VLA models’ shift from narrow imitation-learning policies toward language-conditioned, scalable systems that combine semantic reasoning with precise action generation. It also describes the field’s expansion beyond image-to-action mappings to richer embodied observations, including geometry and depth.

  • 3.1 Vision-Language-Action Models: Traditional imitation-learning models were often designed for isolated skills, limiting generalization in open-world environments.
  • 3.1 Vision-Language-Action Models: Language-conditioned policies addressed this limitation by interpreting multimodal task descriptions to compute control actions.
  • 3.1 Vision-Language-Action Models: Early VLA architectures explored Feature Modulation to fuse visual and linguistic inputs using FiLM-based layers.
  • 3.1 Vision-Language-Action Models: LLM success catalyzed VLA research based on Knowledge Priors and Large-scale Scaling, with LVLM weights providing internet-scale data, reasoning, and semantic understanding.
  • 3.1 Vision-Language-Action Models: VLA action generation developed along two paths: Autoregressive Tokenization for discrete sequential actions and Diffusion-based Synthesis for continuous, multi-modal distributions.These approaches balance high-level logical planning with low-level physical precision and support tasks from single-arm control to complex bimanual manipulation.
  • 3.1 Vision-Language-Action Models: Recent VLA definitions incorporate richer embodied observations to improve perception and interaction in the physical world.
  • 3.1 Vision-Language-Action Models: These extensions include 3D geometric information for stronger spatial representations and depth perception for improved operational precision.

3.2 World Models

World models are internal representations of environmental dynamics that predict how actions affect future states, enabling simulation, decision-making, and planning. They are organized by conditioning signal and modeling space, spanning action- or language-conditioned systems, explicit pixel prediction, and implicit latent dynamics.

  • Definition: World models represent environmental dynamics and action effects to predict future consequences for simulation, decision-making, and planning.The paper distinguishes world models from definitions that omit explicit modeling of how the environment evolves under intervention.
  • Conditioning: Action-conditioned world models predict future states or observations from the current state and executable control signals, capturing causal effects on environmental dynamics.Actions directly intervene in the environment and drive state transitions over time.
  • Modeling space: World models divide by prediction space into explicit models that generate future pixels or video frames and implicit models that learn dynamics in latent representations.Latent-space models encode observations into compact latent states and learn transition functions there.
  • Explicit video prediction: Pixel-level video world models use either autoregressive token prediction or diffusion-based generation to model future visual observations and multimodal trajectories.Autoregressive systems predict discrete future visual tokens from past observations, while diffusion systems iteratively denoise noisy spatiotemporal signals into coherent sequences.
  • Implicit latent dynamics: Latent dynamics models improve efficiency by modeling transitions in compact representations, including recurrent state-space models, transformer state-space models, and predictive representation learning.RSSM-based methods support latent-space planning, while transformer-based models target long-term dependencies and predictive representation methods forecast future embeddings instead of reconstructing raw observations.
  • Conditioning: Language-conditioned world models use high-level linguistic guidance to generate futures consistent with desired scenes, events, or evolution processes.Video foundation models trained on large-scale video-text pairs acquire priors about objects, physical interactions, camera motion, and temporal dynamics.

3.3 World Model for VLA

World models extend VLA beyond direct policy learning by predicting environment evolution, enabling future-state reasoning, imagined trajectories, outcome estimation, and pre-execution policy assessment. They support imitation learning, reinforcement learning, reward modeling, and reproducible virtual evaluation while addressing limited demonstrations, costly deployment, and simulator fidelity gaps.

  • Overview: World models enhance VLA by modeling environment evolution under actions, language, or multimodal contexts and enabling imagined trajectories, outcome estimation, and policy assessment before execution.They provide predictive capabilities beyond direct policy learning from static datasets.
  • Imitation learning (IL): In imitation learning, embodied world models generate diverse training data to mitigate limitations in the amount and diversity of expert demonstrations.The passage identifies world-model-generated data as a way to improve imitation-learning performance.
  • Reinforcement learning (RL): In reinforcement learning, world models predict future states and structured environment representations, supporting policy optimization without relying exclusively on costly and risky real-world deployment.RL also faces reward-design difficulties and unreliable simulation-to-reality transfer.
  • World Models as Surrogate Environments: World models serve as surrogate environments where agents perform imagined rollouts and use predicted trajectories to compute rewards for policy learning.Rewards may come from task-specific functions or rollout-level consistency objectives.
  • Policy Evaluation: Data-driven world models reduce reliance on costly, unsafe, and nonreproducible physical evaluations by enabling large-scale virtual testing with closed-loop, consistent rollouts.They aim to reflect real-world complexity and variability more faithfully than manually designed simulators, which can exhibit a significant sim-to-real gap.

4 Architecture

WAM architectures integrate predictive world modeling directly into policy generation, replacing offline supervision with an internal predictive core. They are organized into cascaded pipelines that sequentially generate future states and actions, and joint models that produce both simultaneously.

  • Architecture: Integrating world modeling into policy architecture turns it from an offline supervisor into an internal predictive core for real-time dynamics reasoning.
  • Cascaded WAMs: Cascaded WAMs first synthesize a visual future plan, then use a separate action model to decode executable robot commands.This decomposition separates long-horizon scene prediction from robot-kinematics reasoning but couples the two stages.
  • Joint WAMs: Joint WAMs unify predictive state modeling and action generation within one cohesive model, producing future states and actions simultaneously.
  • Cascaded WAMs: Cascaded methods use pixel, latent, flow, or structured geometric representations as intermediate planning carriers between world modeling and action decoding.Pixel plans are interpretable and exploit pretrained video models, while latent carriers avoid computationally expensive pixel-level decoding.
  • Action Extraction: Learned action extraction commonly applies inverse dynamics to generated videos, while geometric extraction analytically converts optical flow or tracked object poses into actions.AVDC derives SE(3) transformations from dense optical flow without action annotations; Dreamitate uses 6-DoF tool poses and inverse kinematics for joint commands.

5 Training data · 5.1 Robot-Centric Teleoperation Data

World Action Models require physically grounded embodied data that captures state transitions and action conditioning, while benefiting from both aligned robot trajectories and massive unpaired data. Robot-centric teleoperation therefore supplies precise grounding, with the field scaling across embodiments, modalities, and automated data generation.

  • 5 Training data: WAM training is bottlenecked by embodied data quality and availability because models require physical grounding for state transitions, action conditioning, and intuitive physics.Unlike passively trained language models, WAMs require data grounded in physical interaction.
  • 5 Training data: WAMs combine aligned (o_t, a_t, o_t+1) triplets with massive unpaired data, such as action-free videos, through joint training strategies.This unified data digestion couples internal representations while retaining flexibility across data types.
  • 5.1 Robot-Centric Teleoperation Data: Teleoperation records continuous sensory observations, proprioceptive states, and executable actions through remote human control, providing high-quality expert trajectories.Robot-centric datasets supply the strictly aligned, high-frequency action-state pairs required for WAM training.
  • 5.1.1 Scaling Up: Embodiment, Diversity, and Automated Augmentation: Early scaling efforts expanded trajectory collection, environmental variation, and embodiment coverage through QT-Opt, MT-Opt, RoboNet, and MIME.These works demonstrated massive self-supervised collection and initiated cross-robot and cross-domain generalization.
  • 5.1.1 Scaling Up: Embodiment, Diversity, and Automated Augmentation: Language-conditioned datasets including BridgeData, BC-Z, RT-1, and LanguageTable introduced semantic diversity and sequential reasoning in varied household settings.Language instructions help anchor visual state transitions to generalized human concepts.
  • 5.1.1 Scaling Up: Embodiment, Diversity, and Automated Augmentation: OXE harmonized over 1 million trajectories across 22 robots, while ARIO, RoboMIND, DROID, and UnifoLM-WBT extended aggregation, scene diversity, and embodiment coverage.UnifoLM-WBT specifically extends the paradigm to high-DoF humanoid embodiments.
  • 5.1.1 Scaling Up: Embodiment, Diversity, and Automated Augmentation: Unstructured play data and automated generation reduce the physical labor and annotation bottlenecks of manual teleoperation while broadening exploratory and grasp supervision.Jaco Play provides task-agnostic interaction priors; Grasp-Anything scales grasp data to the million-trajectory level, and UniVoxGen scales spatial reasoning through synthetic voxel generation.
  • 5.1.2 Deepening Perception: Multimodal and Contact-Rich Grounding: Robot-centric datasets increasingly add audio, multi-view 3D geometry, point clouds, tactile feedback, and dexterous supervision to address visual ambiguity and occlusion.Berkeley UR5 and OmniAction introduced audio, while RH20T, Robo360, and RoboData emphasized calibrated 3D capture and alignment.

5.2 Portable Human Demonstration Data (UMI-style)

Portable UMI-style demonstrations address teleoperation’s cost, laboratory, and embodiment bottlenecks by enabling low-cost, in-the-wild human data collection aligned to robot-executable actions. Their growing scale and multimodal richness provide WAMs with diverse environments paired with precise action constraints for learning action-conditioned dynamics.

  • Portable demonstration interface: UMI uses a lightweight handheld 3D-printed gripper and wearable cameras to collect everyday manipulation trajectories from non-experts, then retargets them into robot-executable actions.Vision-based tracking aligns human demonstrations with robot actions.
  • Data expansion: The UMI ecosystem expanded beyond its foundational pipeline through scalable construction, active perception, tactile feedback, and multi-view visual observations.These extensions increase sensory modality coverage and task complexity.
  • Dataset scaling: FastUMI-100K exceeds 100K trajectories with multimodal text annotations, while RealOmin introduces a million-scale corpus spanning over 3,000 diverse household environments.RealOmin captures proprioceptive, IMU, and tactile signals.
  • Dataset scaling: The RDT2-associated dataset provides roughly 10,000 hours of demonstrations across hundreds of real-world scenes, supporting force-aware and cross-view manipulation learning.Hoi! explicitly targets cross-view and force-aware manipulation for intuitive physics and contact dynamics.
  • Relevance to WAMs: Portable datasets combine egocentric-like environmental diversity with high-frequency, centimeter-level action constraints, enabling robust action-conditioned state-transition dynamics in unconstrained physical environments.This pairing is described as nearly impossible to acquire from traditional sources.

5.3 Simulation Data

Simulation offers WAMs scalable, controllable data generation with privileged spatial supervision and increasingly rich physical interaction modeling. Despite the sim-to-real gap and challenges in contact dynamics, simulation supports robust physical priors through domain randomization and synthetic tactile data.

  • Scalable Data Generation: Simulation bypasses teleoperation’s physical labor bottleneck while scaling data quantity, environmental diversity, and task complexity.ManiSkill2 unified manipulation benchmarking, while MimicGen and DexMimicGen procedurally expanded minimal human demonstrations.
  • Scalable Data Generation: 630k trajectories in InternData-A1 and 10 million grasping trajectories in SynGrasp-1B demonstrate simulation’s extreme data-generation scale.Scaling also extends beyond quantity toward greater sequential and semantic complexity.
  • Spatial Supervision: Simulation provides exact spatial ground truth that enables datasets for spatial and temporal world modeling despite real-world video’s 2D projections and occlusions.TesserAct provides 285k aligned RGB, depth, and surface normal video clips targeting 4D space-time modeling.
  • Contact Dynamics: Synthetic tactile supervision extends WAM training beyond vision to invisible, sub-millimeter force interactions in contact-rich manipulation.The TLA Dataset supplies aligned Tactile-Language-Action pairs for fingertip-based assembly, while newer physics engines target friction, deformation, and tactile feedback.
  • Sim-to-Real Transfer: Although the sim-to-real visual gap remains, domain randomization over textures, lighting, and cameras helps simulation imbue WAMs with robust physical priors and spatial reasoning.Underlying principles such as gravity, momentum, and nonpenetration remain physically consistent across simulation and reality.

5.4 Human and Ego-Centric Data

Human and egocentric data complement robot-centric datasets by exposing WAMs to unconstrained visual dynamics and intuitive physics beyond physical robots’ morphological and collection constraints. The field has progressed from short-horizon physical-event videos to action-grounded 3D annotations, future forecasting, and massive mixtures for generalist policy pretraining.

  • Motivation: Human and egocentric videos expose WAMs to long-tail real-world physics unavailable in costly, morphology-constrained robot-centric datasets.Robot datasets provide precise low-level execution but limited exposure to unconstrained physical dynamics.
  • Foundational video datasets: Over 100,000 SSv2 clips teach temporal reasoning and basic state transitions through events such as pushing, dropping, and tearing.Large-scale videos lack explicit proprioception but provide supervision for intuitive physics.
  • Foundational video datasets: Ego4D provides over 3,600 hours of unscripted egocentric video and benchmarks episodic memory and future-state forecasting.Its forecasting tasks train models to predict s_t+1 from current visual contexts, while web-scale datasets broaden open-world semantics.
  • Action-grounded human data: 3D hand and motion annotations convert human videos into action-conditioned dynamics, bridging visual interactions and robotic policy learning.Because raw video lacks action inputs for learning s_t+1 = f(s_t, a_t), researchers treat the human hand as a universal end-effector.
  • Action-grounded human data: Assembly101, H2O, and EgoPAT3D established structured assembly, multi-view 3D hand and 6D object poses, and future spatial anticipation for WAMs.Subsequent datasets scale this precision toward complex contact physics and view-invariant understanding.
  • Generalist pretraining mixtures: Massive data mixtures aggregate diverse human behavioral data into generalist policy-pretraining engines that project it directly into robotic action spaces.The Ego-Centric Human Manipulation Dataset and UniHand pool over 130 million frames and thousands of additional data sources or examples as described in the passage.

6 Evaluation

WAM evaluation should assess predicted-state fidelity, action effectiveness, and ideally their causal alignment, but current protocols evaluate world modeling and action policy separately with module-specific metrics.

  • Evaluation: Existing WAM evaluation follows a decoupled paradigm because no established protocol jointly assesses future-state fidelity, action effectiveness, and their causal alignment.World modeling and action policy capabilities are evaluated through separate, module-specific metrics.

6.1 How to Evaluate World Modeling Capability?

World action model evaluation extends beyond visual realism to test whether generated worlds preserve actionable information and model physically plausible dynamics. Current protocols span visual fidelity, physical commonsense, and motion or trajectory plausibility, with executable action tests exposing gaps that video quality metrics miss.

  • Evaluation Framework: Evaluation of world modeling capability is organized around three parallel aspects: visual fidelity, physical commonsense, and action plausibility.Unlike conventional video generation, WAM evaluation must assess both environmental dynamics and preservation of actionable information.
  • Visual Fidelity: Visual fidelity combines PSNR/SSIM, LPIPS/DreamSim/DINO similarity, and FVD to assess reconstruction, perceptual or semantic consistency, realism, and temporal quality.These metrics operate at pixel, feature, semantic, and distribution levels rather than relying on a single measure.
  • Physical Commonsense: Physical commonsense evaluation covers object dynamics and trajectory plausibility, including continuity, interactions, causal event ordering, long-horizon motion, and control alignment.Benchmarks such as PhyGenBench, VBench-2.0, WorldModelBench, WorldScore, and EWMBench operationalize these dimensions with automated judges, physical-law checks, optical flow, and EEF trajectory metrics.
  • Action Plausibility: Many visually convincing models achieve nearly zero real-world execution success when evaluated by the Inverse Dynamics Modeling Turing Test.The test infers actions from generated videos and evaluates those actions through real-world execution, revealing a gap between visual plausibility and executable representations.

6.2 How to evaluate Action Policy?

Action policy evaluation assesses whether WAMs produce precise, robust, and generalizable control signals across diverse scenarios. The benchmark ecosystem spans over 40 evaluations from 2019–2026, covering environments, sensors, demonstrations, and multidimensional metrics.

  • Evaluation Scope: Over 40 benchmarks proposed from 2019 to 2026 evaluate action policies across simulation setup, sensor modalities, demonstration scale, and multidimensional metrics.The review frames systematic policy assessment as increasingly important as WAMs move from passive video generation toward active robotic control.
  • General Manipulation: General manipulation dominates the benchmark ecosystem, progressing from standardized multi-task suites to larger evaluations emphasizing object, task, and trajectory diversity.MetaWorld and RLBench introduced 50 and 100 manipulation tasks, while ManiSkill and RoboCasa expanded scale across objects, tasks, and trajectories.
  • Generalization and Capability Dimensions: Recent benchmarks broaden policy assessment through visual and environmental perturbations, cross-task zero-shot protocols, language conditioning, real-scene reconstruction, and history-dependent tasks.These dimensions target robustness, multimodal prompt generalization, reality-gap measurement, and memory across temporal, spatial, object, and procedural information.
  • Embodiment and Mobile Manipulation: Bimanual, humanoid, and mobile-manipulation benchmarks impose greater demands through higher-dimensional action spaces, coordinated motion, navigation, planning, and dynamic execution.RoboTwin and BiGym evaluate coordinated dual-arm behavior, while ManipulaTHOR and HomeRobot combine manipulation with scene generalization or navigation.
  • Real-World Evaluation: Simulation offers efficient, reproducible evaluation but cannot reliably predict deployment performance because of the simulation-to-real gap, motivating real-robot benchmarks.RoboChallenge covers 30 real multi-task manipulation scenarios across 4 robot platforms, while Maniparena collects 10,812 real robot manipulation trajectories.

7 Open challenges and Opportunities

WAM research faces open challenges in coupling world prediction with action generation, representing multimodal physical states, using heterogeneous data, planning over long horizons, and achieving efficient, jointly grounded, and safe deployment. Addressing these challenges requires rethinking how WAMs are architectured, trained, evaluated, and verified.

  • Architectural Coupling: Architectural coupling lacks controlled comparisons across cascaded, joint, discrete, and implicit paradigms, while latent transition objectives may offer compact alternatives to pixel prediction.Whether latent-only representations preserve the grounding quality of generative approaches remains open.
  • Multimodal Physical State Representation: RGB-only prediction misses tactile, force, acoustic, and material information central to contact-rich manipulation, motivating modality-adaptive multimodal state prediction.This requires redefining the predicted world state as a joint distribution over relevant modalities rather than merely extending pixel prediction.
  • Data Utilization and Mixture Design: Data-mixture principles remain poorly understood, including each source’s marginal contribution, domain-gap effects, and whether human-video benefits are semantic or dynamical.Non-robot data may transfer knowledge across low-level physical priors, mid-level causal dynamics, and higher-level grounding roles.
  • Long-Horizon Planning and Temporal Abstraction: Long-horizon deployment compounds prediction drift and action errors, motivating modular planners, intrinsic hierarchies, multiresolution prediction, and expanded temporal context.It remains foundational whether these pathways should form a unified learnable hierarchy or a composite of specialized modules.
  • Inference Latency and Computational Efficiency: World prediction imposes a latency tax, so future systems should identify task-specific minimal sufficient predictive fidelity rather than universally accelerating high-fidelity synthesis.DreamZero reached 7Hz through algorithmic acceleration and low-level CUDA optimizations, but millisecond-level control remains difficult.
  • Evaluation Methodology and Safety: WAM evaluation should jointly relate physical prediction quality and action quality, while deployment requires computationally tractable safety verification robust to distributional shift.Pixel metrics can reward visually plausible but physically incorrect futures, whereas downstream success alone does not evaluate the core WAM premise.

8 Conclusions

The survey presents a systematic, critical analysis of the World Action Model landscape and establishes a conceptual framework for understanding it. It formally defines WAMs, distinguishes related methodologies, and organizes architectures into Cascaded and Joint paradigms with further analysis of generation, conditioning, and action decoding.

  • Conceptual framework: The survey formally defines WAMs, disambiguates them from related methodologies, and categorizes their architectures into Cascaded and Joint paradigms.It also examines generation modalities, conditioning mechanisms, and action decoding strategies within these paradigms.
Loading 2605.12090v1…