World Action Models: The Next Frontier in Embodied AI

Motivation

• VLA models such as RT-2 [1], OpenVLA [2], and 𝜋0 generalize semantic understanding into motor behavior but lack explicit prediction of environmental changes under intervention.This missing physical foresight limits generalization when anticipating future states is essential, motivating world-model integration through video prediction, latent dynamics, or unified state-action generation [4–22].Source: §1 · pp. 1–2

Definition

• WAMs unify predictive state modeling with action generation by modeling p(o′, a | o, l), rather than actions alone.Cascaded WAMs factorize future-state prediction and action generation, whereas Joint WAMs co-optimize their joint distribution in a shared representational space.Source: §1 · pp. 2–3

Open Challenges

• The survey concludes by identifying critical hurdles and future directions toward more robust and generalizable World Action Models.It presents a systematic conceptual framework intended to clarify fragmented terminology and guide researchers entering the field.Source: §1 · p. 3

Background and Training Datasets

• The survey traces WAMs from classical model-based reinforcement learning through modern foundation models and analyzes four data sources: robot teleoperation, portable human demonstrations, simulation, and internet-scale egocentric video.World modeling also enables use of large-scale human videos without action annotations, expanding the data foundations for embodied policy learning.Source: §1 · p. 3

Architecture

• The survey organizes WAM architectures into Cascaded and Joint paradigms, further distinguishing generation modality, conditioning mechanism, and action decoding strategy.This framework supports systematic comparison across the field’s design space; Fig. 1 depicts divergent autoregressive, diffusion-based, unified-stream, multi-stream, explicit-alignment, and implicit-alignment trajectories.Source: Figure 1 · p. 2, §1 · p. 3

Evaluation

• Evaluation is organized around visual fidelity, physical commonsense, and action plausibility, while identifying gaps in current protocols for assessing WAM capabilities.The survey synthesizes the emerging evaluation landscape rather than limiting assessment to action-generation performance.Source: §1 · p. 3

2 Definitions and Formalism

• Embodied intelligence is formulated probabilistically, with an agent receiving multimodal observation o and language instruction l before producing action a and subsequent observation o′.Observations may include visual inputs, proprioception, and other sensory modalities.Source: §2 · p. 3

2.1 Foundational Paradigms

• Vision-Language-Action models directly map current observation o and language instruction l to action-token sequences using pretrained LLM or VLM semantic spaces.Their objective models the conditional probability of actions given multimodal context.Source: §2.1 · p. 3
• World models instead act as predictive transition functions that simulate future observation o′ from preceding state o under hypothetical action interventions a.They probabilistically propagate states to represent how environments change in response to actions.Source: §2.1 · p. 5
• WAMs unify environmental dynamics modeling with motor control, requiring both forward prediction of future states and action generation aligned with those anticipated states.Future states may be represented explicitly through video frames or optical flow, or implicitly through physics-grounded latent spaces; action coupling may be joint or cascaded.Source: §2.1 · p. 5
• By modeling joint or conditional future-state and action distributions rather than observation-to-action mappings alone, WAMs leverage spatiotemporal priors for deeper physical understanding and stronger zero-shot generalization.This formalizes WAMs as predictive state-action models rather than purely reactive policies.Source: §2.1 · p. 5

2.2 Disambiguation: WAM vs. Related Concepts

• WAMs subsume Video Action Models because they are modality-independent, allowing predictive targets such as image transitions, point clouds, tactile feedback, and force feedback beyond video frames.The term “World” emphasizes internalized physical laws and causal dynamics rather than a pixel-level video format.Source: §2.2 · p. 6
• Unlike video policies that may map observations directly to actions using video-derived representations, WAMs require an explicit world-modeling objective that predicts the next state o′.WAMs are backbone-agnostic, whereas video policies are conceptually tied to generative video architectures.Source: §2.2 · p. 6
• WAM replaces the earlier Action World Model terminology to frame predictive physics and motor control as co-equal components of a complete embodied foundation model.This nomenclature positions WAMs as agents and conceptual successors to the Vision-Language-Action lineage rather than augmented simulators.Source: §2.2 · p. 6

3.1 Vision-Language-Action Models

• Early VLAs replaced narrow, task-specific imitation learning with language-conditioned policies for open-vocabulary manipulation [247].Initial fusion paradigms included Feature Modulation [119], Cross-attention Mechanisms [1] [214], and Simple Concatenation [214] [246].Source: §3.1 · p. 6
• Action Chunking and Temporal Ensembling improved motion smoothness and temporal consistency in early VLA systems [248] [249].Source: §3.1 · p. 6
• LLM-driven VLA research leveraged pre-trained LVLM weights, knowledge priors, and internet-scale data for stronger reasoning and semantic understanding [1] [2].Source: §3.1 · p. 6
• Autoregressive Tokenization generated discrete action tokens sequentially, while Diffusion-based Synthesis produced continuous, multi-modal action distributions [1] [2] [251] [254].Together, these paradigms balance high-level logical planning with low-level physical precision across single-arm and complex bimanual tasks.Source: §3.1 · pp. 6–7
• VLA models expanded beyond images by incorporating 3D geometry [255], depth [256], and force or tactile feedback [257] for richer physical interaction.These modalities strengthened spatial representations, operational precision, and fine-grained force-controlled assembly.Source: §3.1 · p. 7
• Despite multimodal physical sensing, VLAs remain rooted in reactive mapping and therefore often struggle with manipulation generalization.Source: §3.1 · p. 7

3.2 World Models

• World models internally represent environmental dynamics and action effects, enabling prediction of consequences for simulation, decision-making, and planning.They are organized as action-conditioned, language-conditioned, and embodied world models.Source: §3.2 · p. 7

3.2.1 Action-conditioned World Models

• Action-conditioned world models predict future observations from current states and executable controls, using either explicit pixel-level or implicit latent-space dynamics.Pixel models include autoregressive and diffusion formulations, while latent approaches include RSSM, TSSM, and predictive representation learning.Source: §3.2.1 · pp. 7–8
• Autoregressive models generate arbitrarily long token sequences but accumulate errors, whereas diffusion models capture multimodal futures and improve long-horizon consistency at higher computational cost.Genie [278] infers latent actions from unlabeled videos, while Diffusion World Model [279] reduces step-wise error accumulation through iterative denoising.Source: §3.2.1 · p. 8

3.2.2 Language-conditioned World Models

• Language-conditioned world models use semantic instructions rather than low-level controls to generate plausible visual futures, leveraging video-text priors for world evolution, synthesis, and planning.Their development spans GANs, U-Net and Transformer diffusion backbones, and compact latent representations.Source: §3.2.2 · pp. 9–10

3.2.3 Embodied World Model

• Embodied world models target physical dynamics for robotic simulation, planning, and demonstration synthesis, increasingly learning from unlabeled human and internet-scale videos.Genie [278], SWIM [34], DreamDojo [35], and DexWM [326] address latent action inference, morphology gaps, cross-embodiment transfer, and dexterous interaction modeling.Source: §3.2.3 · pp. 10–11
• Embodied models improve generated demonstrations through compositional control, action alignment, multi-view consistency, physics-aware modeling, multimodal sensing, and direct imagination-to-action integration.RoboDreamer [36], IRASim [327], MiLA [328], Ctrl-World [38], RoboScape [37], WoW [332], VT-WM [333], and PointWorld exemplify these directions.Source: §3.2.3 · p. 11

3.3 World Model for VLA

• World models predict future observations and outcomes under actions or multimodal contexts, enabling trajectory imagination, policy assessment, and reward-informed VLA learning before physical execution.They address expensive, unsafe, and difficult-to-scale real-world data collection and evaluation.Source: §3.3 · p. 11

3.3.1 World Models for Learning

• World models augment imitation learning by generating diverse demonstrations, synthesizing successful trajectories, and enabling one-shot or real-data-comparable policy training, including Ctrl-World’s 44.7% improvement for π0.5-DROID.DREMA [39] supports novel object configurations, while RoboScape [37] produces synthetic data for π0 and Diffusion Policy.Source: §3.3.1 · p. 12
• In model-based RL, world models enable safe imagined rollouts, implicit reward estimation, structured world understanding, and improved sim-to-real robustness without extensive physical interaction.Surrogate-environment approaches train policies from predicted observations or rollout consistency, while reward-modeling approaches derive signals from learned dynamics, video distributions, latent representations, or jointly predicted rewards [40] [41] [42] [44] [45] [43] [48] [49] [52] [53] [54] [55] [336] [337] [50] [51] [57] [58] [59] [47].Source: §3.3.1 · pp. 12–13

3.3.2 World Models for Evaluation

• World-model evaluation addresses physical testing’s cost, safety, nonreproducibility, and sim-to-real limitations, which manually designed simulators such as IsaacGym [339] and MuJoCo often fail to overcome.The central limitation is that conventional simulators inadequately capture real-world complexity and variability.Source: §3.3.2 · p. 13
• Data-driven world-model simulators enable closed-loop, reproducible, and large-scale policy testing grounded in real distributions, including out-of-distribution generalization, safety red teaming, and interactive teleoperation [38] [61] [62] [63] [64].Ctrl-World [38] conditions every frame on actions; Veo Robotics validates predictions against real-world evaluations; Interactive World Simulator [61] adds real-time human control.Source: §3.3.2 · p. 13

4 Architecture

• WAMs replace exogenous world models with internal predictive cores and organize architectures into Cascaded WAMs and Joint WAMs.Cascaded models separate world-state prediction from action decoding, whereas Joint WAMs train both under a unified objective to model their causal interdependencies.Source: §4 · pp. 13–14

4.1.1 Explicit Planning via Pixel-Space Representations

• Cascaded WAMs synthesize an intermediate visual plan before separate action decoding, dividing into explicit pixel-space and implicit latent-space planning.This decomposition separates scene prediction from robot-kinematic reasoning but couples both stages through the chosen planning carrier and decoder.Source: §4 · p. 14, §4.1.1 · p. 14
• Pixel-space cascaded methods extract actions through learned inverse dynamics or geometric computation, using video, optical flow, tracked poses, depth, and 3D motion representations.Learned approaches include hierarchical sub-action generation, keyframe interpolation, alternative action decoders, and contact-triggered policy switching; geometric approaches analytically derive commands from flow or tracked object and tool poses.Source: §4.1.1 · pp. 14, 15, 17

4.1.2 Implicit Planning via Latent Representations

• Implicit cascaded methods keep predicted dynamics in latent space, avoiding pixel decoding to improve efficiency while using latent features, semantic masks, tactile states, or latent actions for control.VPP achieved real-time-compatible planning inference; later methods use distillation, partial denoising, semantic bottlenecks, visuo-tactile prediction, or unsupervised latent-action pretraining to balance fidelity, robustness, and annotation cost.Source: §4.1.2 · pp. 17–18

4.2 Joint World-Action-Model

• Joint WAMs predict future states and actions within one model, using autoregressive token sequencing or diffusion-based parallel refinement as their main generation routes.Autoregressive models serialize heterogeneous variables or continuous latents, whereas diffusion models jointly generate world and action trajectories across multi-step horizons.Source: §4.2 · p. 18, §4.2.1 · p. 18, §4.2.2 · p. 20

4.2.1 Joint Prediction via Autoregressive Generation

• Autoregressive joint models trade unified causal conditioning for error propagation and sequential-decoding latency, spanning decoupled heads, shared discrete tokens, and predictive continuous latents.These designs use routing, masking, hybrid continuous action heads, or latent transition prediction to improve cross-modal grounding and avoid pixel-level reconstruction shortcuts.Source: §4.2.1 · pp. 18–20

4.2.2 Joint Prediction via Diffusion-based Generation

• Diffusion-based joint models avoid autoregressive bottlenecks through unified or multi-stream architectures that couple world and action prediction by shared trunks, cross-attention, hidden states, or shared representations.Unified-stream methods explicitly predict future observations or latents, while designs such as UWM independently control world and action noise to support policy, dynamics, inverse-dynamics, and video-generation modes.Source: §4.2.2 · pp. 20–21, Figure 6 · p. 21

Implicit Future Prediction

• Implicit Future Prediction supervises future-token representations against teacher-encoded observations, embedding world modeling inside the action backbone without generating future frames at inference.FLARE [95] also applies the alignment loss to action-free video, while FRAPPE composes multiple alignment experts through staged training over a frozen RDT backbone.Source: §4.2.2 · p. 22

Multi-Stream

• Multi-Stream architectures distribute world and action computation across coordinated branches, coupling them through cross-attention, hidden-state transfer, or shared representations.This taxonomy distinguishes persistent architectural decomposition from single-trunk joint prediction and organizes multi-engine designs by their dominant coupling interface.Source: §4.2.2 · pp. 22–23

Cross-Attention Coupling

• Cross-attention coupling uses separate Video DiT and Action DiT branches that exchange information explicitly, supporting structured world latents, asynchronous denoising, task switching, and multimodal experts.CoVAR [98], LDA-1B [99], and DUST [101] vary the coupling and noise schedules, while LingBot-VA, DexWorldModel [103], AIM [102], and Being-H0.7 [350] extend shared-attention coordination for memory, grounding, or efficient inference; Motus [19], MotuBrain, and AdaWorldPolicy [106] add semantic, language, or force experts.Source: §4.2.2 · pp. 23–24

Hidden-State Coupling

• Hidden-state coupling transfers temporally informative world-model features to the action branch, enabling efficient control conditioning and, in WAV, value-shaped latent-future planning.DiT4DiT [107] extracts intermediate video denoising states, Fast-WAM [104] removes the future-video branch at inference, and WAV ranks latent rollouts through an explicit value branch.Source: §4.2.2 · p. 24
• Act2Goal [108] uses goal-conditioned latent transitions and multi-scale temporal hashing to maintain dense near-term control while sparsifying distant predictions for long-horizon consistency.It additionally supports reward-free online improvement through HER-style goal relabeling and LoRA-based finetuning.Source: §4.2.2 · pp. 24–25

Shared Representation / Unified Encoder

• Shared-representation designs fuse observations and actions into common latent tokens before separate decoding, allowing direct action inference while supporting multiple masked modeling objectives.UVA decodes future video and actions from shared tokens and can operate as policy, video, forward-dynamics, inverse-dynamics, or combined policy–planner; PhysGen [22] uses unified causal physical tokens with causal action conditioning and parallel lookahead prediction.Source: §4.2.2 · pp. 25–26

5 Training data

• WAMs require physically grounded data for state transitions and action conditioning, combining aligned (o_t, a_t, o_t+1) triplets with scalable action-free videos.Unlike VLAs, which require costly paired (o_t, a_t) trajectories, pure world models lack physical-control grounding; WAMs can strategically mix both data types.Source: §5 · p. 26

5.1 Robot-Centric Teleoperation Data

• Robot-centric teleoperation records continuous observations, proprioception, and executable actions, providing high-frequency action-state pairs with virtually zero sim-to-real gap.These datasets remain central for learning precise action-conditioned physical dynamics while supporting both open-world scaling and deeper sensory grounding.Source: §5.1 · p. 27

5.1.1 Scaling Up: Embodiment, Diversity, and Automated Augmentation

• Early collections such as QT-Opt [112], MT-Opt [117], RoboNet [114], and MIME [113] scaled self-supervised trajectories and cross-robot coverage but remained laboratory-bound and short-horizon.Their limitations motivated subsequent efforts toward greater semantic diversity, sequential reasoning, and embodiment coverage.Source: §5.1.1 · p. 27
• Language-conditioned datasets including BridgeData [116], BC-Z, RT-1 [119], and LanguageTable [120] broaden household task diversity, while TACO-RL [352] and RH20T-P [127] target long-horizon hierarchical reasoning.Language instructions provide semantic anchors for generalizing visual state transitions, and extended tasks require predicting future-state sequences.Source: §5.1.1 · p. 27
• OXE [125] harmonized over 1 million trajectories across 22 robots, while ARIO [129], RoboMIND [128], DROID [126], and UnifoLM-WBT further expand platform, scene, and humanoid coverage.This morphological diversity encourages world models to separate universal physical laws from robot-specific kinematics, supporting embodiment-agnostic generalization.Source: §5.1.1 · p. 27
• Unstructured play data and automated generation—including Jaco Play, Grasp-Anything [355], Interleave-VLA [357], and UniVoxGen Dataset [358]—reduce manual collection bottlenecks and approach near-infinite training data.These mechanisms supply exploratory interaction priors, million-trajectory grasp supervision, image-text interleaving, and synthetic voxel-based spatial reasoning.Source: §5.1.1 · pp. 27–29

5.1.2 Deepening Perception: Multimodal and Contact-Rich Grounding

• Multimodal datasets add audio, 3D geometry, and tactile or force supervision to resolve occlusion and contact-rich dynamics beyond RGB appearance.Berkeley UR5, OmniAction [135], TOTO [354], Cable Routing Dataset [123], REASSEMBLE, RH20T [124], Robo360, RoboData [130], RoboSet [249], DexCap [131], FuSe [132], and AgiBot World [133] cover material identification, deformable manipulation, calibrated geometry, dexterity, and visuotactile interaction.Source: §5.1.2 · p. 29

Value for WAMs

• Portable datasets combine egocentric environmental diversity with high-frequency, centimeter-level action constraints, enabling robust action-conditioned state-transition learning beyond traditional robot datasets.This pairing bridges diverse human videos and precise robot demonstrations in unconstrained physical environments.Source: §5.2 · pp. 29–30

Portable demonstration interface

• UMI uses a handheld 3D-printed gripper and wearable cameras to collect in-the-wild human manipulation demonstrations, then retargets them into robot-executable actions.Vision-based tracking aligns non-expert demonstrations with robot actions.Source: §5.2 · p. 29

Data modality and task expansion

• FastUMI [138] and later extensions add scalability, active perception, tactile feedback, multi-view observations, dexterous manipulation, and whole-body mobile coordination.These developments provide multimodal, contact-rich, complex trajectories for modeling real-world physics.Source: §5.2 · p. 29

Dataset scaling

• FastUMI-100K exceeds 100K trajectories with multimodal text annotations, while RealOmin reaches million scale across over 3,000 household environments with proprioceptive, IMU, and tactile signals.These datasets mark the transition from proof-of-concept collections to large wild-collected corpora.Source: §5.2 · pp. 29–30

Physics-aware coverage

• Hoi! [141] targets cross-view, force-aware manipulation, while the dataset accompanying RDT2 provides roughly 10,000 hours across hundreds of real-world scenes.These resources support learning intuitive physics and contact dynamics.Source: §5.2 · p. 30

5.3 Simulation Data

• Simulation supplies scalable, deterministic data with privileged depth, poses, collision boundaries, and multi-view states for learning physical laws beyond trajectory imitation.Physics engines such as MuJoCo, Isaac Sim, and SAPIEN [360] provide exact computational world models, avoiding partial observability and sensory noise.Source: §5.3 · p. 30

5.3.1 Scaling Up: Procedural Generation and Environmental Complexity

• Procedural generation scales manipulation data from minimal demonstrations to millions of trajectories across diverse environments, embodiments, and long-horizon tasks.MimicGen [150] and DexMimicGen [154] expand demonstrations into tens of thousands of trajectories; InternData-A1 [160] exceeds 630k, SynGrasp-1B [156] reaches 10 million, and RoboCasa provides over 100k trajectories across 120 kitchen scenes.Source: §5.3.1 · p. 30

5.3.2 Spatiotemporal Dynamics: 3D and 4D Embodied Modeling

• Simulation enables dense 3D and temporal supervision for modeling how object geometries and scenes transform under robot actions.TesserAct [66] provides 285k aligned RGB, depth, and surface-normal video clips, while InternData-M1 [159] adds aligned 2D/3D annotations and RoboTwin 2.0 [157] uses photorealistic digital twins.Source: §5.3.2 · p. 31

5.3.3 Contact-Rich Physics: High-Fidelity Tactile Supervision

• Advancing contact simulation adds friction, deformation, tactile feedback, and force-interaction priors needed for contact-rich manipulation.The TLA Dataset [158] provides aligned tactile-language-action pairs for assembly, while domain randomization in SynGrasp-1B and RoboTwin 2.0 helps mitigate the sim-to-real gap and reduce downstream real-world data requirements.Source: §5.3.3 · p. 31

5.4 Human and Ego-Centric Data

• Human and egocentric datasets provide diverse real-world priors for generalist WAMs, complementing robot data limited by morphology, collection cost, and narrow physical exposure [362].They support passive world-dynamics learning and active action-conditioned modeling through human pose and proprioceptive grounding.Source: §5.4 · p. 32

5.4.1 Passive World Modeling: Action Semantics and Visual Dynamics

• Passive video datasets teach physical common sense, temporal state transitions, future forecasting, and open-vocabulary visual dynamics, scaling from SSv2’s 100,000 clips to Ego4D’s 3,600 hours and DreamDojo’s 44,000 hours.Ego4D adds episodic-memory and future-forecasting benchmarks, while HowTo100M and Kinetics-700 provide large-scale video-text alignment.Source: §5.4.1 · p. 32

5.4.2 Bridging the Action Gap: Pose Estimation and Proprioceptive Grounding

• Pose and motion annotations convert human videos into action-conditioned supervision, with datasets spanning 3D hand-object geometry, synchronized viewpoints, spatial anticipation, gaze, and full-body motion.Ego-Exo4D contains 1,286 hours of synchronized egocentric and exocentric video, while ARCTIC, HOT3D, and TACO provide millimeter-accurate 3D hand-object meshes.Source: §5.4.2 · pp. 32–33

5.4.3 Toward Generalist Pretraining Mixtures

• Proprioception-grounded mixtures aggregate human demonstrations at scale and align visual observations with hand or humanoid kinematics, enabling transfer into robotic action spaces.The Ego-Centric Human Manipulation Dataset and UniHand pool over 130 million frames, while EgoDex provides 829 hours across 194 tasks with high-fidelity 3D finger tracking.Source: §5.4.3 · pp. 33–34

• No established protocol jointly evaluates future-state fidelity, generated-action effectiveness, and causal alignment between them.Source: §6 · p. 34
• Existing work follows a decoupled paradigm using separate module-specific metrics for world modeling and action policy capabilities.Source: §6 · p. 34
• World modeling capability assesses the physical integrity and structural consistency of synthesized transitions, whereas action policy capability measures task-fulfillment effectiveness.Source: §6 · p. 34

6.1 How to Evaluate World Modeling Capability?

• World modeling capability is organized into three complementary axes: Visual Fidelity, Physical Commonsense, and Action Plausibility, spanning appearance, environmental dynamics, and executable control information.Unlike conventional video generation, evaluation must capture underlying dynamics and preserve actionable information.Source: §6.1 · p. 34, §6.1.3 · p. 37

6.1.1 Visual Fidelity

• Visual fidelity combines PSNR/SSIM for low-level reconstruction, LPIPS/DreamSim/DINO for perceptual and semantic consistency, and FVD for distribution-level realism and temporal quality [194] [195] [196] [197].These metrics address artifacts, temporal inconsistency, condition following, object identity, scene semantics, and dataset-level video realism.Source: §6.1.1 · pp. 34–35

6.1.2 Physical Commonsense

• Physical commonsense evaluation covers object dynamics and trajectory plausibility, including object continuity, physically grounded interactions, causal event evolution, physical-law adherence, anomaly detection, and future-event prediction [199] [200] [201].Measures include key physical phenomena detection, physics-order verification, naturalness, binary physical-law checks, abnormal-entity detection, and motion- and fidelity-based evaluation.Source: §6.1.2 · pp. 35–36

6.1.3 Action Plausibility

• Many visually convincing models achieve nearly zero real-world execution success under the IDM Turing Test, exposing action plausibility as distinct from visual and physical realism [206].The test infers actions from generated videos and evaluates them through real-world execution success.Source: §6.1.3 · p. 37

6.2 How to evaluate Action Policy?

• Over 40 benchmarks are organized into general, bimanual and humanoid, mobile, contact-rich and deformable, and real-robot manipulation categories.The review covers simulation setup, sensor modalities, demonstration scale, and multidimensional evaluation metrics.Source: §6.2 · p. 38, Table 9 · p. 40

6.2.1 General Manipulation Benchmarks

• General-manipulation benchmarks progressed from standardized multitask suites to evaluations of data scaling, robustness, task-level generalization, language, long horizons, sim-to-real transfer, and memory.MetaWorld [207] and RLBench [208] introduced 50 and 100 tasks, while LIBERO [216] spans 130 tasks and four evaluation dimensions; ManiSkill3 [212], RoboCasa, and RoboVerse [222] emphasize scale and data-performance relationships.Source: §6.2.1 · pp. 38–39

6.2.2 Bimanual and Humanoid Form Benchmarks

• Bimanual and humanoid benchmarks target higher-dimensional coordinated planning through long-horizon, whole-body, dexterous, locomotion, tactile, generalization, and data-scale evaluations.RoboTwin [153], BiGym [232], HumanoidBench [233], and HumanoidGen [234] use platforms including Aloha-AgileX and Unitree H1, with HumanoidBench providing 448 tactile sensing points and 27 tasks.Source: §6.2.2 · p. 39

6.2.3 Mobile Manipulation Benchmarks

• Mobile-manipulation benchmarks jointly evaluate navigation, perception, planning, and manipulation across scenes, ranging from 28 tasks in ManipulaTHOR [235] to 1,000 daily-living tasks in BEHAVIOR-1K [237].HomeRobot [236] evaluates open-vocabulary navigation and grasping in both simulation and real robots, while BEHAVIOR-1K [237] includes rigid bodies, deformable objects, and liquids.Source: §6.2.3 · p. 39

6.2.4 Contact and Deformation Manipulation Benchmarks

• Contact-rich and deformable benchmarks evaluate macroscopic deformation and contact-state feedback, extending purely visual control with tactile sensing and vision-tactile fusion.SoftGym [238], PlasticineLab [239], and DaXBench [240] cover cloth, liquids, plasticine, and other deformable materials, while TacSL [241] and ManiFeel [242] study tactile-driven manipulation and sim-to-real transfer.Source: §6.2.4 · pp. 39, 41

6.2.5 Real-Robot Benchmarks

• Real-robot benchmarks address simulation’s deployment gap by evaluating policies directly on physical platforms, including generalization, long-horizon, and multitask performance.RoboChallenge [244] covers 30 scenarios across 4 robot platforms, while Maniparena [245] collects 10,812 real-robot trajectories on two platforms.Source: §6.2.5 · p. 41

Architectural Coupling

• Architectural coupling lacks controlled comparisons, leaving the necessity of explicit visual prediction and the inductive biases of cascaded versus joint designs unresolved.The field needs matched-scale, matched-data ablations and theoretical analysis to replace architecture-driven fashion with principled choices.Source: §7 · p. 41

Multimodal Physical State Representation

• RGB-only world models miss tactile, force, acoustic, and compliance information critical to contact-rich manipulation, motivating multimodal and modality-adaptive state prediction.Future systems should exploit rich sensor streams when available while degrading gracefully to visual-only inference.Source: §7 · p. 42

Data Utilization and Mixture Design

• Data-mixture principles remain unclear, including each source’s marginal value, human-video benefits, curriculum design, and how to distill transferable physics while suppressing embodiment-incompatible behaviors.Human video may provide low-level physical priors, mid-level causal dynamics, and high-level task logic, requiring embodiment-aware filtering.Source: §7 · p. 42

Long-Horizon Planning and Temporal Abstraction

• Long-horizon WAM reasoning is hindered by prediction drift, compounding action errors, and costly continuous trajectory generation, requiring hierarchical world-action modeling.Candidate pathways include planner-guided modular hierarchies, intrinsic multi-resolution prediction, and longer temporal memory without quadratic attention overhead.Source: §7 · pp. 42–43

Inference Latency and Computational Efficiency

• Inference latency threatens closed-loop control: DreamZero reaches 7Hz, but this remains below the 50Hz standard of contemporary non-generative VLA policies.Task-adaptive predictive fidelity could allocate computation according to task demands and error tolerance rather than pursuing uniformly high-fidelity synthesis.Source: §7 · p. 43

Evaluation Methodology

• Current evaluation separates visual plausibility from task success, motivating joint metrics such as Counterfactual Consistency and Foresight-Conditioned Success for causal prediction-action grounding.A shared benchmark should assess world prediction and action quality together rather than maintaining separate leaderboards.Source: §7 · p. 43

Safety and Reliable Physical Deployment

• WAM safety requires verifying imagined futures before execution because confident predictive errors can produce extended, difficult-to-recover actions with real-world consequences.Prediction-integrated safety should use uncertainty and physical constraints while remaining computationally tractable and robust to distributional shift.Source: §7 · pp. 43–44

Conceptual framework and taxonomy

• The survey formally defines WAMs, distinguishes them from related methodologies, and organizes their architectures into Cascaded and Joint paradigms.It further examines generation modalities, conditioning mechanisms, and action decoding strategies within these paradigms.Source: §8 · p. 44

Data and evaluation

• The survey synthesizes scalable training datasets and multidimensional evaluation protocols covering visual fidelity, physical commonsense, and action plausibility.Source: §8 · p. 44

Open challenges and future directions

• The survey identifies critical open challenges and future trajectories as generative world modeling and robotics continue to converge toward generalist embodied agents.Source: §8 · p. 44