PaperlayerView exact revision on arXiv

Source-linked AI summary

World Action Models: A Survey

Qiuhong Shen, Shihua Zhang, Yue Liao, Qi Li, Zhenxiong Tan, Shizun Wang, Shuicheng Yan, Xinchao Wang

arXiv:2606.20781v1cs.ROcs.CV

TL;DR

World Action Models span inconsistent terminology and architectures, complicating comparison across predictive-action methods. This survey defines common boundaries and organizes the field, finding a shift toward generating less of the future while retaining what control requires.

  • Problem

    WAMs span different communities and vocabularies, so shared labels do not provide a sufficient basis for comparing methods.

  • Method

    The survey defines WAMs as predictive-action models and organizes them by generated future, predictive substrate, backbone, action coupling, and deployment regime.

  • Results

    Across surveyed works, WAM design is moving toward selective imagination that preserves future information needed for action while discarding the rest.

  • Takeaways & Limitations

    WAM design choices trade representational richness against compute, memory, latency, and action-label cost in control loops.

  • Takeaways & Limitations

    Render-and-Decode methods incur the cost of full pixel-generation schedules, while video-quality metrics weakly predict downstream task success.

Abstract

from arXiv · show

World Action Models (WAMs) are embodied predictive-action models that make a forecast of the future available to action. Recent WAMs repurpose large video generation models, and a parallel line relies on language or vision-language backbones without a video-generation core. This rapid expansion has blurred the boundary among broad world models, video generation models, action-grounded video world models, Vision-Language-Action policies, and WAMs. This survey gives the field a common account. It first clarifies these boundaries, then organizes existing works through two complementary views. The first view asks what each method is required to generate, spanning rendered futures, latent futures, and video-generation-free action reasoning. The second view decomposes each method by predictive substrate, backbone, action coupling, and deployment regime. This anatomy supports a unified discussion of interactability, causality, persistence, physical plausibility, and generalization, followed by data, evaluation, and open challenges. Across these axes, a consistent design pattern emerges: WAMs are not simply video generators with action heads, but predictive-action methods whose design choices trade representational richness against compute, memory, latency, and action-label cost. The field is moving toward methods that generate less of the future while preserving what control requires. The survey homepage is available at https://world-action-models.github.io/.

1 Introduction

The survey clarifies World Action Models through a taxonomy of what they generate and a four-axis anatomy, then evaluates the properties embodiment requires. It argues that WAM design increasingly generates less of the future while balancing control-relevant capability against compute, memory, and latency.

  • Taxonomy: The survey places WAMs into Render-and-Decode, Latent-Only, or Video-Generation-Free families according to where action is decoded along the inference path.Render-and-Decode carries generation to pixels, whereas Latent-Only stops at intermediate representations.
  • Core properties: Embodied deployment requires WAMs to be interactable, causal, persistent, physically plausible, and capable of generalization.The survey examines how existing methods address each of these five properties.
  • Survey scope: The survey covers WAM distinctions, design philosophies, formal anatomy, core properties, data and evaluation, and open challenges across the field.Its data discussion spans teleoperation, portable human demonstrations, simulation, and internet-scale egocentric video.
  • Anatomy: WAMs are organized by predictive substrate, architectural backbone, action coupling, and deployment regime, recording each method as a 4-tuple.These four axes provide a common anatomy for comparing methods in Tables 1 and 2.
  • Design trade-offs: Every WAM design choice affects compute, memory, and latency inside a control loop, so the field is moving toward generating less of the future while preserving control requirements.The survey frames the key issue as trading these costs against the properties demanded by embodiment.

2 The Emergence of World Action Models

World Action Models emerge by connecting future-observation prediction to action production, selection, or training, distinguishing them from direct VLAs and standalone world models. They span cascaded, consequence-scoring, and jointly trained future-action designs.

  • Boundary from VLAs: VLAs bind vision, language, and action but do not require predicting post-action observations; a model becomes a WAM when a predicted future helps produce, choose, or check the action.RT-2, OpenVLA, and the π series exemplify influential VLA predecessors.
  • Boundary from world models: A world model predicts a future observation under an intervention, which may be pixels, hidden features, geometric state, affordances, audio, symbols, or tokens.These predictive representations support policy learning, planning, simulation, evaluation, and data generation.
  • Cascaded WAMs: Historically early cascaded WAMs generated or scored future visual trajectories before recovering executable actions with inverse dynamics, pose tracking, optimization, planning, or a separate policy.UniPi, VLP, and AVDC exemplify this design.
  • Joint WAMs: Joint designs predict future observations and actions together, using autoregressive, diffusion, or hybrid backbones, while feature-tap methods show that the future branch need not be rendered video.GR-1 and GR-2 helped establish the shared video-action direction; later examples include PAD, UWM, WorldVLA, DreamZero, and AIM.
  • WAM definition: WAMs connect predicted futures to actions through predict-then-act cascades, action-consequence scoring, or joint future-action prediction within one model.This definition distinguishes a WAM from a VLA modeling p(a | o, l) or a world model modeling p(o′ | o, a, l) or p(o′ | o, l).

3 Three Design Philosophies of World Action Models

The survey identifies three mutually exclusive WAM design philosophies based on where action prediction is grounded along the inference path: rendered pixels, intermediate video-derived representations, or non-video-generation representations. This taxonomy is separable from action coupling and backbone choices, and reflects a shift toward bypassing pixel rendering to reduce control-time costs.

  • Three design philosophies: Latent-Only preserves a video-derived predictive prior but grounds action prediction in an intermediate latent or feature before pixel decoding.Bypassing the renderer can enable real-time inference for several methods, at the cost of reduced direct visual interpretability.
  • Taxonomy and chronology: Chronologically, WAMs move from direct video-to-control paths toward latent and video-generation-free approaches that place action prediction earlier or outside the video-generation path.This progression preserves useful predictive structure while reducing the latency and memory burden of full pixel rendering.
  • Taxonomy and chronology: The philosophy taxonomy is exhaustive and independent of whether prediction and action are coupled jointly or arranged as a cascade.It classifies methods by the last future representation required before action decoding, rather than by training pipeline alone.
  • Three design philosophies: Video-Generation-Free replaces video generation with predictive supervision in embedding, token, feature, audio-latent, affordance, or geometry spaces.Representative methods include teacher-target or foundation-feature prediction, latent transition models, and token-based predictive representations.
  • Three design philosophies: Render-and-Decode generates a rendered visual future before decoding actions, exposing appearance, motion, contact, and scene dynamics to the actor.Its defining limitation is the computational price of producing pixels at every prediction step, although the rendered future remains directly inspectable.

4 What Makes a World Action Model

This section replaces paper-by-paper enumeration with a common mathematical account of World Action Models, which vary across communities, vocabularies, and backbones. It then organizes methods by four ingredients and the practical trade-offs associated with their choices.

  • Motivation: World Action Models span communities, vocabularies, and backbones, so shared labels do not reliably indicate shared implementation details.The section notes that closely related methods may use different terminology, while methods sharing the WAM label can differ in nearly every implementation detail.
  • Motivation: The section treats World Action Models as instances of a common mathematical object rather than comparing them through paper-by-paper enumeration.This provides the section’s alternative route for comparing methods across heterogeneous terminology and implementations.

4.1 A Unified Notation for World Action Models

World Action Models share an abstract action-facing contract: a parameterized conditional joint distribution over future predictions and future actions. Their practical identity is determined by four interacting choices—predictive substrate, action coupling, architectural backbone, and deployment regime—that specify what is predicted, how it is produced, and how it is used in control.

  • Unified WAM object: A WAM models a conditional joint distribution over a future trajectory in substrate space S and a same-length future action chunk.The conditioning context includes observation history, past action history, and an instruction or auxiliary modality; the prediction horizon is H.
  • Predictive substrate: The predictive substrate is the action-facing representation exposed for action production or evaluation, including decoded frames, latents, learned features, geometric primitives, or affordance maps.The interface encoder defines the exposed substrate space, but need not correspond to the generator’s internal implementation.
  • Four-axis decomposition: These four axes separate what future variable is represented, how it is predicted, how actions enter or are recovered, and how the model operates inside a control loop.They are conceptually distinct but interact in practice, and the survey represents each WAM as a 4-tuple over these choices.
  • Architectural backbone: The architectural backbone realizes future prediction through iterative denoising, autoregressive decoding, joint-embedding prediction, hybrids, or language and vision-language models with action decoders.Each backbone family has its own parameterization for the future-prediction factor.
  • Action coupling: Action coupling determines how the future substrate and action chunk are related through action-conditioned rollout, post-prediction recovery, or joint generation.Action-conditioned rollout may fix an action chunk before prediction or update actions after each predicted step, whereas joint generation produces both from one coupled model.
  • Deployment regime: Deployment regimes specify when the WAM runs and over what window, ranging from open-loop rollout to chunked closed-loop, single-step closed-loop, and interactive operation.Open-loop uses H ≈T once, chunked closed-loop uses H =K every K control steps, and single-step closed-loop uses H =1 at every control step.

4.2 Predictive Substrate: Where WAMs Dream

WAMs are organized by the space in which their predicted future lives, called the predictive substrate, rather than by backbone or action-head tensor. The survey distinguishes pixel-grounded, feature, geometric, and task-map substrates, each trading predictive content and supervision against control needs.

  • Predictive substrate: The predictive substrate is the future variable’s space, and decoder-bound video latents remain pixel-grounded even when control skips pixel decoding.A fixed decoder mapping the latent back to video preserves an observation-coordinate interpretation; intermediate policy states without such a decoder are instead feature substrates.
  • Pixel-grounded substrates: Pixel-grounded substrates represent future observations as decoded RGB, RGB-D, multi-view video, or fixed-decoder VAE/VQ latents.This category includes direct video futures, pixel-decodable latent trajectories, sparse keyframes, and decoder-bound acoustic observation latents.
  • Feature substrates: Feature substrates predict learned states, teacher embeddings, or VLM tokens without a fixed observation decoder, routing those futures through action pathways or embedding losses.Examples include hidden future-shaped states, frozen-encoder targets, and visual chain-of-thought or next-state tokens used as future carriers rather than action discretization.
  • Geometric substrates: Geometric substrates predict physical-coordinate futures such as optical flow, point tracks, depth, pose, waypoints, or motion-vector grids.Their smaller structured tensors can train stably with less data, but they are most reliable when motion, contact, or geometry matters and less suitable when appearance or semantics is essential.
  • Task-map substrates: Task-map substrates expose spatial value, affordance, contact-likelihood, segmentation, or progress information as a task-specific answer rather than an observation, feature, or physical coordinate field.The substrate contains C channels carrying task-relevant labels or score-map information.

4.3 Action Coupling: How Action Enters and Leaves

Action coupling defines how actions interact with predictive substrates, yielding action-conditioned rollout, joint generation, and post-prediction heads. These families trade counterfactual controllability, coupled prediction, and inference efficiency through different placements of the action representation.

  • Action-conditioned rollout: Action-conditioned rollout supplies candidate actions to a predictor that evaluates their consequences before selection.Chunk-level rollout evaluates many candidates in parallel, whereas step-wise rollout can react after each predicted state but loses that parallelism.
  • Joint generation: Joint generation produces substrate and action from one shared generative process, coupling video and action through diffusion, autoregression, or hybrid experts.Its benefit is a coupled sample with mutual consistency, while separate generation and action losses can destabilize shared training.
  • Post-prediction heads: Post-prediction heads decode actions from a generated or predicted substrate, often keeping the pretrained predictor frozen while adapting a smaller action expert across embodiments.Video-plan then action-recovery systems synthesize future image trajectories before recovering executable controls with inverse-dynamics or policy modules.
  • Post-prediction heads: Optional-future methods train with a substrate generator but shorten or bypass it at inference when an internal state supports the action path.VidMan mounts a policy head on frozen video-diffusion features rather than waiting for a rendered future.

4.4 Architectural Backbone: How the Prediction Is Produced

This section organizes WAM architectural backbones into five function families and separates backbone parameterization from action-coupling regime. Diffusion and autoregressive designs support distinct coupling patterns, while hybrid designs trade coupled consistency or counterfactual scoring against runtime cost.

  • WAM backbones are grouped into five families, each defined by its parameterization, canonical training objective, and representative models.
  • Diffusion: Diffusion backbones generate futures through iterative reverse denoising, commonly using space-time video diffusion transformers or adapted image-diffusion models.Flow matching changes the score parameterization but remains within the iterative reverse-chain diffusion family.
  • Diffusion: Diffusion WAMs use action-conditioned rollout, joint generation, or post-prediction heads, with joint generation denoising future substrates and actions together.Post-prediction heads may consume decoded observations, pixel-latent or geometric substrates, or feature-level latent states.
  • Autoregressive: Autoregression is a backbone parameterization that serializes whichever generative factor is selected, rather than inherently implying joint action prediction.Its stream may generate frames, latent blocks, visual or action tokens, or compact latent states; future actions can be supplied, generated in-stream, or decoded later.
  • Hybrid: Hybrid backbones maintain separate world-prediction and action paths, gaining coupling-dependent consistency, cheap post-head action expertise, or counterfactual scoring at added runtime cost.Repeated candidate evaluation can dominate the cost of action-conditioned rollout.

4.5 Deployment Regime: Interactive, Rollout, Open Loop, Closed Loop

Deployment regime is defined by the prediction horizon and invocation cadence relative to the control loop, spanning open-loop rollout, chunked and single-step closed-loop control, and truly interactive simulation. These regimes trade backbone amortization and latency against replanning frequency, chunk staleness, and persistent-state computation.

  • Regime Definition: Deployment regimes allocate forward-pass cost across task length T according to horizon H and invocation cadence relative to control frequency fctrl.The survey distinguishes regimes by how often new observations can revise the generated future.
  • Open-Loop Rollout: Open-loop rollout sets H ≈T and invokes the WAM once, generating a long future substrate before execution for a separate actor to consume.Its WAM compute is paid once per scenario and does not scale with fctrl; listed examples include UniPi, AVDC, Dreamitate, This&That, ARDuP, Gen2Act, RoboEnvision, LVP, and MVISTA-4D.
  • Chunked Closed Loop: Chunked closed-loop control sets H =K, refreshes every K control steps, and amortizes a large backbone across multiple ticks when Nfwd(K)/K < 1/fctrl.Its main limitation is staleness when the world changes during chunk execution, while adaptive variants can trigger earlier replanning when predicted and observed latents diverge.
  • Single-Step Closed Loop: Single-step closed-loop control sets H = 1 and invokes the WAM every control step, subject to the hard constraint Nfwd(1) < 1/fctrl.The regime produces one substrate prediction and one action per call; cited examples include WorldVLA, VidMan, GR-1, PAD, VPP, DreamZero, PhysGen, and OmniVTA.
  • Interactive Simulation: Truly interactive simulation continuously incorporates user or actor inputs without a fixed endpoint, invoking the WAM each step while reusing history through a key-value cache or persistent latent.For attention-based backbones, cached per-step computation is linear in M(t), while cumulative cost is quadratic in t.

4.6 Putting the Four Ingredients Together

Sections 4.1–4.5 define each World Action Model with a four-coordinate tuple covering substrate, backbone, action coupling, and deployment. The census uses this tuple to organize existing methods and highlights a design asymmetry between training-time backbone and substrate choices and more inference-flexible coupling and deployment choices.

  • Four-axis anatomy: A WAM is characterized by a compact 4-tuple whose coordinates specify predictive substrate, backbone, action coupling, and deployment regime.The substrate spans pixel-grounded, feature, geometric, and affordance representations, with narrower variants recorded where relevant.
  • Four-axis anatomy: The tuple places diverse methods in a common design space, including F1, WorldVLA, FLARE, and UWM with distinct substrate, coupling, backbone, and deployment combinations.For example, F1 and UWM use pixel-decodable latent grids with joint generation, hybrid coupling, and chunked deployment, whereas WorldVLA uses autoregressive coupling and single deployment.
  • WAM construction: Video-generation models such as Wan and CogVideoX are not WAMs alone because they specify only the predictive substrate and backbone; adding action coupling and deployment turns them into WAMs.This wrapper-around-a-frozen-backbone pattern appears in CosmosPolicy, VidMan, and VideoVLA.
  • Design direction: Substrate and backbone receive substantial training-time attention, while action coupling and deployment remain more inference-flexible coordinates where many recent mechanisms appear.Choosing a pretrained backbone is often the expensive design decision.
  • Census organization: The census places every WAM on the tuple, separating Render-and-Decode and Latent-Only methods from Video-Generation-Free methods and ordering rows by arXiv first-submission date.Tables 1 and 2 cover the first two philosophies, while Table 2 covers methods without a video-generation backbone in the predictive path.

5 Core Properties of World Action Models

World Action Models must remain interactable, causal, persistent, physically useful, and transferable when deployed in control loops. These properties create competing pressures, favoring compact embodiment-grounded predictions over futures optimized mainly for visual quality.

  • Interactability: Interactability depends on where action enters prediction: post-prediction interfaces are modular but pay full generation cost, whereas earlier binding more directly shapes the future.The progression runs from extracting control after generation, through coupled branches and shared backbones, to latent actions injected at every denoising step.
  • Causality: Causality requires predictions based only on past observations and chosen actions, enforced through causal streams, caching, and leakage-free attention masks.PhysGen predicts multiple future action tokens while executing only the leading token; chunked designs forbid action tokens from attending to future pixels or later actions.
  • Persistence: Persistence is threatened by compounding drift, growing full-history memory costs, and forgetting scene identity when finite context removes earlier observations.These failures emerge during repeated action, observation, and replanning as rollouts leave the data manifold or lose track of occluded objects.
  • Physical usefulness: Action usefulness depends more on whether an imagined future constrains control than on how realistically it looks, favoring coarse embodiment-grounded representations.Flow, masks, tactile and force signals, and proprioceptively consistent latents can be more action-relevant than pixels, especially around contact.
  • Generalization: Generalization requires predicting at the invariant level that still constrains control, because pixels, flow, masks, and latent forecasts transfer different priors and impose different grounding costs.No design dominates across all shifts; claims of generalization should specify the target shift, transferring design axis, and task-specific adapter.
  • Cross-property trade-offs: The five properties impose competing pressures rather than independent checks: stronger action interfaces constrain causality, bounded memory limits plausibility, and transferable abstractions complicate evaluation.A WAM therefore cannot be tuned for one property alone.

6 Data and Evaluation

WAM data and evaluation are coupled: training sources determine learnable futures, trusted action labels, and embodiment transfer, while evaluation must test both prediction quality and usefulness in control loops. The field therefore balances scale, label fidelity, physical grounding, access, realism, closed-loop success, plausibility, coherence, and deployment cost.

  • Data: WAM training data falls into five groups, each trading off scale, action-label fidelity, physical grounding, and access.The groups are robot teleoperation, portable human demonstrations, internet-scale egocentric and instructional video, simulation, and synthetic data from WAMs themselves.
  • Data: Teleoperation provides the cleanest action-conditioned trajectories, while portable demonstrations and internet video increase collection throughput and visual diversity but usually weaken direct robot-action supervision.Teleoperation records commands with visual observations; portable capture uses human-worn or handheld setups, and internet video usually lacks robot actions.
  • Data: Simulation supplies exact action labels, controlled curricula, and low marginal cost after authoring, while WAM-generated synthetic trajectories can fill gaps that are expensive to collect physically.In practice, mixtures combine internet video for visual and physical priors, teleoperation for trusted labels, portable data for scale, simulation for coverage, and synthetic trajectories for costly gaps.
  • Evaluation: WAM evaluation must combine visual prediction metrics with closed-loop task performance, because plausible video alone does not establish action utility.Visual metrics reward realism, whereas closed-loop benchmarks measure whether policies complete tasks; neither perspective is sufficient alone.
  • Evaluation: Current evaluation is cheapest for visual fidelity, more predictive but costlier in closed-loop simulation, and physically valid yet expensive on real robots, while plausibility and long-horizon metrics remain underdeveloped.Protocols should also assess physical realizability, long-horizon coherence, compute, memory, and latency.

7 Open Challenges · 7.1 Dream More or Act More? · 7.2 What Data Should Each Stage Learn From?

The open challenges separate how much future computation WAMs should spend at runtime from which data sources should supervise each training stage. Across both questions, the field seeks controllable computation budgets and stage-specific data assignments that preserve action-relevant prediction and grounding.

  • 7 Open Challenges: The open-challenge framework distinguishes future computation, training data, memory, generalization, action grounding, physical plausibility, and evaluation as related but distinct questions.The boundaries are analytical rather than absolute because data, action abstraction, and inference cost affect multiple design choices.
  • 7.1 Dream More or Act More?: WAMs face a fidelity-latency tradeoff: richer futures and larger backbones improve action-side substrates but lengthen the path from observation to control.S-VAM distills video diffusion into geometric and semantic foresight features, while Fast-WAM removes the future-video branch at inference.
  • 7.1 Dream More or Act More?: The central design goal is a controllable fidelity-latency curve that lets WAMs choose how much imagination to retain in the control path.DreamZero, CosmosPolicy, and NovaPlan keep stronger imagination while paying for it through optimization, repeated queries, or video generation before grounding.
  • 7.1 Dream More or Act More?: Runtime budgeting should adapt regeneration, replanning, horizon, and search depth to task demands rather than applying fixed schedules or binary execute-or-replan decisions.Routine transit and contact-sensitive manipulation may require different forecast budgets, but current coarse switches cannot determine how much remaining horizon to regenerate.
  • 7.1 Dream More or Act More?: Video world models and WAMs increasingly converge at the predictive-substrate level, while WAMs seek only the future information needed for action instead of full detailed futures.Video generators add action conditioning, and WAMs borrow latent or joint-embedding substrates from video backbones.
  • 7.2 What Data Should Each Stage Learn From?: Training should assign each data source to the stage, objective, and model component it supervises instead of pooling sources with different evidence.Internet video supports visual regularities, human demonstrations align motion with intent, teleoperation grounds action decoding, and rollouts can train gates or budget controllers.
  • 7.2 What Data Should Each Stage Learn From?: Video-pretraining benefits are domain- and split-dependent: robot-domain data can help, mismatched egocentric data can hurt, and human-video gains vary by evaluation split.GR-2, DreamDojo, and VPP illustrate broad video pretraining before robot-specific adaptation, while VidMan and VLA-JEPA expose transfer limitations.
  • 7.2 What Data Should Each Stage Learn From?: Alignment data remains unsettled because egocentric, paired human-robot, and wearable formats introduce different biases, while scene diversity may matter more than raw volume.EgoScale separates broad human pretraining from aligned mid-training, EgoMimic uses first-person demonstrations, and EgoVerse emphasizes scene diversity.

7.3 Can Memory Keep Up?

Useful WAM memory must preserve task state during action production while scaling with scene complexity rather than episode length. Existing approaches offer bounded recall and compact latent state, but combining bounded memory, spatial indexing, and observation replacement remains open.

  • Memory requirements: WAM memory should preserve task state during action production, with cost scaling by scene complexity rather than episode length.Autoregressive prediction exposes temporal drift as a direct failure mode, including in IRASim despite overlapping context.
  • Promising ingredients: Promising mechanisms include bounded per-step recall, compact latent state for continual imagination, spatial indexing, and observation replacement.Test-time memory risks forgetting earlier state, while Dreamer 4 leaves object-level spatial detail unresolved across long manipulation episodes.

7.4 How Can WAMs Generalize? · 7.5 What Grounds Abstract Action?

WAM generalization depends on matching data, predictive substrate, and adaptation to the specific shift, rather than scaling alone. Abstract action representations enable learning from weakly labeled video but can lose physical grounding through estimator dependence, coarse codes, or embodiment-specific decoding.

  • 7.4 How Can WAMs Generalize?: Generalization must target distinct shifts in appearance, dynamics, morphology, action space, camera placement, or contact regime rather than treating it as one scalar property.The passage states that these shifts stress different parts of a WAM and that scale alone is insufficient.
  • 7.4 How Can WAMs Generalize?: Harder randomized conditions sharply reduce Act2Goal performance, while FRAPPE remains far below a practical rate even when leading the compared baselines.These results show that randomization-based robustness remains limited under more difficult conditions.
  • 7.4 How Can WAMs Generalize?: Video-conditioned policies improve transfer to new motions and objects, but the hardest splits retain a large gap, showing that prediction helps only when its substrate matches the required action shift.The passage frames substrate–shift alignment as the condition under which future prediction supports transfer.
  • 7.4 How Can WAMs Generalize?: Cross-embodiment transfer requires matching data, substrate, and adaptation to changing action spaces; DreamZero transfers from short video-only adaptation sets, but morphology shift remains unsolved by scale alone.Removing video pretraining sharply reduces task completion, indicating that the borrowed video prior matters.
  • 7.5 What Grounds Abstract Action?: Latent codes, flow fields, point tracks, and structured transitions replace raw control with abstract targets that support learning from action-free or weakly labeled video.The substitution is useful because it reduces reliance on direct action labels.
  • 7.5 What Grounds Abstract Action?: Abstract action can lose a physical handle: two-dimensional object flow cannot represent depth translation, out-of-plane rotation, or contact force.This is identified as an estimator-dependence failure mode.
  • 7.5 What Grounds Abstract Action?: LAPA improves action-free pretraining but struggles with fine grasping under coarse codebooks, while ALAM structures latent transitions without explaining force, torque, or contact.Both examples show that predictive structure need not provide a physical explanation.
  • 7.5 What Grounds Abstract Action?: LDA-1B learns a broad latent action space, yet execution still depends on a decoder mapping that space back to an embodiment-specific implementation.The passage identifies embodiment-specific decoding as a remaining source of opacity.

7.6 When Is a Future Physical? · 7.7 What Should Evaluation Report? · The challenges are coupled

Physical futures must be judged by embodiment and executability, not visual plausibility alone, while evaluation must report accuracy under matched latency and horizon budgets. These demands are coupled because richer physical grounding, memory, and action abstractions trade off against compute, evaluation cost, and physical fidelity.

  • 7.6 When Is a Future Physical?: Visual plausibility can hide embodiment failures, including momentum errors when a one-frame-conditioned flow model cannot encode object velocity.Push tasks fail when objects continue moving after contact ends.
  • 7.6 When Is a Future Physical?: Physical plausibility spans visual geometry, tactile and force prediction, and proprioceptive or kinematic coherence.RoboScape, RoboScape-R, OmniVTA, DexWM, and AdaWorldPolicy incorporate geometric, consistency, tactile, dexterous, or action-conditioned structure, but remain partial.
  • 7.7 What Should Evaluation Report?: Evaluation should compare visual screens, simulator tests, and closed-loop robot evaluations using variables that make the tiers comparable.MotuBrain reports only a weak correlation between world-model score and downstream success.
  • 7.7 What Should Evaluation Report?: A single success number can conceal regime differences, as schedules useful for transit may fail at precise interaction.HarmoWAM illustrates why evaluation must distinguish task regimes rather than rely on aggregate success alone.
  • 7.7 What Should Evaluation Report?: Reports should pair accuracy with compute budget, control-rate latency, and task horizon because slow or short-horizon success is not equivalent to sustained real-time performance.Latency is often reported separately from accuracy and under different hardware or task settings; Fast-WAM provides an emerging example of isolating test-time video generation.
  • The challenges are coupled: The challenges interact: deeper memory search raises compute, richer substrates raise evaluation and inference cost, and latent actions reduce labels while weakening physical grounding without calibrated decoding.Contact-aware prediction also requires force, tactile, or failure data that cheap visual screens do not provide.

8 Conclusion

The survey defines World Action Models as a contract between prediction and control: they anticipate actionable futures from context while producing the next action. It organizes the field through generation requirements and model anatomy, then evaluates the properties and benchmarks relevant to deployment in control loops.

  • Conclusion: WAMs jointly anticipate the actionable future and produce the action that should follow from past observations, past actions, and task context.This contract distinguishes WAMs from broad world models, video generation models, and plain VLAs.
  • Conclusion: The survey uses complementary views that classify methods by required generation and by predictive substrate, backbone, action coupling, and deployment regime.The first is a philosophy-level view; the second is a component-level anatomy.
  • Conclusion: These views foreground interactability, causality, persistence, physical plausibility, and generalization as core properties once models enter control loops.They also establish that data and evaluation matter because benchmarks validate only the properties they expose.
Loading 2606.20781v1…