World-Value-Action Model: Implicit Planning for Vision-Language-Action Systems

• Direct action prediction in most VLA approaches lacks mechanisms for reasoning over long-horizon trajectories and evaluating their outcomes, limiting coherent multi-step decision making.Source: §1 · p. 2
• World models predict future observations but typically lack principled trajectory evaluation, while supervised unified architectures and simulation-based approaches face limitations in long-horizon quality or out-of-distribution reliability.Source: §1 · p. 2
• WAV jointly models future dynamics and trajectory value, generating actions by inferring latent trajectories that are dynamically feasible and high-value.Source: §1 · p. 2
• Latent planning mitigates the exponential decay of feasible-trajectory probability that arises when planning directly in action space.Source: §1 · p. 2
• WAV achieves significant improvements in task success, generalization, and robustness across simulated benchmarks and real-world scenarios, particularly in long-horizon and compositional settings.Source: §1 · p. 2

2.1 World Models for VLA

• World-model approaches for VLA jointly model policy execution and future states, with later systems improving prediction accuracy, manipulation performance, simulation-based training, or synthetic-reward policy refinement.Source: §2.1 · pp. 2–3
• WAV instead introduces implicit planning as an internal inference mechanism, rather than scaling architecture capacity or relying on external simulation.Source: §2.1 · p. 3

2.2 Model-Based Reinforcement Learning

• Model-based reinforcement learning improves sample efficiency and long-horizon decision making through learned environment models for planning or policy optimization.Source: §2.2 · p. 3
• Latent world models support imagined rollouts from high-dimensional observations, while TD-MPC, TD-MPC2, and subsequent work combine latent planning, value learning, scalability, and task generalization.Source: §2.2 · p. 3
• WAV treats planning as an emergent property of inference in a structured generative model rather than explicit planning or iterative policy optimization.Source: §2.2 · p. 3

• A language-conditioned VLA receives visual observations, language instructions, and optionally proprioceptive state, then outputs robot actions from an expert dataset.Source: §3 · p. 3
• The VLA models action sequences as a conditional distribution over current context and trains them by maximum likelihood on expert demonstrations.Source: §3 · p. 3
• MPC selects actions by optimizing predicted future trajectories over a finite horizon and evaluates candidates using a learned world model.Source: §3 · p. 3
• MPC explicitly reasons over future trajectories and achieves improved robustness and sample efficiency compared with purely reactive, model-free policies.Source: §3 · p. 3

4.1 Theoretical Analysis of Latent Planning

• Latent planning reweights trajectory search toward dynamically and semantically valid regions, assigning exponentially larger feasible-trajectory probability than direct action-space sampling as horizon grows.This addresses the vanishing feasible mass caused by the thin manifold of valid state-action trajectories.Source: §4.1 · p. 4
• Feasibility alone is insufficient because feasible trajectories can have widely varying returns; iterative inference is therefore required to concentrate probability on semantically aligned, high-value trajectories.Under a subexponential sample budget, one-shot latent sampling does not guarantee discovery of a near-optimal trajectory.Source: §4.1 · pp. 4–5

4.2 Model Architecture and Training Objectives

• WAV comprises a language-conditioned video generator, a trajectory value module, and an action module that jointly support implicit planning over predicted futures.The video component predicts future visual trajectories, the value component evaluates long-horizon utility, and the action component generates executable actions.Source: §4 · p. 3, Figure 1 · p. 5
• The video generator conditions autoregressive visual rollouts on multi-view history, stochastic latent variables, and language semantics to represent task intent and spatiotemporal dynamics.It uses a frozen T5-XXL language encoder with a DiT backbone and produces the visual subspace of a full state-action trajectory.Source: §4.2 · p. 5
• Training uses three flow-matching stages: video generation, trajectory-value prediction after freezing the video model, and joint action decoding.The objectives correspond to video flow, cumulative discounted reward prediction, and action flow losses.Source: §4.2 · p. 6

4.3 Latent Planning and Iterative Inference

• During inference, WAV samples video and value latents, scores value predictions with SNR, selects elite explorations, updates noise distributions, and decodes actions from optimized latents.Variance decay and exponential smoothing are used to prevent distribution collapse and improve iteration stability.Source: §4.3 · pp. 6–7

5.1 Generalization Comparison in Simulation

• 98.1 overall average score on LIBERO, with state-of-the-art or competitive performance across all four task suites.The benchmark includes Spatial, Object, Goal, and Long suites probing different forms of generalization.Source: §5.1 · p. 7
• Removing latent trajectory planning lowers average performance by 1.7 points, with Long-suite performance decreasing from 94.4 to 91.8.The performance gap widens as task horizon increases.Source: §5.1 · p. 8

5.2 Real-World Evaluation

• Real-world evaluation compares WAV with GE-ACT across tasks involving spatial reasoning, deformable objects, visual grounding, and long-horizon planning.The quantitative comparison averages each result over 15 trials, while the evaluation uses a strict binary success metric.Source: §5.2 · p. 8, Figure 2 · p. 8

5.3 Ablation Study

• Increasing K from 1 to 5 improves success rate, but K=10 yields marginal gains; performance is more sensitive to video samples M than value samples N.Increasing M helps especially in the low-sample regime, whereas N saturates earlier.Source: §5.3 · p. 9, Figure 4 · p. 9
• Small smoothing parameters sharply reduce success, after which increasing α and β produces rapid gains followed by saturation.The results indicate that stabilizing latent-distribution updates is important, while excessive smoothing adds limited benefit.Source: §5.3 · p. 9, Figure 5 · p. 9
• K=3 provides a favorable performance–efficiency balance because gains saturate beyond three iterations while inference time continues increasing.Figure 6 reports success rate, inference time, and GPU memory usage across iteration counts.Source: §5.3 · p. 9, Figure 6 · p. 9

• WAV consistently improves task success, generalization, and robustness over previous baselines across simulation and real-world settings.Its latent trajectory planning maintains compact plausible futures and reshapes search toward high-value, low-uncertainty regions.Source: §6 · p. 10
• Deployment currently requires additional time and storage overhead, motivating future work on richer instructions and real-time physical-robot deployment.Source: §6 · p. 10

Vanishing feasible mass

• Uniform exploration encounters approximately feasible trajectories with probability that decays exponentially as the planning horizon increases.The analysis attributes this scaling challenge to growing ambient dimension versus low intrinsic dimensionality of physically and semantically feasible behaviors.Source: A Theoretical Analysis Details of… · pp. 14–15

Latent reweighting

• Latent planning assigns exponentially larger feasible-trajectory probability than direct action-space sampling when its learned generator approximately captures the feasible set.This is a conditional comparison based on reweighting probability toward dynamically and semantically valid regions.Source: A Theoretical Analysis Details of… · p. 15

Iterative latent planning

• Iterative latent inference is therefore necessary to adaptively reweight samples toward high-value feasible trajectories rather than relying on one-shot latent sampling.Examples include importance-weighted resampling, cross-entropy optimization, and gradient-based latent refinement.Source: A Theoretical Analysis Details of… · p. 16

Value concentration

• Feasibility alone does not ensure near-optimal discovery because latent probability mass may remain dispersed across trajectories with widely varying returns.A fixed sample budget growing subexponentially with horizon cannot generally guarantee finding an ε-optimal trajectory without additional concentration mechanisms.Source: A Theoretical Analysis Details of… · p. 16

Real-world setup

• Real-world evaluation uses a Piper robot with approximately 300 successful drawer-opening and bowl-organization trajectories and 2,000 towel-flattening trajectories.Each trajectory combines three RGB camera views with 14-dimensional robot joint states and lasts tens of seconds.Source: B Algorithm Implementation Details · p. 16
• The study trains one task-specific model per real-world task, enabling controlled evaluation across diverse manipulation challenges.Source: B Algorithm Implementation Details · pp. 16–17

Reward design

• Dense rewards retain task progress, behavior smoothness, and task completion components while omitting the sub-goal achievement term.The resulting reward uses nine terms corresponding to 16 scalar components in the bimanual setting.Source: B Algorithm Implementation Details · p. 17