PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient-WAM: A 1B-Parameter World-Action Model with Low-Cost Future Imagination

Jiajun Li, Tiecheng Guo, Yifan Ye, Rongyu Zhang, Xiaowei Chi, Qianpu Sun, Ying Li, Yunfan Lou, Yan Huang, Zhihe Lu, Meng Guo, Shanghang Zhang

arXiv:2606.10040v2cs.RO

TL;DR

WAMs couple future prediction with action generation, but reliance on photorealistic video can impose high inference latency and hardware demands that hinder real-time deployment. Efficient-WAM jointly optimizes model size, token count, and denoising steps to reduce inference costs, enabling low-latency real-world deployment while preserving strong world priors and control performance. 86.7% average success in simulation and 66.25% in real-world tasks were achieved with 98 ms per-chunk latency on a local consumer GPU, comparable to or better than heavyweight WAM baselines.

  • Problem

    WAMs couple future prediction with action generation, but reliance on photorealistic video can impose high inference latency and hardware demands that hinder real-time deployment.

    Source: §1 · p. 2

  • Method

    Efficient-WAM jointly optimizes model size, token count, and denoising steps to reduce inference costs, enabling low-latency real-world deployment while preserving strong world priors and control performance.

    Source: §1 · p. 2

  • Results

    86.7% average success in simulation and 66.25% in real-world tasks were achieved with 98 ms per-chunk latency on a local consumer GPU, comparable to or better than heavyweight WAM baselines.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results support using coarse, action-relevant future guidance rather than pursuing photorealistic future rendering for the reported manipulation tasks.

    Source: §1 · p. 2

    Coarse, low-resolution future latents may be less suitable for extreme pixel-level precision or micro-manipulation, such as thread insertion.

    Source: §6 · p. 8

Abstract

from arXiv · show

World-Action Models (WAMs) have emerged as a promising paradigm for embodied control by coupling future visual prediction with action generation. However, most existing WAMs rely on photorealistic future prediction, which incurs high inference latency and makes real-time robot deployment difficult. This motivates a more efficient WAM design that preserves the control benefits of future visual prediction while reducing its inference cost. We introduce Efficient-WAM, a World-Action Model that reduces the cost of future imagination while preserving its control benefit. Efficient-WAM improves inference efficiency via a compact video expert transferred from WAN-2.2-5B, token-sparse video latents, and asymmetric video-action denoising that allocates fewer sampling steps to video than to actions. Instead of optimizing the future branch for visual fidelity, Efficient-WAM treats future video prediction as a compact guidance signal for action generation. Comprehensive experiments on RoboTwin 2.0 and real-world manipulation tasks show that Efficient-WAM maintains strong action performance despite visibly coarse future predictions. While maintaining competitive control capabilities, our 1B-parameter model can reduce per-chunk latency to around 100 ms during physical deployment, achieving a 30x speedup over existing WAMs.

1 Introduction

Efficient-WAM addresses the latency and hardware demands of WAMs by treating future video as action-relevant guidance rather than a photorealistic goal. It combines a compact, knowledge-transferred video branch with fewer future tokens and asymmetric denoising to reduce inference cost while preserving control performance.

Motivation

  • WAMs couple future video prediction with action generation, but large video generators bring compute, latency, and hardware demands that impede real-time deployment [1] [2].The belief that sharper, more photorealistic futures yield better actions contributes to this costly design choice [5] [8].Source: §1 · p. 2

Design principle

  • The paper reframes future imagination around task-relevant geometry, motion tendencies, and contact cues rather than photorealistic images.VPP [4] reports effective action generation with one denoising step, while Fast-WAM [10] remains competitive without explicit future generation at inference.Source: §1 · p. 2

Approach

  • Structured transfer from WAN-2.2-5B prunes the video expert around geometry, dynamics, and contact, supporting lower-resolution future latents and fewer video denoising steps.The design allocates fewer sampling steps to video and more to action refinement.Source: §1 · p. 2

Contribution

  • Efficient-WAM jointly optimizes model size, token count, and denoising steps as an action-centric system, aiming for efficiency gains with minimal compromise to control performance.The paper contrasts this integrated approach with early-exit, layer-skipping, and single-step denoising or diffusion-policy distillation methods [14] [15] [17] [18].Source: §1 · p. 2

Results

  • 86.7% average success in simulation and 66.25% in real-world tasks, with 98 ms per-chunk latency on a local consumer GPU, are comparable to or better than heavyweight WAM baselines.The results are achieved despite intentionally coarse future predictions.Source: §1 · p. 2

2 Related Works

Prior WAMs use future prediction to inject physical priors into robot policies, but often inherit the high computational cost of video generators. Efficient-WAM focuses on compressing the video-imagination branch while retaining a lightweight future-guidance signal.

World-Action Models

  • WAMs couple future visual prediction with action generation to inject physical priors, while large backbones, dense tokens, and iterative denoising make many approaches computationally heavy [7] [19] [8] [9] [11].Being-H0.7 [20] avoids raw-pixel prediction, Fast-WAM [10] skips explicit future generation at inference, and other variants explore action-centered or asynchronous designs [21] [22].Source: §2 · p. 3

Efficient robot policies

  • Efficiency methods for VLA and generative robot policies target backbones or action sampling, complementing Efficient-WAM’s focus on the WAM video-imagination bottleneck [23] [14] [24] [15] [25] [26] [17] [18].Efficient-WAM transfers spatiotemporal knowledge from WAN-2.2-5B [13] into a compact, action-oriented video expert rather than reproducing a large video generator.Source: §2 · p. 3

3 Method

Efficient-WAM preserves joint future-and-action prediction while treating future video as coarse guidance for control. It reduces video-branch cost through a transferred compact expert, sparse future latents, and fewer video denoising steps, alongside staged training of the video and action branches.

3.1 Design Formulation

  • A WAM models future scene evolution and control actions jointly, and Efficient-WAM factorizes this into future imagination followed by future-conditioned action generation.The future branch predicts dynamic context; the action branch extracts executable control signals, without requiring photorealistic future latents.Source: §3.1 · p. 3

3.2 Compact Architecture with World-Knowledge Transfer

  • A compact Mixture-of-Transformers architecture pairs a video expert pruned from WAN-2.2-5B with a dedicated action expert, transferring task-relevant physical priors while shedding capacity for high-fidelity rendering.Selected teacher-layer weights are copied into the student, and teacher-guided distillation aligns hidden states and temporal changes; the experts interact layer-wise, with action tokens attending to video tokens.Source: §3.2 · p. 4

3.3 Coarse Future Prediction with Multiscale Video-Latent Layout

  • High-resolution current-observation tokens are concatenated with low-resolution future latents, preserving current-state detail while using future video as a coarse dynamic guide.The stated example resolutions are 384 × 320 for current observations and 192 × 160 for future frames; the lower future-token density preserves action accuracy while reducing attention cost and accelerating inference.Source: §3.3 · pp. 4–5

3.4 Asymmetric Video-Action Denoising

  • Asymmetric inference denoising gives actions more refinement steps than video, reusing cached video features between refreshes once actionable dynamics are clear.The action branch may receive 5 to 10 steps while the video branch receives only the initial 2 steps.Source: §3.4 · p. 5

3.5 Training Objectives

  • Training uses conditional flow matching in three phases: adapt the video expert with distillation, train the action expert with the video branch frozen, then co-train both experts end-to-end.The frozen-video phase combines action flow matching with a weighted video flow-matching loss; the final phase uses the joint objective for unified refinement.Source: §3.5 · p. 5

4 Experiments

Efficient-WAM and its real-time variant are evaluated in simulation, on the Astribot S1, and through component and system-level analyses. The results show strong manipulation performance alongside substantially lower inference latency, including for intentionally coarse future predictions.

4.1 Experimental Setup and Model Variants

  • The experiments compare a structural baseline with a deployment variant using low-resolution future latents and asymmetric denoising; both predict closed-loop action chunks of H = 16.RoboTwin simulation and ablations measure latency on an A800, while real-world tasks use a local RTX 4090.Source: §4.1 · p. 5

4.2 Evaluation in Simulation Environment

  • At 1B parameters, Efficient-WAM reaches 86.7% clean and 85.7% randomized success, beating 4B LingBot-VLA and 5B GigaWorld-Policy while trailing 8B Motus by 2.0% clean.RoboTwin 2.0 includes 50 bimanual tasks under clean and randomized visual settings, with 100 trials per task per setting.Source: §4.2 · p. 6, Table 1 · p. 6

4.3 Real-World Experiments

  • 66.25% average success and 98 ms per chunk: Efficient-WAM-RT slightly exceeds Motus in real-world success while delivering a 32× speedup.The Astribot S1 evaluation covers precise grasping, object transfer, semantic sorting, and bimanual coordination.Source: §4.3 · p. 6, Table 2 · p. 6, Figure 3 · p. 7

4.4 Ablation Studies

  • Teacher-guided distillation and inherited layer-slicing priors improve on randomly shrinking the video expert, reducing latency from 2013 ms to 430 ms.The ablation describes world-knowledge transfer as essential for an effective lightweight future branch.Source: §4.4 · p. 7
  • Reducing future tokens from 240 to 60 lowers latency from 430 ms to 377 ms while preserving task success, supporting the use of coarse structural cues.Table 3b evaluates lower-resolution future latents.Source: §4.4 · p. 7, Table 3 · p. 7
  • Reducing video denoising from [10] to [2] [10] cuts latency from 430 ms to 139 ms while success changes from 87.1% to 86.3%.The action denoising budget stays fixed; Figure 4 maps latency to bars, success to a line, and marks the selected configuration with shading.Source: §4.4 · p. 8, Figure 4 · p. 8

4.5 System-Level Efficiency Analysis

  • 98 ms per action chunk and a 30× speedup over standard WAMs are achieved by the hardware-optimized Efficient-WAM-RT on a local RTX 4090.The system analysis attributes intermediate latency reductions to compacting the video expert and to lower-resolution prediction and asymmetric denoising.Source: §4.5 · p. 8

5 Conclusion

Efficient-WAM targets the deployment bottleneck in World-Action Models by prioritizing control-relevant physical priors over photorealistic futures. Its compressed video branch maintains action accuracy comparable to heavyweight WAMs while reducing latency for real-time, closed-loop robot control.

5 Conclusion

  • Efficient-WAM combines world-knowledge transfer, low-resolution future latents, and asymmetric video-action denoising to reduce latency while maintaining action accuracy comparable to heavyweight WAM baselines.The conclusion frames its action-centric future-imagination principle as a way to support real-time, closed-loop robot control.Source: §5 · p. 8

6 Limitations

Coarse, low-resolution futures trade visual fidelity for speed, and may be less suitable for pixel-precise tasks. Physical deployment also uses a fixed asymmetric denoising schedule, with dynamic allocation proposed for future work.

6 Limitations

  • Higher-resolution visual guidance may benefit tasks requiring extreme pixel-level precision or micro-manipulation, unlike standard macroscopic manipulation.Examples include thread insertion versus grasping, transferring, and sorting.Source: §6 · p. 8
  • Coarse, low-resolution future latents trade visual fidelity for inference speed.The authors describe the approach as effective for standard macroscopic manipulation, while noting possible value in higher-resolution guidance for fine-grained tasks.Source: §6 · p. 8
  • Efficient-WAM-RT uses a fixed asymmetric denoising schedule during physical deployment, such as [2] [10] video-action steps.The authors propose exploring dynamic video-denoising allocation for uncertain or physically complex task dynamics.Source: §6 · p. 8

A Training Details

Training builds a compact video expert from WAN-2.2-5B, then attaches and jointly refines an action expert. The recipe uses teacher distillation, independently sampled video and action noise schedules, and different training setups for simulation and real-world tasks.

A Training Details

  • The staged recipe constructs a compact video expert, trains an attached action expert with the video backbone frozen, then jointly refines both experts.Stage 1 constructs the video expert; Stage 2 optimizes action-side parameters; Stage 3 updates both experts with distinct learning rates.Source: A Training Details · p. 12, Table 4 · p. 12
  • The compact video expert is initialized by structured slicing of WAN-2.2-5B, using selected teacher layers and reduced width.The 12-layer student uses 2048 hidden dimensions, 8192 FFN dimensions, and 16 attention heads.Source: A Training Details · p. 12
  • Stage 1 combines video flow matching with hidden-state and temporal-motion distillation from the teacher.The hidden-state loss uses cosine similarity; the distillation weight decays from 0.2 to 0.1 to 0 during training.Source: A Training Details · p. 12
  • After attaching the action expert, training independently samples flow-matching timesteps for action chunks and future video latents.The shared joint-attention backbone predicts both velocities, while stage-specific loss weights set distinct action and video supervision intensities.Source: A Training Details · pp. 12–13
  • Simulation uses one multitask policy across RoboTwin tasks, whereas real-world experiments train a separate policy for each task.Each stage spans approximately 2.5 epochs in simulation and 5 epochs for real-world tasks.Source: A Training Details · p. 13

B Real-World Evaluation Details

Real-world evaluation uses task-specific policies on the Astribot S1, with standardized trials and receding-horizon execution. The section also details task requirements and recurring failure types, with examples across the evaluation suite.

B Real-World Evaluation Details

  • Policies are tested on the Astribot S1 using 100 training demonstrations and 20 trials per task, with randomized feasible object resets.Inputs are three RGB views and 31-dimensional joint states; each trial is capped at 3 minutes and scored by strict task-specific binary criteria.Source: B Real-World Evaluation Details · p. 13
  • Pipette-tray grasping requires lifting and leveling the tray without touching its top, while reagent-bottle transfer requires avoiding drops, abrupt impacts, and collisions.The tray task uses a bottom-support end-effector; the bottle task tests navigation around adjacent containers.Source: B Real-World Evaluation Details · p. 13
  • LEGO sorting requires placing all 3 to 5 scattered blocks into matching containers, while pen uncapping requires coordinated bimanual stabilization and cap extraction.The LEGO task involves repeated localization, grasping, and placement; pen uncapping tests precision with a small cap.Source: B Real-World Evaluation Details · pp. 13–14
  • Successful trials typically take approximately 30 seconds with Efficient-WAM-RT versus around two minutes with heavyweight Motus.All evaluated models use action chunks of horizon H = 16 and execute 4 or 5 sampled steps before replanning.Source: B Real-World Evaluation Details · p. 14
  • Real-world failures include spatial misalignment, incomplete scene coverage, and contact or collision errors.Figure 6 examples include a tray end-effector snagging its rack, a bottle striking a nearby tray, a pen catching its holder rim, and an unhandled LEGO block.Source: B Real-World Evaluation Details · p. 14, Figure 6 · p. 14

C Latency Measurement Protocol and Summary

Latency measurements isolate the wall-clock cost of predicting one action chunk. The reported configurations distinguish component ablations on A800 from the fully optimized real-world deployment profile on RTX 4090.

C Latency Measurement Protocol and Summary

  • Latency is measured as the wall-clock time for one policy call to predict an action chunk with horizon H = 16.Measurements follow one warm-up run and use cached text embeddings.Source: C Latency Measurement Protocol and… · p. 15
  • The protocol excludes one-time T5 instruction encoding, robot execution time, and external observation acquisition to isolate policy-side prediction cost.Source: C Latency Measurement Protocol and… · p. 15
  • A800 rows report controlled simulation ablations, while Efficient-WAM-RT reports the complete real-world deployment configuration on a local RTX 4090.The ablations independently apply low-resolution latents and asymmetric denoising to the compact structural baseline; the deployment profile enables all three optimizations.Source: C Latency Measurement Protocol and… · p. 15, Table 5 · p. 15

D Qualitative Future Prediction Results

Efficient-WAM-RT produces visibly blurrier, less detailed futures than Full WAN, yet retains success rates close to the uncompressed model. The comparison illustrates the paper’s focus on coarse, task-relevant visual cues rather than photorealistic appearance.

D Qualitative Future Prediction Results

  • The comparison pairs the uncompressed video expert with Efficient-WAM-RT’s compact expert, low-resolution future latents, and asymmetric denoising.The configurations are presented as visual evidence for the claim that future prediction can support control without photorealistic fidelity.Source: D Qualitative Future Prediction Results · p. 15
  • 83.1% (clean) and 82.0% (randomized) success for Efficient-WAM-RT trail Full WAN’s 86.4% and 85.5% by a narrow margin.The passage attributes this resilience to preserving coarse, task-relevant geometry and motion cues without photorealistic appearance.Source: D Qualitative Future Prediction Results · p. 15
  • Full WAN produces coherent frames with distinct object boundaries, while Efficient-WAM-RT shows blur, ghosting, and diminished texture detail.Figures 7 and 8 show Full WAN and Efficient-WAM-RT, respectively.Source: D Qualitative Future Prediction Results · p. 15, Figure 7 · p. 16, Figure 8 · p. 16

E RoboTwin Detailed Results

The RoboTwin results show that fewer video denoising steps can substantially reduce latency while preserving success rates within the tested range. The detailed tables cover per-task clean and randomized evaluations and a clean-setting denoising sweep.

E RoboTwin Detailed Results

  • Table 6 reports per-task success rates for RoboTwin under clean and randomized evaluation settings.Efficient-WAM and Efficient-WAM-RT results come from the authors’ checkpoints over 100 rollouts per task; baseline scores are primarily drawn from original publications.Source: Table 6 · p. 17, E RoboTwin Detailed Results · p. 17
  • 3.1× speedup (430 ms to 139 ms) accompanies success rates of 87.1% versus 86.3%/86.2% when video steps fall from [10] to [2] [10].The sweep used 20 rollouts per task in RoboTwin’s clean setting; a single video step or aggressive action-side compression degraded performance.Source: E RoboTwin Detailed Results · p. 18, Table 7 · p. 18
  • Table 7 covers the asymmetric-denoising sweep on RoboTwin’s clean setting, where early video updates are used to extract action-centric structures.The passage describes this as avoiding compute on late-stage visual refinement.Source: E RoboTwin Detailed Results · p. 18, Table 7 · p. 18
Loading 2606.10040v2…