Source-linked AI summary
Light-WAM: Efficient World Action Models with State-Fusion Action Decoding
Ziang Li, Dongzhou Cheng, Yibin Wang, Shiyue Wang, Xiaoyang Xu, Lingxuan Weng, Juan Wang, Jiaqi Wang
TL;DR
Efficient deployment of World Action Models is limited by the cost and latency of large generative architectures. Light-WAM retains future-video supervision through compact latent representations and single-pass action decoding, achieving strong benchmark performance with substantially lower resource use.
Problem
Current World Action Models couple future-video prediction and action generation in large generative architectures, limiting efficient training and low-latency closed-loop deployment.
Method
Light-WAM combines a compact video backbone, downsampled latent-space future-video supervision, and a StateFusionActionExpert for single-pass action-chunk prediction.
Results
Light-WAM maintains strong LIBERO performance and achieves 76.4% average success across 50 RoboTwin 2.0 tasks while reducing trainable parameters to 0.44B and inference latency to 72.03ms.
Takeaways & Limitations
Light-WAM provides a favorable performance-efficiency trade-off for robot manipulation across LIBERO, RoboTwin 2.0, and real-world dual-arm tasks.
Takeaways & Limitations
Larger WAMs and embodied-pretrained policies continue to achieve higher success in more challenging multi-task settings, indicating that model capacity and large-scale embodied data remain important.
Abstract
from arXiv · showhide
World Action Models (WAMs) extend robot policy learning by incorporating future prediction as an additional training objective, encouraging the policy to encode task-relevant temporal structure in its representations. Current WAMs often rely on large-scale generative architectures that incur high training costs and inference latency, making them difficult to deploy as efficient closed-loop policies. We propose Light-WAM, a lightweight World Action Model for efficient robot manipulation. Specifically, it is built with a compact video backbone and performs future-video supervision in a downsampled latent space, reducing the cost of video co-training while retaining its benefits for representation learning. For action prediction, Light-WAM introduces the StateFusionActionExpert, which reads adapted states from multiple backbone layers, fuses them through learned-query pooling, and directly predicts action chunks in a single forward pass. This design provides an efficient interface between video backbone representations and robot actions, avoiding the need for heavy generative action experts. Experiments demonstrate that Light-WAM maintains strong performance on LIBERO and achieves usable multi-task performance on RoboTwin 2.0, while using only 0.44B trainable parameters. It also achieves 72.03ms inference latency with 4.1GiB peak GPU memory and improved training throughput.
1 Introduction
Light-WAM is a lightweight World Action Model that retains training-time future-video representation learning while reducing video-supervision and action-decoding costs. It combines downsampled latent-space supervision with StateFusionActionExpert decoding and achieves strong manipulation performance with substantially lower resource use.
- Architecture: Light-WAM uses a frozen Wan2.1-T2V-1.3B video backbone adapted with lightweight modules for efficient robot manipulation.The compact backbone and lightweight adaptation support the model’s reduced training and inference costs.
- Experimental Results: 97.2% average success on LIBERO is achieved without embodied pretraining, while RoboTwin 2.0 reaches 76.4% average success across 50 tasks.The LIBERO result is competitive with larger WAM baselines, and the RoboTwin result demonstrates usable multi-task performance.
- Efficiency Results: 0.44B trainable parameters, 4.25× training-throughput improvement, 72.03ms inference latency, and 4.1GiB peak GPU memory are reported versus Fast-WAM.Compared with Fast-WAM, trainable parameters decrease from 6.02B to 0.44B, alongside improved throughput and lower inference resource requirements.
- Efficient World Action Modeling: Light-WAM retains future-video co-training benefits while applying supervision in a downsampled latent space to reduce WAM training cost.The approach is motivated by evidence that test-time future-video generation is unnecessary for strong policy performance, with video prediction primarily benefiting training-time representation learning.
- StateFusionActionExpert: StateFusionActionExpert fuses multi-level adapted backbone states through learned-query pooling and predicts action chunks in one forward pass.It bridges video representations and robot actions without test-time future-video generation or a generative action expert.
2 Related Work
Related work spans instruction-following Vision Language Action models and World Action Models that augment robot action learning with future-video prediction. Light-WAM pursues efficient WAM policies, drawing on efficiency-focused VLA research and Fast-WAM while improving the overall pipeline rather than performing inference-time video rollout.
- Vision Language Action Models: VLA models condition robot actions on visual observations and language instructions, supporting instruction-following manipulation and scalable multi-task learning.Recent work, including SmolVLA, also explores compact architectures for more efficient training and deployment.
- World Action Models: WAMs couple robot action learning with future-video prediction to encourage representations of object motion, interaction dynamics, and task progress.Motus, LingBot-VA, and Fast-WAM exemplify this direction.
- Light-WAM Positioning: Light-WAM shares VLA research’s efficiency-oriented goal but applies it to WAM policies, using future-video prediction to shape visual representations for robot control.Its focus is efficient WAM policy design rather than VLA-only deployment.
- Light-WAM Positioning: Like Fast-WAM, Light-WAM treats video prediction as training-time supervision without executing it during inference, while targeting efficiency across the overall WAM pipeline.This distinguishes the work from approaches centered on inference-time video rollout.
3 Methodology
Light-WAM combines a compact, minimally adapted video backbone with downsampled latent future-video supervision and a direct StateFusionActionExpert action interface. The action branch fuses sparse multi-level backbone states through learned-query pooling and predicts action chunks directly from the current observation at inference.
- Video backbone adaptation: Light-WAM freezes the pretrained Wan2.1-T2V-1.3B backbone, applies low-rank updates, and inserts sparse bottleneck adapters for lightweight robot-domain adaptation.The adapters are placed at selected transformer depths, while low-rank updates modify attention and feed-forward projections.
- Multi-level state interface: The action branch reads a small set of adapted states from different backbone levels, exposing multi-granular visual information without decoding from all activations.These selected states form the interface between the video backbone and the action expert while keeping action decoding efficient.
- Latent-space video supervision: Future-video supervision operates on spatially downsampled latent videos, while action prediction retains the original-resolution current-observation latent.This separates lower-cost temporal supervision from the higher-resolution observation needed for manipulation.
- StateFusionActionExpert: The StateFusionActionExpert compresses each dense backbone state with learned-query pooling, then fuses the resulting level-wise representations into a fixed-width action state.The number of learned queries controls the information bottleneck: too few can discard manipulation-relevant details, whereas too many weaken compression and burden decoding.
- Action decoding and inference: At inference, Light-WAM predicts the complete action sequence directly from the current observation without future-video rollout.Training combines future-video supervision with action regression, but deployment uses the direct action pathway.
4 Experiments
Light-WAM achieves competitive LIBERO performance and usable 50-task RoboTwin 2.0 performance without embodied pretraining. It also substantially reduces training and inference costs while retaining effective latent-space video supervision and single-pass action decoding.
- LIBERO: 97.2% average success ranks Light-WAM first without embodied pretraining and third overall on LIBERO.Its suite success rates are 98.2% Spatial, 99.6% Object, 97.8% Goal, and 93.0% Long.
- RoboTwin 2.0: 76.4% average success on RoboTwin 2.0 demonstrates usable performance across 50 bimanual manipulation tasks without embodied pretraining.Light-WAM outperforms π0 and X-VLA and is competitive with Motus without embodied pretraining, though it trails Fast-WAM and the strongest embodied-pretrained WAMs.
- Efficiency: 13.7× fewer trainable parameters and 3.4× fewer total training-time parameters than Fast-WAM reduce Light-WAM’s training burden.Trainable parameters decrease from 6.02B to 0.44B, total training-time parameters from 6.73B to 1.99B, peak memory from 70.7GiB to 43.1GiB, and throughput rises from 0.49 to 2.08 steps/s.
- Efficiency: 72.03ms overall inference latency with 4.1GiB peak GPU memory makes Light-WAM substantially faster and lighter than prior WAM methods.The action branch takes 2.1ms, while the reported latency includes VAE encoding and policy forward with cached language context.
- Ablations: 99.0% LIBERO-Spatial success with original-resolution video latents improves over 98.2%, but substantially increases training cost.Adding adapter layers from 3 to 5 changes success from 98.2% to 98.0%, whereas reducing learned queries from 16 to 8 decreases success to 95.4%.
- Qualitative analysis: Downsampled latent predictions remain smoother than reference frames while capturing main motion and scene changes, indicating useful temporal information.The qualitative analysis compares predicted and reference future frames at t = {+8, +16, +24, +32}.
5 Conclusion and Limitations
Light-WAM combines compact video modeling, latent-space video supervision, and StateFusionActionExpert decoding to improve World Action Model efficiency. Experiments indicate a favorable performance-efficiency trade-off, while challenging multi-task settings remain a limitation.
- Conclusion: Light-WAM is presented as a lightweight World Action Model for efficient robot manipulation.The paper evaluates it on LIBERO, RoboTwin 2.0, and real-world dual-arm tasks.
- Conclusion: Compact video backbones, downsampled latent-space video supervision, and StateFusionActionExpert decoding improve WAM training and inference efficiency.These components are combined in the proposed Light-WAM design.
- Limitations: A favorable performance-efficiency trade-off is demonstrated on LIBERO, RoboTwin 2.0, and real-world dual-arm tasks.The passage identifies larger WAMs as relevant in more challenging multi-task settings, indicating a remaining limitation.
A Algorithmic Details
Light-WAM freezes the Wan2.1-T2V-1.3B video backbone while adapting it with LoRA and sparse residual bottleneck adapters. Adapted states from layers {8, 16, 24} are pooled and fused by the StateFusionActionExpert for action prediction, while the final backbone output supports future-video co-training.
- Backbone adaptation: Light-WAM freezes Wan2.1-T2V-1.3B and applies LoRA across self-attention, cross-attention, and feed-forward projections, with sparse WAM adapters at layers {8, 16, 24}.Each WAM adapter is a residual bottleneck module with a 256-dimensional bottleneck.
- Backbone adaptation: The selected adapted states feed the StateFusionActionExpert, while the final backbone output feeds the video prediction head for future-video co-training.The default adapter scale is γ = 1.0.
- State-fusion: The StateFusionActionExpert uses layer-specific learnable queries to pool adapted video tokens from each selected backbone layer.This query-based pooling provides the interface from backbone states to action chunks.
- State-fusion: 16 queries and 8 attention heads are used per layer, with pooled states projected to 4608 dimensions and concatenated into a 6144-dimensional fused state.The fused state is further processed by a residual MLP block before temporal decoding and action prediction.
B Training and Implementation Details
Light-WAM training uses AdamW with cached video latents, frozen pretrained components, and task-specific batch sizes on four H100 GPUs. Training jointly optimizes future-video and action objectives, with checkpoints selected at suite-dependent training steps.
- Training setup: 1 × 10−4 learning rate, 1 × 10−2 weight decay, cosine scheduling, and 1,000 warmup steps define the AdamW training setup.All models are trained on 4 NVIDIA H100 GPUs; global batch sizes are 64 for LIBERO and 128 for RoboTwin 2.0.
- Training setup: Cached Wan2.1 VAE latents remove online VAE encoding during training, while evaluation uses online encoding and pretrained video-backbone weights remain frozen.The setup also freezes the VAE, while trainable adaptation modules are optimized around the frozen pretrained components.
- Checkpoint selection: 60K, 60K, 12.5K, and 80K steps select LIBERO checkpoints for Spatial, Goal, Object, and Long, respectively, while RoboTwin 2.0 uses 460K steps.Checkpoint selection is performed separately for each LIBERO suite.
- Parameter breakdown: 0.44B trainable parameters comprise most of the StateFusionActionExpert and backbone LoRA modules within a 1.99B-parameter model.The pretrained video backbone and VAE remain frozen.
- Training algorithm: Training jointly optimizes future-video and action losses, using downsampled noisy video latents for video supervision and multi-level adapted states for action prediction.The RoboTwin 2.0 procedure constructs a three-camera canvas, subsamples frames with stride 4, and uses L = Lvideo + Laction.
C Full RoboTwin 2.0 Results
Table 7 presents the complete per-task results for RoboTwin 2.0.
- Table 7 reports full RoboTwin 2.0 per-task results.
D Real-World Rollouts
Additional real-world rollouts visualize Light-WAM on three dual-arm tasks and compare its future-video predictions with ground-truth future frames.
- Figure 5 presents additional real-world rollout frames for Light-WAM.
- The rollouts cover three dual-arm tasks.
- Future-video predictions are shown alongside ground-truth future frames.