Beyond Future Prediction: Denoising as Generative Adaptation for Robot Control

1 Introduction

• Generative DiTs provide pixel-level visual and language-conditioned structure, but how to transfer that prior to control remains unclear; existing approaches commonly use future visual prediction.The paper asks whether forward temporal semantics explain the value of generative co-training.Source: §1 · p. 2
• NowWAM removes the separate future target and denoises the current action-facing visual stream, coupling generative adaptation with action learning.The action expert sees visual-language context across denoising states during training, while inference uses the clean endpoint in one forward pass.Source: §1 · pp. 2–3
• 87.7% success with FLUX2-Klein and 87.8% with Z-Image on LIBERO-Plus; NowWAM outperforms the compared state-of-the-art baselines and halves training visual tokens from 784 to 392.Its gains are concentrated on challenging visual perturbations while maintaining near-saturated performance on standard LIBERO.Source: §1 · p. 2
• Controlled analyses find that past and future visual targets perform comparably, while restricting adaptation to the clean endpoint substantially reduces robustness.This suggests future prediction is not uniquely privileged, whereas the denoising trajectory remains important for control adaptation.Source: §1 · pp. 2–3
• The authors report generalization across benchmarks and architectures, including transfer from image-editing backbones to a pure text-to-image DiT.The simpler interface also halves the visual tokens used during joint training.Source: §1 · p. 3

Pretrained VLM backbones for robot control

• Pretrained VLM policies reuse understanding-oriented visual-language representations for action prediction, generally without changing the underlying visual-language interface.The related-work passage situates these policies among models scaling across datasets, embodiments, tasks, and action spaces.Source: §2 · p. 3

Generative backbones for robot control

• Pretrained generative representations support visual understanding and dense prediction beyond their native synthesis interface, motivating their reorganization for joint action learning.The cited applications include semantic correspondence, recognition, matching, geometry, depth, and segmentation.Source: §2 · p. 3
• Generative robot policies use interfaces including visual subgoals, joint denoising, and predictive representations, with future prediction especially prominent.Fast-WAM and ImageWAM retain future visual targets for training; NowWAM removes that target and applies denoising to the current action-facing representation.Source: §2 · p. 3

3.1 Generative Co-training with a Separate Visual Target

• Separate-target co-training adds a visual stream for generative adaptation, while the action stream uses current-observation context and does not directly consume target tokens.At deployment, the auxiliary target stream is absent and action prediction depends on the language instruction and current observation.Source: §3.1 · pp. 3–4

3.2 NOWWAM

• NowWAM applies the pretrained generative interface to the current visual stream, so the same trajectory-conditioned representation supports generative prediction and action learning.Unlike a separate current-frame target, the denoising trajectory is applied directly to the representation used for control.Source: §3.2 · p. 4

3.3 Trajectory-Conditioned Training, Clean-Endpoint Control

• NowWAM trains the action-facing representation across the current observation’s denoising trajectory, then uses its clean endpoint for single-pass inference.The visual stream supports both generative prediction and action learning during training.Source: Figure 2 · p. 4, §3.3 · p. 5

4.1 Experimental Setup

• Evaluation spans standard LIBERO, 100-shot RoboCasa transfer, and seven-perturbation LIBERO-Plus; controlled analyses vary one intervention under fixed settings.Main results use the full 10,030 LIBERO-Plus episodes and final EMA checkpoints; controlled analyses use a fixed 1,923-episode subset and non-EMA checkpoints.Source: §4.1 · p. 5

4.2 In-Distribution and Few-Shot Manipulation

• NowWAM reaches 98.4% average success on standard LIBERO and 64.9% over 1,200 RoboCasa episodes using 100 demonstrations per task.On RoboCasa, the same 100-shot budget yields 58.3% for DIAL and 56.0% for Fast-WAM.Source: §4.2 · p. 6, Table 1 · p. 6

4.3 Robustness under Distribution Shift

• 87.7% success on LIBERO-Plus with FLUX2-Klein improves over the future-target generative baseline’s 81.6% by 6.1 points; Z-Image reaches 87.8%.The benchmark covers seven perturbation categories.Source: §4.3 · p. 6, Table 2 · p. 7
• Under RobotInit and camera-view perturbations, Fast-WAM and ImageWAM show grounding or execution failures, while NowWAM completes the illustrated tasks.The examples include transferring a black bowl and opening a drawer before completing placement.Source: §4.3 · pp. 6–7, Figure 3 · p. 7

4.4 Ablations and Analysis

• Generative pretraining raises Edit-objective success from 58.14% to 83.05%, while past and future auxiliary targets perform comparably.The past target at t−16 reaches 83.80%; future targets at t+16 and t+32 reach 82.89% and 83.70%.Source: §4.4 · p. 8
• Clean-endpoint training reaches 77.48%, below uniform denoising-trajectory sampling at 83.46%; removing visual supervision also reduces success to 78.73%.Shifting sampling toward lower noise further improves success to 84.97%.Source: §4.4 · p. 8, Table 3 · p. 8

4.5 Training Efficiency

• NowWAM keeps only the current visual stream (392 tokens) rather than processing current and separate target streams (784 tokens), cutting step time from 2.85s to 1.63s (1.8×) without additional acceleration.The comparison uses the FLUX2-Klein-4B backbone on the same 2×H200 setup with global batch size 64.Source: Table 4 · p. 8, §4.5 · p. 9

4.6 Attention Visualization

• Across successful LIBERO-Plus and RoboCasa rollouts, attention focuses on task-relevant objects and interaction regions.Figure 4 places each RGB observation before attention maps over the rollout.Source: §4.6 · p. 9, Figure 4 · p. 9

• Across LIBERO-Plus and RoboCasa, NowWAM improves robustness and few-shot transfer, extends to pure text-to-image backbones, and reduces training cost while retaining competitive in-distribution performance.Source: §5 · p. 9
• Controlled studies find past and future visual targets perform comparably, while restricting adaptation to the clean denoising endpoint substantially reduces robustness.Source: §5 · p. 9
• NowWAM applies native denoising to the current visual stream and removes the separate future-target branch, coupling generative adaptation with action learning.Source: §5 · p. 9
• Future modeling may remain useful for explicit dynamics or long-horizon planning, although the studied manipulation settings do not require a distinct future target.Source: §5 · p. 9