PaperlayerView exact revision on arXiv

Source-linked AI summary

AdaWorldPolicy: World-Model-Driven Diffusion Policy with Online Adaptive Learning for Robotic Manipulation

Ge Yuan, Qiyuan Qiao, Jing Zhang, Dong Xu

arXiv:2602.20057v1cs.ROcs.AI

TL;DR

Robotic manipulation needs policies that anticipate physical outcomes and adapt under changing conditions without extensive human demonstrations. AdaWorldPolicy combines a world model, action expert, force predictor, and AdaOL closed-loop updates, achieving strong benchmark performance and improved OOD results. Its world-model predictions remain less robust under extreme shifts such as continuously varying lighting.

  • Problem

    Robotic manipulation policies struggle to anticipate dynamics and generalize to unseen environments while adapting without new human demonstrations.

    Source: §1 · p. 1, §3.1 · p. 3

  • Method

    AdaWorldPolicy integrates a world model, action expert, and force predictor in interconnected diffusion transformers, with AdaOL switching between action generation and future imagination for online updates.

    Source: §3.3 · p. 5

  • Results

    AdaWorldPolicy achieves state-of-the-art results across diverse benchmarks, while AdaOL improves out-of-distribution performance by over 5% and in-domain results by approximately 1%.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    World-model prediction errors can provide self-supervised signals for reactive adaptation to visual and physical domain shifts with minimal computational latency.

    Source: §5 · p. 8

    The Cosmos-Predict2 world model degrades under drastic domain shifts such as continuously varying lighting, indicating limited robustness in extreme OOD scenarios.

    Source: §8 · p. 15

Abstract

from arXiv · show

Effective robotic manipulation requires policies that can anticipate physical outcomes and adapt to real-world environments. Effective robotic manipulation requires policies that can anticipate physical outcomes and adapt to real-world environments. In this work, we introduce a unified framework, World-Model-Driven Diffusion Policy with Online Adaptive Learning (AdaWorldPolicy) to enhance robotic manipulation under dynamic conditions with minimal human involvement. Our core insight is that world models provide strong supervision signals, enabling online adaptive learning in dynamic environments, which can be complemented by force-torque feedback to mitigate dynamic force shifts. Our AdaWorldPolicy integrates a world model, an action expert, and a force predictor-all implemented as interconnected Flow Matching Diffusion Transformers (DiT). They are interconnected via the multi-modal self-attention layers, enabling deep feature exchange for joint learning while preserving their distinct modularity characteristics. We further propose a novel Online Adaptive Learning (AdaOL) strategy that dynamically switches between an Action Generation mode and a Future Imagination mode to drive reactive updates across all three modules. This creates a powerful closed-loop mechanism that adapts to both visual and physical domain shifts with minimal overhead. Across a suite of simulated and real-robot benchmarks, our AdaWorldPolicy achieves state-of-the-art performance, with dynamical adaptive capacity to out-of-distribution scenarios.

1. Introduction

AdaWorldPolicy addresses robotic manipulation under dynamic, contact-rich conditions by combining a world model, action policy, force prediction, and online adaptation. Its AdaOL loop uses real-world prediction discrepancies to update the system with minimal human involvement.

Motivation

  • AdaWorldPolicy targets manipulation policies that can anticipate outcomes and adapt to unseen or dynamically changing environments without extensive new demonstrations.Existing VLA and world-model approaches are limited by demonstration demands, passive world-model use, or offline-only adaptation.Source: §1 · pp. 1–2

AdaWorldPolicy

  • The framework unifies a world model, action expert, and force predictor as interconnected Flow Matching Diffusion Transformers.Multi-modal self-attention enables feature exchange while preserving distinct computational pathways; force-torque feedback addresses dynamic force shifts.Source: §1 · p. 2

AdaOL

  • AdaOL alternates between Action Generation and Future Imagination, using discrepancies between predicted and actual visual and force feedback for online updates.The action model generates behavior from current observations, while the world model predicts action-conditioned future observations and the force predictor compares predicted and actual force readings.Source: Figure 1 · p. 1, §1 · p. 2

Contributions

  • The unified framework and AdaOL strategy enable reactive closed-loop adaptation to visual and physical domain shifts.Updates affect network parameters across the three modules and are designed for fast adaptation during dynamic deployment.Source: §1 · p. 2
  • AdaWorldPolicy achieves state-of-the-art results across PushT, CALVIN, and LIBERO, while AdaOL improves OOD performance by over 5% and in-domain results by approximately 1%.Real-robot experiments also validate AdaOL and the force-aware extension on challenging manipulation tasks.Source: §1 · p. 2

2. Related Work

Prior robotic control methods range from model-free policies to model-based and hybrid systems, while diffusion policies often lack explicit dynamics modeling. AdaWorldPolicy integrates world modeling directly into diffusion control and uses prediction error for physically grounded online adaptation.

World Models for Robotic Control

  • Model-free visuomotor policies benefit from large-scale pretraining but lack explicit physical or dynamic reasoning, limiting performance in novel or out-of-distribution environments.These methods directly map visual or multimodal inputs to actions.Source: §2 · p. 2
  • Model-based methods learn environment dynamics for planning or policy optimization, but have traditionally focused on narrower domains or simulated settings.Their internal world models provide stronger physical grounding and better use of training data.Source: §2 · p. 2
  • Hybrid approaches integrate world models with policies, but AdaWorldPolicy makes the world model an active supervisor and correction mechanism rather than a passive validator.Its fusion mechanism combines world modeling, perception, and policy within one architecture.Source: §2 · pp. 2–3

Diffusion Models for Decision Making

  • Diffusion decision policies model robot trajectories effectively but can generate physically inconsistent actions because behavior cloning often ignores dynamics and outcome modeling.Recent alternatives add hierarchical generation, kinematic constraints, or contact-aware rollouts; AdaWorldPolicy instead incorporates a world model within the diffusion process.Source: §2 · pp. 2–3

Online Adaptation for Robotics

  • Robotic test-time adaptation uses unlabeled inputs and parameter-efficient updates, while AdaOL extends this paradigm with world-model prediction error as a self-supervised signal.The strategy targets both visual distribution mismatch and dynamic force shifts with fast, physically grounded updates.Source: §2 · p. 3

3. Methodology

AdaWorldPolicy addresses test-time adaptation for goal-conditioned manipulation by integrating visual prediction, force prediction, and action generation into a unified diffusion framework. Its dual-mode closed loop uses real-world prediction errors to update the policy online with limited parameter overhead.

3.1. Problem Formulation

  • AdaWorldPolicy targets goal-conditioned manipulation that adapts to novel dynamics at test time without new human demonstrations.The policy uses multimodal observation histories containing visual and force-torque information to generate future actions.Source: §3.1 · p. 3

3.2. Method Overview

  • The framework combines a pretrained World Model, lightweight Force Predictor, and Action Model as parallel Flow Matching Diffusion Transformers linked by multi-modal self-attention.Shared text, robot-state, and diffusion-noise conditioning is injected into the three backbones through adaLN layers.Source: §3.2 · p. 3, Figure 2 · p. 4, §3.3 · p. 5

3.3. World-Model-Driven Diffusion Policy

  • The Force Predictor extends visual prediction to future force-torque readings, complementing the World Model in contact-rich manipulation.It is implemented as a lightweight 0.4B-parameter DiT conditioned on current state and action.Source: §3.3 · p. 4
  • Mode I generates actions from noisy action tokens, whereas Mode II conditions on known actions to predict future visual and force-related states.The action-input mask selects the operational mode, allowing the same architecture to support policy execution and future prediction.Source: §3.3 · pp. 4–5
  • The joint Flow Matching objective alternates between Action Generation and Future Imagination losses to develop a shared representation of world dynamics.The weighted objective combines the action vector-field prediction term with the future-observation prediction term, controlled by p_a.Source: §3.3 · p. 5

3.4. Closed-loop Online Adaptive Learning

  • AdaOL executes an action, compares the resulting real state with the imagined state, and uses the prediction discrepancy to produce an online corrective update.The cycle grounds the world model and force predictor in reality before the next iteration.Source: §3.4 · p. 5, Figure 3 · p. 5
  • LoRA makes online adaptation practical by updating fewer than 0.1% of parameters instead of backpropagating through the full multi-billion-parameter model.This reduces computational and memory overhead for deployment on resource-constrained robots.Source: §3.4 · p. 6

4. Experiments

Experiments evaluate AdaWorldPolicy on simulated and real-world manipulation benchmarks, including domain shifts, implementation efficiency, and component ablations. Across these settings, online adaptation improves robustness, while the full architecture achieves strong benchmark performance.

4.1. Experimental Setup

  • The real-robot evaluation covers four long-horizon tasks with visual and physical perturbations, using cameras and force sensing to test adaptation.Source: §4.1 · p. 6, Figure 5 · p. 7

4.2. Implementation Details

  • Offline training uses a Cosmos-Predict2-based 2B-parameter world model and two 0.4B-parameter DiTs, while simulator benchmarks omit the force predictor.Source: §4.2 · p. 6
  • LoRA-based test-time adaptation updates selected backbone layers with two gradient steps per incoming sample, making inference approximately 5% slower than the non-adaptive baseline.Source: §4.2 · pp. 6–7

4.3. Main Results

  • The base AWP achieves state-of-the-art performance on LIBERO-10 in both static-camera and full multi-modal settings without online adaptation.Source: Table 1 · p. 6, §4.3 · p. 7
  • AWP (ol) consistently and substantially recovers performance under Variant PushT visual shifts, becoming the top-performing method across all challenging out-of-distribution scenarios.Source: §4.3 · p. 7, Table 2 · p. 7
  • AWP achieves state-of-the-art long-sequence task completion on CALVIN, while online learning provides a further consistent performance boost.Source: §4.3 · p. 7, Table 3 · p. 7
  • Under real-world domain shifts, AWP (ol) outperforms AWP and all baselines across tasks and perturbations, improving Pour success from 80% to 90% under Object Change.The closed-loop process runs at an average of 4Hz.Source: §4.3 · pp. 7–8, Figure 6 · p. 8

4.4. Ablation Studies

  • Ablations yield 76.3% average success for the full model, versus 72.5% without AdaOL, 53.8% without force prediction, and 46.3% without world-model supervision.Replacing MMSA with concatenation or cross-attention reduces performance to 36.3% or 50.0%, respectively.Source: §4.4 · p. 8, Table 4 · p. 8

5. Conclusion

The paper introduces AdaWorldPolicy, which combines a world-model-driven diffusion policy with AdaOL for online adaptation. Experiments show robust performance under out-of-distribution conditions with low computational latency, while future work targets long-horizon failures and larger models.

  • AdaWorldPolicy integrates a world model, diffusion policy, and AdaOL to adapt to visual and physical domain shifts in simulation and real-world settings.Source: §5 · p. 8
  • The method significantly outperforms strong non-adaptive baselines under out-of-distribution conditions while maintaining minimal computational latency.Source: §5 · p. 8
  • Future work will address long-horizon planning failures and scale the adaptation mechanism to larger network architectures.Source: §5 · p. 8

6. Additional Results

Additional results show that AdaWorldPolicy produces effective real-world demonstrations, predicts future task dynamics with useful fidelity, and benefits from adaptive learning, efficient sampling, and multimodal fusion.

6.1. Real-world Video Results

  • The supplementary videos cover four manipulation tasks in both in-domain settings and domain-shift conditions involving tablecloths, distractors, object instances, and lighting.The demonstrations include Sweep Coffee Beans, Pick-and-Place Eggs, Pour Water, and Wipe Whiteboard.Source: §6.1 · p. 11

6.2. Visualization of Imagined Future Frames

  • The imagined future frames closely match observations in PushT, CALVIN, and Libero10, capturing robot and object dynamics despite minor artifacts.Simulation predictions are highly consistent with ground truth, while real-world predictions preserve gripper motion and key interactions.Source: §6.2 · p. 11, Figure 7 · p. 12
  • Real-world imagined frames generally capture task dynamics, but artifacts and blurring increase in cluttered scenes such as the egg-manipulation task.The reported artifacts are attributed to limitations of the Cosmos-Predict2 backbone, while structural and semantic consistency remains sufficient for manipulation.Source: §6.2 · p. 11, Figure 8 · p. 13

6.3. Ablation Studies on Libero10

  • Enabling AdaOL with 10 sampling steps improves success rate from 95.53% to 96.05%, while reducing denoising steps to 2 retains a competitive 94.00% success rate.The improvement indicates AdaOL mitigates subtle distribution shifts and improves policy precision; the reduced-step result demonstrates robust, efficient traversal of a high-quality manifold.Source: §6.3 · p. 13
  • MMSA achieves 95.53% success, exceeding Cross-Attention at 91.21% and concatenation at 89.67% when other components remain fixed.The comparison supports MMSA as the stronger multimodal fusion mechanism for precise manipulation.Source: §6.3 · p. 14

7. Implementation Details

Implementation details tailor AdaWorldPolicy’s architecture, preprocessing, and evaluation protocol to computational constraints, haptic feedback, and real-world domain-shift testing.

7.1. Network Details

  • Cosmos-Predict2 supplies the world-model backbone, while a separate action branch uses MMSA to preserve pretrained weights and support flexible action modeling.The 2B backbone balances generation quality and computational efficiency; multi-view inputs concatenate view tokens temporally with shared RoPE.Source: §7.1 · p. 14
  • Real-world experiments use approximately 0.4B-parameter action and force models, whereas simulation increases the action model to 0.6B when force data is unavailable.The adjustment reallocates computation toward policy expressivity with only a marginal increase in overhead.Source: §7.1 · p. 14

7.2. Data Processing

  • Real-world inference predicts four sparse future frames across a 32-step action horizon to reduce video-generation latency.Skipped-frame prediction covers the temporal span without generating a dense sequence.Source: §7.2 · p. 14, Table 6 · p. 14
  • Quantile normalization using the 1st and 99th percentiles robustly preprocesses variable force measurements and improves convergence speed and final policy performance.This avoids instability and compression caused by absolute-extreme min-max scaling.Source: §7.2 · pp. 14–15

7.3. Real-world Evaluation Protocol

  • Each real-world task uses 150 expert demonstrations, 30 trials per task and domain, and a 1500-step execution limit.AdaOL evaluation adapts during Trials 1-15, freezes weights during Trials 16-30, and reports the overall 30-trial success rate.Source: §7.3 · p. 15
  • The evaluation defines task-specific success criteria and compares against DP-Force, a Diffusion Policy baseline augmented with 6-dimensional force data.The criteria cover residual beans, collision-free egg placement, transferred water volume, and remaining marker-trace length.Source: §7.3 · p. 15

8. Limitation and Future Work

The authors identify limitations in world-model generalization and online-adaptation tuning, especially under substantial domain shifts and changing deployment conditions.

Limitations

  • Cosmos-Predict2 generalizes imperfectly under significant domain shifts, with predicted-frame quality degrading under drastic changes such as continuous varying lighting.The authors note that policy execution can remain effective despite minor visual artifacts and expect stronger world models to reduce distortions.Source: §8 · p. 15

Future Work

  • Future improvements are directed toward stronger world models and automatic adaptation-parameter tuning for more robust deployment under extreme shifts.These directions follow directly from the reported backbone-generalization and fixed-AdaOL-parameter limitations.Source: §8 · p. 15
  • AdaOL uses one fixed hyperparameter set across all domain-shift experiments rather than task- or environment-specific tuning.The authors propose adaptive or meta-learning methods to tune these parameters during deployment.Source: §8 · p. 15

Related papers

Loading 2602.20057v1…