Source-linked AI summary
Do Robotic World Models Really Follow Actions? Diagnosing and Aligning Action-Conditioned Generation for Policy Learning
Sixiang Chen, Jiaming Liu, Jixian Wu, Yichen Guo, Tinghao Wang, Siyuan Qian, Hao Chen, Jiajun Cao, Jian Tang, Shanghang Zhang
TL;DR
Action-conditioned world models are used as learned simulators, but whether they faithfully follow arbitrary valid actions remains insufficiently evaluated beyond expert demonstrations. The paper introduces WorldEcho to measure visual integrity and SE(3) trajectory alignment across off-expert queries, and WorldSync to improve action-conditioned generation. WorldSync improves WorldEcho performance on RoboTwin and real-robot tasks and enables higher policy success rates during iterative policy improvement.
Problem
Action-conditioned world models rely on an insufficiently verified assumption that generated futures faithfully reflect arbitrary valid actions, while existing evaluation is typically limited to expert demonstrations.
Method
WorldEcho evaluates demonstrated and off-expert action queries using visual integrity and SE(3) trajectory alignment, while WorldSync improves generation through distributional coverage, representational grounding, and intervention-effect alignment.
Results
WorldSync improves WorldEcho performance across demonstrated and off-expert queries while maintaining visual integrity, and enables policies to achieve higher success rates in iterative policy improvement.
Takeaways & Limitations
Faithful action following is treated as a prerequisite for using action-conditioned world models as reliable simulators for policy evaluation and post-training.
Takeaways & Limitations
Comprehensively evaluating long-horizon interactions across diverse embodiments and open-world environments remains a challenge for future work.
Abstract
from arXiv · showhide
Action-conditioned world models are increasingly used as learned simulators for policy evaluation and improvement, yet their effectiveness rests on an unverified assumption: generated futures faithfully reflect arbitrary valid actions. Existing benchmarks are typically confined to expert demonstrations, leaving off-expert action following inadequately evaluated. To address this gap, we introduce WorldEcho, which probes action following over a broader action distribution using visual integrity and SE(3) trajectory alignment. Our diagnosis shows that current world models reasonably execute expert actions but struggle with diverse off-expert trajectories, either ignoring the commanded actions or producing visually invalid rollouts. We further propose WorldSync, which strengthens action following along three complementary axes: distributional coverage, representational grounding, and intervention-effect alignment. It broadens the training distribution over action consequences, grounds intermediate video representations in action-induced robot dynamics through an Action-Forcing Expert, and aligns predicted changes under action interventions with the corresponding changes in ground-truth futures. Experiments on RoboTwin benchmarks and real-robot tasks show that WorldSync improves WorldEcho metrics and serves as a more reliable simulator for iterative policy improvement, enabling policies to achieve higher success rates.
1 Introduction
The paper identifies a gap in evaluating whether action-conditioned world models follow off-expert actions, then introduces WorldEcho for diagnosis and WorldSync for improving action-conditioned generation. Experiments show improved action-following evaluation and higher policy success rates in simulation and real-robot policy-improvement settings.
- Motivation: Action-conditioned world models support policy evaluation and improvement but assume, largely without verification, that generated futures accurately respond to arbitrary valid actions.Unreliable action following can misrepresent candidate-action consequences and reduce the usefulness of synthetic supervision.
- Diagnosis: WorldEcho evaluates demonstrated and four off-expert action-query categories using visual integrity and SE(3) end-effector trajectory alignment.The categories are Cross-State Replay, Local Perturbation, Policy Rollout, and Feasible-Space Sampling.
- Diagnosis: Expert-only action-conditioned world models follow demonstrated actions reasonably well but fail on off-expert actions through action mismatch or severe visual degradation.Observed failures include visually plausible yet overly optimistic futures that disregard commands, and distorted robot arms or disappearing grippers.
- Improvement: WorldSync strengthens action-conditioned generation through distributional coverage, representational grounding, and intervention-effect alignment.Its training recipe broadens action consequences, uses an Action-Forcing Expert to ground video representations in robot dynamics, and aligns predicted intervention changes with ground-truth changes.
- Validation: WorldSync improves WorldEcho performance across demonstrated and off-expert queries while maintaining visual integrity, and enables higher policy success rates in iterative policy improvement.The reported experiments cover RoboTwin benchmarks and real-robot tasks.
2 Related Work
Related work uses action-conditioned world models as policy-facing simulators and expands training beyond expert data, but policy gains do not directly establish faithful action following. Existing robotic world-model benchmarks assess motion and embodied consistency, motivating direct action-conditioned evaluation.
- Policy-facing simulators: Action-conditioned world models reduce costly real-robot interaction by supporting imagined rollouts for policy evaluation, ranking, and synthetic experience.These applications make their response to queried actions directly relevant to simulator reliability.
- Distribution shift: Policy exploration, failures, and updates create distribution shifts away from expert-dominated action-conditioned world-model data.Prior systems address this mismatch with exploratory or corrective interactions, filtering, or joint policy–world-model evolution.
- Evaluation gap: Downstream policy gains alone do not reveal whether a world model faithfully responds to queried actions or merely provides useful visual augmentation.This ambiguity motivates direct assessment of action following.
- Evaluation benchmarks: Robotic world-model benchmarks increasingly evaluate motion, physical consistency, and embodied functionality beyond generic video quality.Related protocols compare generated motion with references or recover actions from generated videos to test executability.
3 Method
This method formulates action-conditioned generation as predicting future visual observations from current observations, instructions, and robot actions, then evaluates whether generated rollouts remain visually valid and follow action-specific robot motion. WorldEcho broadens evaluation beyond expert replay using complementary action queries, visual-integrity checks, trajectory alignment, and integrity-gated aggregation.
- 3.1 Problem Formulation: The world model generates future multi-view observations conditioned on the current observation, task instruction, and an H-step robot-action sequence.The same action sequence is executed from the corresponding initial environment state to obtain the ground-truth future.
- 3.1 Problem Formulation: A reliable rollout must remain visually valid and produce an end-effector trajectory that agrees with the ground-truth consequence of the queried actions.This evaluation targets action sequences extending beyond the expert action distribution.
- 3.3 WorldEcho: Benchmarking Action Following: WorldEcho compares generated and simulator-reference rollouts across five query categories, including demonstrated actions and four progressively more off-expert action distributions.The off-expert categories are Cross-State Replay, Local Perturbation, Policy Rollout, and Feasible-Space Sampling.
- 3.3 WorldEcho: Benchmarking Action Following: Visual integrity combines image quality, motion smoothness, end-effector visibility, and arm integrity, and a rollout passes only when all four conditions are satisfied.The quality and smoothness criteria use fixed thresholds, while visibility and arm integrity provide binary decisions.
- 3.3 WorldEcho: Benchmarking Action Following: WorldEcho measures pose-aware NDTW between generated and ground-truth end-effector trajectories, retaining absolute pose-discrepancy scale and interpreting lower values as stronger agreement.The official score preserves ungated NDTW diagnostically but assigns visually invalid rollouts a fixed penalty κ.
- 3.3 WorldEcho: Benchmarking Action Following: The evaluation averages integrity-gated errors within each task before macro-averaging across tasks, preventing tasks with more samples from dominating the ranking.It also reports visual-gate pass rate, ungated NDTW error, and results stratified by action-query category.
4 Experiments
Experiments show that expert-only evaluation understates off-expert action-following errors, while WorldSync improves trajectory alignment, visual integrity, and downstream policy improvement across simulation and real-robot settings.
- 4.2 Benchmark Diagnosis: Integrity-gated error increased by 0.029–0.099 m on off-expert queries across all six expert-trained models.This indicates that demonstrated-action evaluation systematically understated errors under feasible but unseen controls.
- 4.2 Benchmark Diagnosis: Off-expert queries increased raw NDTW by 0.010–0.043 m and visual failure rate by 6.3–28.1 percentage points across all models.The failure gap reflected both trajectory inconsistency and visual degradation.
- 4.2 Benchmark Diagnosis: WorldEcho evaluates a substantially broader action distribution because its four off-expert categories extend beyond the compact region occupied by expert actions.The protocol combines action-distribution coverage with visual-integrity and trajectory-alignment metrics.
- 4.3 Main Action-Following Evaluation: Expanded Action Coverage lowered integrity-gated error and raw NDTW for all six baseline backbones, while visual-pass-rate effects varied by backbone.Visual pass rate improved for three backbones, remained nearly unchanged for two, and decreased for one.
- 4.3 Main Action-Following Evaluation: WorldSync achieved the lowest integrity-gated error at 0.066 versus CtrlWorld’s 0.067 and the highest visual pass rate at 84.5% versus Motus’s 84.3%.Cosmos-Predict2.5 nevertheless achieved lower raw NDTW than WorldSync, 0.013 versus 0.022.
- 4.4 Policy Improvement: WorldSync improved policy success from 51–52% to 65% in RoboTwin and from 48% to 68% on the real-robot task after two refinement rounds.CtrlWorld reached 56–57% in simulation and 56% on the real-robot task under the compared conditions.
- 4.5 Component Contributions and Interactions: With coverage expansion, IE, and AFE evaluated separately, coverage improved action consistency, IE drove trajectory gains, and AFE improved visual pass rate.The full model achieved its best gated result at 0.0695, while IE alone achieved raw NDTW of 0.0170.
5 Conclusion and Limitations
WorldEcho reveals degradation under feasible off-expert actions, while WorldSync improves rollout reliability and iterative policy improvement across RoboTwin and real-robot experiments. Long-horizon evaluation across diverse embodiments and open-world environments remains an important challenge.
- WorldEcho evaluates action-conditioned world models beyond expert demonstrations using visual integrity and action-induced trajectory alignment.
- Evaluated models consistently degrade under feasible off-expert actions, exposing failures missed by expert-only protocols.
- WorldSync combines distributional coverage, representational grounding, and intervention-effect alignment to improve action-conditioned generation.
- Across RoboTwin and real-robot experiments, WorldSync produced more reliable rollouts and greater gains during iterative policy improvement.
- Comprehensively probing long-horizon interactions across diverse embodiments and open-world environments remains a shared challenge for future work.