PaperlayerView exact revision on arXiv

Source-linked AI summary

WorldEval: World Model as Real-World Robot Policies Evaluator

Yaxuan Li, Yichen Zhu, Junjie Wen, Chaomin Shen, Yi Xu

arXiv:2505.19017v1cs.ROcs.CVcs.LG

TL;DR

Evaluating generalist robot manipulation policies in real-world settings is difficult at scale. WorldEval uses Policy2Vec’s policy-network latent representations to generate action-following videos for policy evaluation, and experiments show a strong correlation with real-world performance.

  • Problem

    Evaluating generalist robot manipulation policies across many tasks in real-world settings is challenging, and policy-evaluation world models must capture how robot actions affect behavior.

    Source: §1 · p. 2, §3.2 · p. 4

  • Method

    Policy2Vec uses manipulation-policy latent representations as action encodings in a pretrained video-generation model, which WorldEval integrates into an automated policy-evaluation suite.

    Source: §1 · pp. 2–3

  • Results

    Experiments with multiple publicly available manipulation policies show a strong correlation between WorldEval measurements and performance in real-world experiments.

    Source: §1 · p. 3

  • Takeaways & Limitations

    WorldEval’s relative performance rankings can provide practitioners an accessible metric to inform iterative improvements in policy design.

    Source: §3.1 · p. 4

    WorldEval is not intended to replace real-world evaluations or perfectly replicate policy behavior, given discrepancies between generated video and actual performance conditions.

    Source: §3.1 · p. 4

Abstract

from arXiv · show

The field of robotics has made significant strides toward developing generalist robot manipulation policies. However, evaluating these policies in real-world scenarios remains time-consuming and challenging, particularly as the number of tasks scales and environmental conditions change. In this work, we demonstrate that world models can serve as a scalable, reproducible, and reliable proxy for real-world robot policy evaluation. A key challenge is generating accurate policy videos from world models that faithfully reflect the robot actions. We observe that directly inputting robot actions or using high-dimensional encoding methods often fails to generate action-following videos. To address this, we propose Policy2Vec, a simple yet effective approach to turn a video generation model into a world simulator that follows latent action to generate the robot video. We then introduce WorldEval, an automated pipeline designed to evaluate real-world robot policies entirely online. WorldEval effectively ranks various robot policies and individual checkpoints within a single policy, and functions as a safety detector to prevent dangerous actions by newly developed robot models. Through comprehensive paired evaluations of manipulation policies in real-world environments, we demonstrate a strong correlation between policy performance in WorldEval and real-world scenarios. Furthermore, our method significantly outperforms popular methods such as real-to-sim approach.

1 Introduction

WorldEval uses a world model to evaluate real-world-trained manipulation policies through generated videos, aiming to make evaluation scalable, reproducible, and safer before deployment. Its Policy2Vec approach uses policy-network representations to guide video generation, and experiments report correlation with real-world performance.

Introduction

  • Evaluating many tasks, comparing predecessors or checkpoints, and detecting hazardous behavior make real-world-only testing impractical and potentially harmful to operators, equipment, and robots.The passage gives a thousand distinct tasks as an example of an impractical evaluation load.Source: §1 · p. 2
  • WorldEval frames world-model simulation as an automated way to evaluate and rank real-world-trained policies, complementing real-world assessments; prior work used video generation mainly as an auxiliary or pretraining objective [1] [2] [3] [4] [5].The introduction poses evaluation and ranking as a distinct question from improving manipulation performance.Source: §1 · p. 2
  • Direct policy actions or explicitly learned high-dimensional action encodings can be ambiguous and difficult to visualize, while unfamiliar object physics limits scalability; such encoding is common in driving and games [6] [7].Robot actions such as incremental joint movements are semantically ambiguous, and predicted actions can be hard to represent in generated video.Source: §1 · p. 2
  • Policy2Vec injects policy-network latent action representations into a pretrained video model, avoiding separate action encoders and using generated videos to distinguish effective from ineffective policies.The policy network itself serves as the action encoder, tailoring the world model to each evaluated policy.Source: §1 · p. 2
  • WorldEval’s measurements strongly correlate with real-world performance, and FID rapidly ranks policies on simple tasks while evaluations extend to new environments and novel objects.Experiments cover Diffusion Policy [8], OpenVLA [9], DexVLA [10], and π0 [11].Source: §1 · p. 3

2 Related Work

Related work covers robot-policy evaluation, action representations, world models, and imitation learning; WorldEval is positioned as an online world-simulator evaluator addressing limits in existing evaluation approaches.

Evaluation for robot policies

  • Existing robot-policy evaluation uses real-world or simulation tests [12] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24], SIMPLER [25], or laboratory AutoEval [26], but faces cost, setup, embodiment, and scale limits.WorldEval is presented as enabling online evaluation regardless of task complexity or robot embodiment.Source: §2 · p. 3

Robot action representation

  • Robot actions are represented through discretization [27], function approximation [8] [28] [29] [30] [31] [33], latent spaces [34], or language-token sequences [35, 9, 36, 37]; this work uses the policy network itself as the action representation.The approach explores function approximation to improve the world model as a robust evaluator.Source: §2 · p. 3

World model for robotics

  • Robot world models support navigation and manipulation [38] [39] [40] [6] [43] [44] [1] [2] [3] [4] [45] [46] or policy learning via generated video [47, 48, 49, 50, 51, 52, 53, 54]; 1X-world proposed policy evaluation without explaining how.Unlike work using video prediction or world models to improve policies, WorldEval trains a world simulator for online evaluation of real robot policies.Source: §2 · p. 3

Imitation learning for robotic control

  • Imitation-learning methods, particularly vision-language-action models, attract robotics interest for scalability, efficiently learn within-domain tasks, and adapt to numerous out-of-domain scenarios, while real-world evaluation is time-consuming and inadequate in novel environments.The work proposes a world simulator for comprehensive and efficient policy evaluation.Source: §2 · p. 3

3 WorldEval: World Simulator for Evaluation of Robot Manipulation

WorldEval uses an action-conditioned world simulator to compare robot policies by relative performance, targeting correlation with real-world rankings rather than exact behavioral replication. Policy2Vec conditions generated videos on policy latent actions, while an automated verifier judges task success.

3.1 Problem Formulation

  • WorldEval aims to make simulator-based policy rankings strongly correlate with real-world rankings, not replace real tests or reproduce behavior one-to-one.The simulator is intended to provide practitioners with an accessible metric for iterative policy improvement.Source: §3.1 · p. 4

3.2 Policy2Vec: Action-Conditioned Robot Video Generation

  • Unlike instruction-only video generation followed by inverse dynamics, Policy2Vec directly accepts robot actions to generate corresponding realistic video outputs.The central challenge is constructing a world model that captures real-world behavior well enough for policy evaluation.Source: §3.2 · p. 4
  • Policy2Vec extracts latent action embeddings from a policy’s visual, language, and action model, then uses them with the instruction and initial frame to guide future video frames.The embeddings encode the policy’s internal representations and intended behavior.Source: §3.2 · pp. 4–5
  • For WAN 2.1, policy latent actions are projected, scaled by learnable α, and added to language embeddings before the fused signal enters DiT; LoRA [85] fine-tunes a 14B-scale image-to-video model.The policy network is not required during fine-tuning: captured latent actions, initial observations, and language instructions form a dataset; tuning primarily trains the projection layer and lightly fine-tunes DiT-block linear layers.Source: §3.2 · p. 5

3.3 Automated Policy Verifier for Policy Success

  • Gemini-2.0 [86] evaluates generated videos with prompts as a success detector approximating binary ground-truth task-success labels from image states.For example, it can answer whether an apple was placed on a plate based on the generated video.Source: §3.3 · p. 5

4 Experiments

Across five real-world manipulation tasks, WorldEval’s policy rankings correlate with real performance and outperform real-to-sim evaluation. Policy2Vec improves action encoding, while generated videos also support action-fidelity checks, safety screening, and lightweight evaluation on simpler tasks.

4 Experiments

  • Experiments cover five tasks, with 40 rollouts per task using the same objects and more than 1,000 real-world test trials.The tasks are Bussing Table, Collect Toy, Place Cup, Handover Block, and Strike Block.Source: §4 · p. 6

4.2 Comparison with Real-to-Sim Evaluation

  • WorldEval achieves lower MMRV and higher Pearson r than real-to-sim, which remains weaker despite efforts to bridge visual gaps between matching-embodiment simulation and reality.A direct SIMPLER comparison was infeasible because of equipment and robot constraints; instead, the authors applied SIMPLER’s real-to-sim techniques in RoboTwin across three tasks.Source: Table 1 · p. 7, §4.2 · pp. 7–8

4.3 More Experiments and Ablation Study

  • Policy2Vec outperforms VQVAE and one-hot across reported metrics, reducing MMRV by 0.1 versus VQVAE and 2.24 versus one-hot encoding.Table 2 reports Pearson correlation, MMRV, FID, and eventual rank across five tasks.Source: Table 2 · p. 7, §4.3 · p. 9

4.1 WorldEval Show Strong Performance Correlations with Real-World Evaluations

  • WorldEval rankings correlate robustly with real-world performance across four policies: DexVLA and π0 score highly, while OpenVLA scores lower in both evaluations.The study reports low MMRV and high Pearson r, supporting WorldEval as a proxy for real-world policy evaluation.Source: §4.1 · p. 7, Figure 4 · p. 7

4.3 More Experiments and Ablation Study

  • WorldEval flags hazardous actions: when DexVLA raises both arms straight upward, its generated video collapses into a mosaic rather than meaningful output.The authors hypothesize that poorly learned action features cause the collapse and present it as a safeguard against dangerous deployment behavior.Source: Figure 6 · p. 8, §4.3 · p. 9
  • Generated videos track π0’s real actions across successful and failed tasks, including Collect Toy with an unseen instruction, Pikachu object, and task configuration.The novel Collect Toy scenario is described as absent from the training dataset; its generated video mirrors the real robot’s behavior.Source: Figure 5 · p. 8, §4.3 · p. 9
  • FID correlates with real-world success on most tasks, especially simple ones, but its correlation falls significantly on intricate Table Busing, limiting it to preliminary evaluation there.The authors suggest using automated methods such as Gemini-2.0 for more complex tasks.Source: §4.3 · pp. 9–10, Figure 7 · p. 9

5 Conclusion

WorldEval uses Policy2Vec to generate action-following videos and evaluate robot policies with strong correlation to real-world performance, while supporting scalable, risk-free assessment.

5 Conclusion

  • WorldEval’s assessments correlate strongly with real-world performance and outperform conventional real-to-sim approaches.The conclusion also reports generalizability to novel environments and objects.Source: §5 · p. 10
  • WorldEval ranks policies, identifies checkpoint improvements, and detects unsafe behaviors without physical deployment.Its world-model evaluations synthesize policy behaviors in a controlled, risk-free environment.Source: §5 · p. 10
  • Policy2Vec uses latent representations of policy networks to generate action-following videos, bypassing explicit action encoding and high-dimensional inputs.Source: §5 · p. 10
  • World models offer a robust, automated alternative to real-world evaluation challenges in scalability, safety, and reproducibility.Source: §5 · p. 10
  • WorldEval generalizes to novel environments and objects.Source: §5 · p. 10

A Limitations

WorldEval remains limited by action-only control inputs, persistent video artifacts, constrained adaptability to novel scenarios, and imperfect capture of latent actions.

A Limitations

  • The framework uses only policy action outputs to control video generation; adding language instructions or intermediate reasoning may better bridge simulated evaluations and real-world performance.The proposed additions are potential ways to address this gap.Source: A Limitations · p. 17
  • Persistent artifacts—especially for low-performing policies—include object deformation, unnatural dynamics, hallucinated appearances or disappearances, overexposure, and robotic-arm ghosting.The passage attributes these artifacts as possibilities to insufficient data coverage for low-success scenarios, generative-model limitations, or inadequately detailed control inputs.Source: A Limitations · p. 17
  • Reliance on pretrained generative models may limit adaptability in significantly novel scenarios; robustness to unseen tasks and environments requires further research.Potential directions include architectural refinements and additional fine-tuning or retraining.Source: A Limitations · p. 17
  • The current method does not fully capture latent actions in generated videos, and the authors believe training on a large-scale dataset could help mitigate this issue.Source: A Limitations · p. 17

B More Experiments

The experiments show that WorldEval’s FID tracks policy changes across training checkpoints, while its evaluation remains aligned with real-world performance on novel tasks and in unseen environments.

WorldEval performance across policy checkpoints

  • Across three Bussing-table tasks, success rate rose and FID fell with training, while FID correlated strongly with real-world success and may screen simple tasks.Training-stage action distributions differ, affecting generated video quality; the video model is fine-tuned with latent actions from the final policy checkpoint.Source: B More Experiments · p. 17, Table 3 · p. 17

Effective Prediction of Policy Robustness Under Distribution Shifts

  • On the “collect toy” task, involving unseen objects and instructions, WorldEval retained robust video reconstruction and previously observed alignment with real-world testing.These findings extend evaluation to novel tasks combining unfamiliar objects and instructions.Source: B More Experiments · p. 17
  • MMRV was 0.047 across office, living-room, and kitchen tasks versus 0.044 in-domain, with Pearson r = 0.927.The tasks involved placing a cup, striking a block, and handing over a block; MMRV was significantly higher than typical real-to-simulation approaches.Source: B More Experiments · pp. 17–18

C More on Experimental Setup

The setup details video-model training, a visual domain-bridging method for Robotwin, and five manipulation tasks used in experiments.

Implementation details

  • The WAN2.1 14B image-to-video model was trained on 1,400 real-world robot trajectories, with generated videos using 50 diffusion steps.Training used a 1e-4 learning rate, batch size 8, LoRA alpha and rank 16, and 30 epochs; trajectories were sampled three times into 81-frame, 480 × 480 samples, and training took about 11 hours on 8 H800 GPUs.Source: C More on Experimental Setup · pp. 18–19

Evaluation on Robotwin

  • For Robotwin evaluation, simulation observations are transformed into more realistic images with MidJourney before being fed to the policy.This visual-model approach addresses the simulation-to-real visual gap, unlike SIMPLER’s texture modification; the policy then generates the next action sequence.Source: C More on Experimental Setup · p. 19

More description on tasks

  • The experiments cover five tasks: Bussing Table, Collect Toy, Place Cup, Handover Block, and Strike Block.They test sorting and disposal, collecting unseen toys in an out-of-domain setup, cup placement, coordinated two-arm block handover, and hammer strikes on a small block. Bussing Table uses 100 trajectories; Place Cup, Handover Block, and Strike Block use 50 each, with object positions randomized for Bussing Table evaluation.Source: C More on Experimental Setup · p. 20

D More Visualization

The section visualizes success and failure cases and shows that WorldEval captures how data collection frequency affects generated robot videos.

D More Visualization

  • Figures 8 and 9 show real-world images alongside generated-video success and failure cases.Source: D More Visualization · p. 20
  • With equal-trajectory datasets and one WorldEval model, 50Hz-trained policies produce smoother, less segmented actions and slower, more fluid videos than 10Hz-trained policies.The frequency comparison further demonstrates that WorldEval interprets latent actions.Source: D More Visualization · p. 20

Related papers

Loading 2505.19017v1…