Source-linked AI summary
dWorldEval: Scalable Robotic Policy Evaluation via Discrete Diffusion World Model
Yaxuan Li, Zhongyi Zhou, Yefei Chen, Yaokai Xue, Yichen Zhu
TL;DR
Scalable robotics policy evaluation is difficult because existing world models can mishandle erroneous actions and physical interactions. dWorldEval uses discrete diffusion with unified multimodal tokens, sparse keyframe memory, and joint progress prediction; experiments report improved controllability and success-rate alignment with real execution across simulation and real-robot settings.
Problem
Existing world-model evaluators struggle with out-of-distribution actions and physical consistency, limiting their reliability as robotics policy-evaluation proxies.
Method
dWorldEval trains a discrete-diffusion world model that jointly represents visual observations, language, actions, keyframe history, and progress tokens in a unified token space.
Results
dWorldEval substantially improves action controllability, while predicted success rates correlate strongly with actual execution performance with Pearson r ≈0.9.
Takeaways & Limitations
Joint action-token modeling, sparse memory, and progress prediction bring scalable policy evaluation closer to practice and enable accurate policy ranking across diverse architectures.
Takeaways & Limitations
Without keyframe memory, generated videos can suffer severe drift, making progress-token predictions unreliable.
Abstract
from arXiv · showhide
Evaluating robotics policies across thousands of environments and thousands of tasks is infeasible with existing approaches. This motivates the need for a new methodology for scalable robotics policy evaluation. In this paper, we propose dWorldEval, which uses a discrete diffusion world model as a scalable evaluation proxy for robotics policies. Specifically, dWorldEval maps all modalities - including vision, language, and robotic actions - into a unified token space, modeling them via a single transformer-based denoising network. In this paper, we propose dWorldEval, using a discrete diffusion world model as a scalable evaluation proxy for robotics policy. Specifically, it maps all modalities, including vision, language, and robotics action into a unified token space, then denoises them with a single transformer network. Building on this architecture, we employ a sparse keyframe memory to maintain spatiotemporal consistency. We also introduce a progress token that indicates the degree of task completion. At inference, the model jointly predicts future observations and progress token, allowing automatically determine success when the progress reaches 1. Extensive experiments demonstrate that dWorldEval significantly outperforms previous approaches, i.e., WorldEval, Ctrl-World, and WorldGym, on LIBERO, RoboTwin, and multiple real-robot tasks. It paves the way for a new architectural paradigm in building world simulators for robotics evaluation at scale.
1 Introduction
Existing world-model evaluators struggle to reflect out-of-distribution robot actions and maintain physical consistency, limiting scalable policy evaluation. dWorldEval addresses these issues with unified token modeling, sparse keyframe memory, and joint progress prediction, and reports stronger controllability and policy-ranking alignment.
- World models offer a scalable alternative to costly real-world execution or asset-heavy simulations for evaluating robot policies.
- Existing evaluators often ignore erroneous out-of-distribution actions and produce physically inconsistent artifacts such as warped or vanishing objects.
- dWorldEval maps visual observations, language instructions, and action chunks into a unified token space and models them jointly with self-attention.
- Sparse keyframe memory maintains spatiotemporal consistency by mitigating long-horizon drift, while a discrete progress token quantifies task completion.
- dWorldEval achieves substantially better action controllability and estimated success rates correlate strongly with execution performance, with Pearson r ≈0.9.
2 Related Work
Prior policy-evaluation world models largely adapt video-generation architectures that treat actions as auxiliary conditions. Discrete diffusion provides an alternative foundation for jointly modeling multimodal robotic inputs and outputs.
- World models aim to scale policy assessment beyond real-world rollouts and physics-based simulators, but current video-based evaluators have limited reliability.
- WorldGym, Ctrl-World, and WorldEval inject actions into visually dominated denoisers through mechanisms such as AdaLN modulation or cross-attention.
- Discrete diffusion has been extended from language modeling to multimodal generation and robotic policy learning through token-based action prediction.
3 Methodology
dWorldEval imagines policy rollouts by predicting future visual states and task progress from language, observations, history, and future actions. Its unified tokenization, sparse keyframe memory, and masked discrete diffusion jointly support controllable, consistent, and automatically scored rollouts.
- 3.1 Problem Formulation: The world model predicts a future visual outcome and task progress score from a language instruction, current observation, history, and future actions.
- 3.1 Problem Formulation: Reliable evaluation requires action controllability, spatiotemporal consistency, and discriminative task completion.
- 3.1 Problem Formulation: An imagined rollout closes the loop between policy π and world model Wθ, and performance is measured by the imagined success rate over generated trajectories.
- 3.2 World Modeling via Discrete Diffusion: Specialized tokenizers serialize RGB observations, language, and continuous action chunks into one discrete sequence so a transformer models their joint distribution.
- 3.2 World Modeling via Discrete Diffusion: Sparse keyframe memory samples recent frames at a fixed stride and preserves temporal order with absolute frame-index tokens.
- 3.2.3 Discrete Progress Token for Automatic Success Detection: Progress scores are converted into discrete tokens and jointly generated with visual outcomes, enabling automatic success computation when decoded scores reach the task threshold.
- 3.3 Joint Visual-and-Progress Denoising: Masked Discrete Diffusion masks the target suffix while retaining context, then iteratively parallel-decodes future visual states and progress scores.
4 Experiments
The experiments test dWorldEval’s action controllability, long-horizon consistency, automatic success scoring, and policy-ranking reliability across simulated and real-world settings. Results show that keyframe memory and action-faithful generation support strong agreement with actual policy performance.
- Experimental Questions: The evaluation spans five research questions covering action adherence, memory-based consistency, intrinsic success scoring, policy ranking, and the diagnostic value of ∆-LPIPS.Experiments use LIBERO, RoboTwin, and real-world tasks, with multiple baselines and heterogeneous policies.
- Automatic Policy Scoring: Jointly predicted progress scores closely track real execution and human judgment, including non-monotonic performance changes across policy checkpoints.The score exhibits sharp transitions at task completion and is used for automatic success or failure detection.
- Policy Evaluation: dWorldEval ranks policies reliably across settings, with correlations of r = 0.910 on LIBERO multi-view, r = 0.927 on RoboTwin, and r = 0.918 in the real world.On single-view LIBERO, it achieves MMRV = 0.013, compared with baseline rank violations up to 0.039.
- Cross-Platform Robustness: Across camera configurations and robot morphologies, dWorldEval maintains ∆LPIPS scores of approximately 0.31–0.36, while video-diffusion baselines worsen with longer horizons.The reported degradation reflects both spatiotemporal drift and insufficient adherence to action sequences.
5 Conclusion
dWorldEval is a discrete diffusion world model designed to improve the reliability of robotic policy evaluation. Its action-aware, memory-augmented rollouts better track real-world execution and support policy ranking across architectures.
- dWorldEval unifies action tokens with sparse keyframe memory to generate consistent long-horizon rollouts.The paper quantifies rollout consistency with the proposed ΔLPIPS metric.
- dWorldEval significantly enhances controllability, with predicted success rates closely tracking real-world execution.
- Accurate policy ranking across diverse architectures brings scalable robotic policy evaluation closer to practice.
Appendices
This appendix section introduces the additional experimental details covered in the paper. It focuses on task definitions, data collection pipelines, and model hyperparameters.
- The section provides further details on task definitions, data collection pipelines, and model hyperparameters.
A.1 Detailed Task Descriptions
The experiments cover real-world AgileX manipulation, RoboTwin contact-rich tasks, and LIBERO tabletop tasks. They include diverse bimanual, precision, dynamic, synchronization, stacking, placement, and spatial-relation challenges.
- Domains: dWorldEval is evaluated across Real-World AgileX, RoboTwin, and LIBERO domains.
- Real-World Tasks: The AgileX evaluation uses five bimanual tasks, including sorting, precision placement, handover, tool striking, and synchronized bottle lifting.
- RoboTwin Stacking & Placement: RoboTwin evaluates 10 contact-rich tasks organized into Precision Stacking and Constrained Placement categories.
- LIBERO Suites: LIBERO evaluation uses the Object, Spatial, Goal, and LIBERO-100 suites, covering tabletop manipulation such as drawer opening and obstacle-aware rearrangement.
- Action Corruption Tests: Figure 9 tests action corruption by replacing action chunks with chunks from other episodes and tracks controllability and success-rate correlation as corruption increases.
- Implementation: The model uses K = 4 sparse 256 × 256 keyframes and a prediction horizon aligned with the action chunk size.
- Implementation: Success rates average 20 episodes per simulation task and 30 episodes per real-world task, with full trajectories taking approximately 30–90 seconds.
B Verifying Causal Dependency via Action Shuffling
The action-shuffling experiment tests whether dWorldEval depends on the supplied action sequence rather than static visual priors. Increasing action misalignment degrades controllability metrics and policy-ranking reliability.
- Experimental Protocol: The experiment deliberately disrupts action–outcome alignment while preserving marginal action distributions.
- Results: At p = 1, shuffled actions consistently degrade action-controllability indicators, showing sensitivity to the action input.
- Results: Increasing swap probability p worsens ΔLPIPS and sharply reduces ranking correlation with real success rates.
- Results: Accurate policy ranking is achievable only in the low-ΔLPIPS regime, making adherence to input actions essential for reliable evaluation.
C VLM Supervision Details
The evaluation pipeline uses few-shot VLM supervision to assign discrete progress scores from independently assessed frames. Batch processing and explicit phase definitions stabilize scoring for manipulation tasks.
- VLM Supervision: Few-shot in-context learning aligns the VLM’s progress-scoring distribution with human intuition.The prompts include task definitions, scoring rules, and anchor examples with pre-labeled intermediate scores.
- Sanity Check: Action shuffling degrades action-controllability indicators by breaking action–outcome alignment while history and language remain fixed.This sanity check tests whether predicted outcomes respond to the supplied future action chunks.
- VLM Supervision: The VLM evaluates batches of query frames independently, typically using 10 frames per query.This avoids relying on frame order or inferred temporal context during scoring.
- VLM Supervision: Batch processing stabilizes outputs and enforces adherence to discrete scoring criteria.The scoring procedure returns values from 0 through 1.0 in fixed increments of 0.2.
- Progress Scoring: For Libero-Object pick-and-place tasks, scoring criteria represent sequential manipulation phases and subtle state changes.Few-shot examples help distinguish stages such as approach, lifting, transit, pre-placement, and completion.
- Progress Scoring: Progress labels range from 0.0 for idle or start to 1.0 for stable placement and task completion.Intermediate scores encode contact, lifting, transit, and pre-placement states.
C.2 Visualizing Progress Scores: Labels vs. Generation
The progress-score visualizations compare VLM ground-truth labels with dWorldEval’s generated scores across successful and failed manipulation trajectories.
- Labels vs. Generation: Ground-truth scores rise stepwise as manipulation milestones are achieved and stall at a low value for failed execution.The examples use the LIBERO-Object task of picking up alphabet soup and placing it in a basket.
- Labels vs. Generation: dWorldEval predicts a successful rollout whose progress score rises accurately to 1.0 and a failure case whose score remains low.The generated scores are described as consistent with the corresponding visual outcomes.
D Visualizing Baseline Consistency
The visualizations assess consistency across LIBERO, RoboTwin, and real-world settings by comparing generated rollouts with initial scenes, simulations, and physical executions.
- Baseline Consistency: At H = 20, dWorldEval restores the initial LIBERO scene structure at terminal step t = 2HΔ, unlike WorldEval and WorldGym.The baselines show significant visual deviation from the initial state.
- Baseline Consistency: The baseline round-trip failures combine spatiotemporal drift with poor adherence to the inverse action sequence.The observed deviations include hallucinated objects and failure to return to the starting configuration.
- Progress Visualization: Generated progress scores reach 1.0 for successful rollouts and remain low for failure cases, matching visual outcomes.These examples complement the consistency visualizations by linking predicted completion to generated trajectories.
- Simulation Visualization: On RoboTwin, dWorldEval generates synchronized videos across top-down, left-wrist, and right-wrist views while preserving object details and spatial layout.The generated videos are compared with ground-truth simulation trajectories.
- Real-World Visualization: In real-world scenarios, dWorldEval generates synchronized three-view videos that preserve object details amid visual complexity.The comparison uses ground-truth physical robot executions conditioned on language instructions.