RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstruction

Motivation

• Existing video, general world-model, and embodied-world-model benchmarks do not jointly measure agreement with ground-truth 3D scene state and motion executability under a consistent taxonomy.Prior metrics can reward perceptual quality despite physically impossible or contextually incorrect predictions, while existing 3D measures assess internal consistency or downstream reasoning rather than reference-scene agreement.Source: §1 · pp. 1–2

Benchmark protocol

• RoboPhys-3D compares generated and ground-truth videos within the same reconstructed 3D scene, isolating generation-induced error from reconstruction-induced error.The benchmark covers 50 manipulation tasks across four regimes, with 100 episodes per task rendered from five camera views, and decodes generated rollouts into executable actions for simulator replay.Source: §1 · pp. 2–3

Evaluation scores

• Average Full Score consolidates all 50 normalized metrics, while RoboPhyscore averages non-task-success metrics strongly correlated with task-success measurements.AFS provides a comprehensive diagnostic summary, whereas RoboPhyscore is designed as a compact, task-aligned capability indicator and is validated against human judgment.Source: §1 · pp. 2–3

Evaluation framework

• The framework organizes 50 complementary metrics into 18 sub-dimensions spanning four levels: pixel fidelity, 3D geometry consistency, state understanding, and task completeness.Together, these levels characterize visual, geometric, physical, semantic, and execution-level behavior.Source: §1 · p. 3

2.1. 3D & 4D Reconstruction

• 3D reconstruction progresses from SfM [49] and MVS [27] to NeRF [45], 3DGS [29], feed-forward methods [68], and dynamic 4D representations [73] [89].These methods recover geometry, camera parameters, novel views, or time-varying scene structure from images or video.Source: §2.1 · p. 3
• Reconstruction evaluation combines rendering metrics such as PSNR, SSIM, and LPIPS [86] with geometric measures including Chamfer distance [11], depth error [59], and camera-pose accuracy [35].Recent work also probes video-world-model geometric coherence, which RoboPhys-3D extends to actionable robotic manipulation scenes.Source: §2.1 · p. 3

2.2. Video World Model

• Video world models generate action- or instruction-conditioned future rollouts for interactive control, planning, data synthesis, and policy learning, with EWMs including Cosmos [1], RoboDreamer [90], and EnerVerse [20].The field has progressed from U-Net diffusion models to scalable flow-matching DiTs [77] producing longer, higher-resolution, temporally coherent videos.Source: §2.2 · pp. 3–4
• Existing EWM benchmarks cover perception, prediction, generalization, reconstruction consistency, and robot-action decoding, but lack consistent taxonomies and same-scene reconstruction-based comparison with ground-truth manipulation motion.General video and GWM benchmarks omit action inputs, interaction, or manipulation-scene geometric agreement, while prior embodied benchmarks generally do not jointly assess planning and execution.Source: §2.2 · p. 4

3. RoboPhys-3D Dataset

• The dataset pipeline collects multi-view videos, generates scene annotations and prompts, reconstructs videos in 3D, and conditions candidate world models on an initial frame and prompt.The final evaluation contains 145,000 videos: 5,000 ground-truth, 20,000 reconstructed, and 120,000 generated.Source: §3 · p. 4

3.1. Problem Formulation

• Each instance represents autoregressive video generation from an initial frame and view-aware prompt, with ground-truth actions and privileged simulator scene state defining the target.Actions encode end-effector position, orientation, and gripper state, using d=7 for single-arm and d=14 for dual-arm regimes.Source: §3.1 · p. 4

3.2. Ground-Truth Data Collection

• RoboPhys-3D retains 50 RoboTwin 2.0 tasks with 100 episodes each, producing 5,000 episodes and 25,000 five-view ground-truth videos.Tasks vary by single- versus dual-arm manipulation and by whether success requires placement at a designated target.Source: §3.2 · pp. 4–5

3.3. Prompt Construction and Scene Annotation

• View-aware captions from Qwen3-VL [3] augment native instructions [7] with observable gripper, approach, and trajectory details while separating static appearance from temporal dynamics.Manual review removes hallucinated or physically ambiguous captions, and each episode retains three conditioning levels for controlled specificity studies.Source: §3.3 · p. 5

3.4. 3D Reconstruction

• Four reconstruction methods process every main-perspective video identically, while ground-truth residuals calibrate reconstruction error and method disagreement exposes artifacts affecting 3D evaluation.The methods are VGGT [68], VGGT-Ω, 4DGS [73], and 4C4D [89].Source: §3.4 · p. 5

3.5. Video Generation

• Four open-source models—Wan 2.2 [66], CogVideoX 1.5 [77], Cosmos 3 [1], and RoboDreamer [90]—generate future frames that are reconstructed and decoded into executable actions.Episodes use sequence-wise 70%/15%/15% train-validation-test splits, matched output settings, and simulator replay of IDM-decoded actions.Source: §3.5 · p. 5

4. RoboPhys-3D Evaluation

• RoboPhys-3D evaluates generated rollouts at four levels: pixel fidelity, 3D geometric consistency, state-level understanding, and task-level completeness.Most metrics are computed automatically, while embodied task success is evaluated separately.Source: §4 · p. 5

4.1. Four-Level Metric Taxonomy

• The taxonomy covers 50 metrics across image and motion quality, scene geometry, object and trajectory state, physical and semantic adherence, instruction following, collision safety, and task success.Task success evaluates direct action replay, a π0.5 policy trained on synthetic rollouts, and VLM-judged completion.Source: §4.1 · pp. 5–6

4.2. Average Full Score

• AFS hierarchically averages all 50 normalized metrics, equalizing sub-dimensions, levels, and the four task regimes while assigning each evaluation level 25%.This prevents metric-rich sub-dimensions or levels from dominating the overall score.Source: §4.2 · p. 6

4.3. RoboPhyscore

• RoboPhyscore averages eight metrics whose Pearson correlations exceed 0.70 with each of the Action Planner, Data Engine, and VLM-2 success measurements.The selected metrics are JEPA Similarity, Semantics Consistency, Subject Adherence, Normalized Score, Average Recall, Physical Commonsense, nDTW, and FVD.Source: §4.3 · p. 6

5.1. Setup

• RoboPhys-3D scores four open-source models on raw videos and four 3D reconstruction substrates, with Ground Truth processed identically as a reference ceiling.The evaluation also varies three scene annotations and three inverse dynamic models.Source: §5.1 · pp. 6–7

5.2.1. Per-Dimension Analysis

• 0.6330 RoboPhyscore makes Cosmos the strongest model, followed by Wan (0.5359), RoboDreamer (0.3492), and CogVideoX (0.3363), at 92.7% of Ground Truth.Cosmos ranks first under every reconstruction method.Source: Figure 2 · p. 7, Table 2 · p. 7, §5.2.1 · p. 7
• Reconstruction retains 94.7% and 96.1% of Ground Truth for VGGT and VGGT-Ω, but only 85.4% and 78.2% for 4C4D and 4DGS.Perceptual quality can diverge from grounded correctness: Wan reaches 44.2% of Ground Truth in distribution similarity despite higher aesthetic and motion quality, while CogVideoX’s geometry score is 3.0919 versus 2.9145 for Ground Truth.Source: Table 3 · p. 7, §5.2.1 · p. 7, Table 4 · p. 8

5.2.2. Per-Task Analysis

• 0.6464 Ground Truth RoboPhyscore on dual-arm tasks with a target is lower than 0.7117 on single-arm tasks without a target, and Task 4 is hardest for every model.Cosmos retains 91.0% of Ground Truth in Task 4, compared with 73.6% for Wan; dual-arm, target-directed coordination drives the clearest degradation.Source: Table 5 · p. 8, §5.2.2 · p. 8

5.2.1. Per-Dimension Analysis

• Cosmos retains 85.9% of Ground Truth in execution-grounded task success, while CogVideoX reaches only 16.5% despite achieving the highest collision safety at 0.9650.Cosmos’s state-level object localization and trajectory accuracy retain 61.2% and 73.2%, respectively, showing that plausible interactions can conceal spatial and dynamical errors.Source: §5.2.1 · p. 8

5.2.3. Ablation Study

• View-aware prompting raises Wan’s RoboPhyscore from 0.4962 to 0.5782, a 16.5% improvement, while Cosmos gains 4.3%; DreamGen records MAE = 0.0112 and Action Planner success of 0.9825.DreamGen also achieves Data Engine success of 0.5875 and is adopted for execution-grounded evaluations.Source: Table 6 · p. 8, Table 7 · p. 8, §5.2.3 · p. 8

5.2.4. Cross-Dimension Analysis

• RoboPhyscore agrees strongly with human evaluation, reaching Pearson r = 0.9761 and Spearman ρ = 0.8962, while outperforming alternative unified scores.Human judgments came from 70 annotators rating coherent, physically plausible, task-executing interactions on a 10-point scale.Source: §5.2.4 · p. 8, Figure 3 · p. 9

5.3. Qualitative Results

• In the Adjust Bottle examples, Cosmos preserves embodiment and viewpoint while completing the task, whereas Wan, CogVideoX, and RoboDreamer show embodiment, completion, or visual-structural failures.The comparisons use the same view-aware instruction across evaluated world models and Ground Truth.Source: Figure 4 · p. 9, §5.3 · p. 9

6. Conclusion

• 50 tasks across four regimes are evaluated by comparing generated rollouts with ground-truth manipulation trajectories in a shared reconstructed 3D scene.The benchmark is built on RoboTwin 2.0.Source: §6 · p. 9
• 0.6330 RoboPhyscore makes Cosmos 3 the top model, reaching 92.7% of ground truth across every reconstruction method and task regime.This comparison spans four representative world models.Source: §6 · p. 9
• 50 metrics across 18 sub-dimensions and four complementary levels provide comprehensive diagnostic coverage through Average Full Score.RoboPhyscore complements this coverage as a compact, task-aligned indicator.Source: §6 · p. 9
• Pearson r = 0.9761 and Spearman ρ = 0.8962 show strong agreement between RoboPhyscore and human judgment.The correlations support RoboPhyscore as a task-aligned evaluation indicator.Source: §6 · p. 9
• Perceptual, VLM-based, and collision-safety scores can remain strong while object localization, trajectory accuracy, and downstream task success are substantially weaker.These results reveal a gap between plausible generation and correct embodied prediction.Source: §6 · p. 9
• 21.8% is the maximum RoboPhyscore shift caused by reconstruction quality, requiring reconstruction-induced and generation-induced errors to be separated.Without this separation, 3D-level conclusions may be confounded by reconstruction quality.Source: §6 · p. 9
• The benchmark currently covers one simulation platform and embodiment, with future extensions planned for cross-embodiment, cross-dataset, sim-to-real, and broader VLA-policy evaluation.These directions address the stated scope limitation.Source: §6 · p. 9

A.1. Dataset Statistics and Task Taxonomy

• RoboPhys-3D contains 50 manipulation tasks across four regimes, using episode-level 70%/15%/15% train/validation/test splits that keep all views and derivatives together to prevent leakage.The regimes vary by active manipulators and designated-target presence.Source: §A.1 · p. 14

A.2. Model Configurations

• The evaluation standardizes generated videos to a common resolution and 81 frames, compares models spanning compact U-Nets to large Transformers, and includes three distinct video-to-action mechanisms.Inference times are measured under identical hardware and software conditions and describe evaluation cost rather than intrinsic model efficiency.Source: §A.2 · p. 14

A.3. Prompt for Egocentric Video Caption.

• The view-aware caption prompt conditions generation on motion dynamics from a static first-person seed frame, requiring the Aloha-AgileX gripper, manipulated object, ordered trajectory, final state, and only observable events.The prompt also enforces objective captions, correct arm identification, temporal completeness, and exact JSON output.Source: §A.3 · p. 16

B.1. Pixel-Level Fidelity

• Pixel-level fidelity covers image quality, aesthetics, distribution similarity, motion quality, and content consistency through reference-based, reference-free, feature-distribution, optical-flow, and temporal-stability metrics.The suite includes PSNR, SSIM, FVD, LPIPS, Inception Score, LAION-AP, HPSv3, CLIP-AP, CLIP-IQA, JEPA Similarity, Dynamic Degree, Flow Score, Motion Smoothness, Temporal Flickering, and five consistency measures.Source: §B.1 · p. 18, §B.1.1 · p. 18, §B.1.2 · p. 18

B.2. 3D Geometric Consistency

• 3D geometric consistency evaluates reconstructed scene coherence, depth and perspective, camera trajectories, and roundtrip persistence using reconstruction, reprojection, depth, camera-control, and revisitation metrics.These measures distinguish geometric incoherence, long-horizon drift, depth-scale discrepancies, spurious camera motion, and failure to preserve revisited scenes.Source: §B.2 · p. 19, §B.2.1 · pp. 19–20, §B.2.2 · p. 20

B.3. State-Level Understanding

• State-level understanding measures object localization, trajectory accuracy, interaction quality, event and action adherence, physical-law adherence, and semantic alignment against structured scene or action requirements.Localization uses simulator masks; trajectory metrics assess shape, timing, and dynamics; interaction and action protocols evaluate contacts, grasping, completion, anomalies, and instructed behavior.Source: §B.3 · p. 20, §B.3.1 · p. 20, §B.3.2 · p. 20

B.3.2. Trajectory Accuracy

• Trajectory metrics jointly assess path deviation, temporal alignment, decision-step length, and velocity or acceleration dynamics rather than visual appearance alone.They use Mean Trajectory Length, Symmetric Hausdorff Distance, normalized Dynamic Time Warping, and Dynamic Consistency.Source: §B.3.2 · p. 20

B.3.3. Interaction Quality

• Interaction and action metrics test whether generated manipulation is physically plausible and instruction-compliant, while Physical Commonsense evaluates Newtonian motion, mass conservation, mechanics, impenetrability, and gravitation.The protocols assess contact behavior, force transmission, interpenetration, grasp correctness, event completion, detail accuracy, anomaly absence, and unnatural motion.Source: §B.3.3 · p. 21, §B.3.4 · p. 21, §B.3.5 · p. 21

B.3.6. Semantic Adherence

• Semantic adherence evaluates fidelity to the textual condition and world setting using six metrics spanning captions, step descriptions, commonsense logic, instruction execution, subject attributes, and scene consistency.The metrics include BLEU [81], CLIP Score [81], Logics [81], VLM-1 Score [59], Subject Adherence [79], and Scene Adherence [79].Source: §B.3.6 · p. 21

B.4. Task-Level Completeness

• Task-level completeness assesses whether videos decoded into executable actions by an IDM complete manipulation tasks through instruction following, collision safety, and task success.This evaluation is organized into three task-level sub-dimensions.Source: §B.4 · p. 21

B.4.1. Instruction Following

• Instruction following measures achievement of the instructed final state using Normalized Score [59] and Four-Level Score [31].Normalized Score [59] evaluates action type, target object, and task state with Qwen3-VL [3], while Four-Level Score [31] assigns levels 0–3.Source: §B.4.1 · p. 21

B.4.2. Collision Safety

• Collision safety measures unintended robot-environment and robot–self collisions during simulated execution using Collision Count [71].Collision Count [71] ranges from [0,+∞)↓, with lower values indicating fewer collisions.Source: §B.4.2 · p. 21

B.4.3. Task Success

• Task success is measured with three complementary metrics: Action Planner, Data Engine, and VLM-2 Score.Action Planner replays decoded actions with DreamGen’s IDM in RoboTwin 2.0 [7]; Data Engine measures gains from synthetic-data training; VLM-2 Score [59] judges success from generated video alone.Source: §B.4.3 · p. 22

B.5. Score Normalization

• All 50 metrics are mapped to normalized scores sm ∈[0, 1] with higher values indicating better performance, using native orientations, clipped affine maps, absolute anchors, or dedicated transforms.Thirty-three metrics require no transformation, 11 use clipped affine normalization, two use absolute anchors, and four use dedicated transforms.Source: §B.5 · p. 22

B.5.1. Upstream Benchmark Bounds

• Upstream benchmark bounds normalize flow score, motion smoothness, photometric consistency, depth accuracy, and style consistency, reversing direction for lower-is-better quantities.WorldArena [59] supplies bounds for flow, motion smoothness, photometric consistency, and depth accuracy, while WorldScore [9] supplies the style-consistency threshold.Source: §B.5.1 · p. 22

B.5.2. RoboPhys-3D Empirical Bounds

• RoboPhys-3D empirical bounds use the 1st and 99th percentiles over the evaluation pool when source benchmarks lack usable bounds.This strategy applies to inception score, CLIP-AP, reconstruction fidelity, reprojection photometric consistency, mean trajectory length, and collision count.Source: §B.5.2 · p. 22

B.5.3. Custom Normalization

• Custom normalization uses WoW-World-Eval [10] absolute anchors of UPSNR = 50 and UFVD = 2000 for PSNR and FVD, with clipped linear mappings.These mappings replace WoW-World-Eval’s nonlinear human-preference calibration.Source: §B.5.3 · p. 22
• Additional custom transforms align LPIPS as lower-is-better and normalize physical commonsense, four-level score, and scene adherence to their task-specific scales.Physical commonsense uses a 1–5 Likert scale, four-level score uses a 0–3 scale, and scene adherence has an effective maximum of 0.5.Source: §B.5.3 · pp. 22–23
• After transformation, every metric shares a common [0, 1] higher-is-better scale, with the corresponding numerical bounds reported in Table 13.Table 13 summarizes the bounds used for score normalization.Source: §B.5.3 · p. 23, Table 13 · p. 23

B.6. Metrics Classification

• Across 50 metrics, 13 are video-grounded, 12 VLM-grounded, 5 simulator-grounded, and 20 reference-free.Video-grounded metrics compare reconstruction-matched ground truth, VLM-grounded metrics use Qwen3-VL, and simulator-grounded metrics require privileged simulation information.Source: §B.6 · p. 23, Table 14 · p. 24
• Metric deployment depends on grounding: reference-free and VLM-grounded metrics transfer without ground truth, video-grounded metrics require paired references, and simulator-grounded metrics require simulation.RoboPhyscore selects four video-grounded, three VLM-grounded, and one reference-free metric.Source: §B.6 · p. 23

B.7. Analysis of Average Full Score

• AFS analyzes 18 sub-dimensions, normalized by theoretical maxima so radar axes span [0, 1] and remain comparable across native score ranges.Figures 6 and 7 visualize these sub-dimensions for world models and reconstruction methods.Source: §B.7 · p. 23

B.7.1. Video World Models

• AFS ranks Cosmos first at 2.7797 (97.9% of Ground Truth), ahead of Wan (2.5714), CogVideoX (2.2517), and RoboDreamer (2.1246).Cosmos also achieves the strongest and most balanced performance among the evaluated world models.Source: §B.7.1 · p. 23
• Cosmos retains 93.7% of Ground Truth in pixel fidelity and 92.0% in state understanding, while exceeding it in 3D geometry consistency (102.2%) and task completeness (102.7%).These level-specific results accompany its highest overall AFS.Source: §B.7.1 · p. 23
• High geometry does not ensure embodied accuracy: CogVideoX reaches the best geometry score but only 72.4% and 69.9% of Ground Truth in state and task levels, while RoboDreamer’s task completeness is 59.7%.Wan instead shows strong perceptual and semantic behavior but declines to 33.2% in object localization and 63.3% in trajectory accuracy.Source: §B.7.1 · p. 23

B.7.2. Reconstruction Methods

• 2.5598 and 2.5292 AFS values make VGGT-Ω and VGGT the best-performing reconstruction methods, retaining 97.0% and 95.8% of reference performance.Figure 7 compares the methods against the reference AFS of 2.6400.Source: §B.7.2 · p. 24
• 4C4D retains 92.8% of reference performance with an AFS of 2.4500, while 4DGS shows the largest degradation at 2.3865, or 90.4%.The reference AFS is 2.6400.Source: §B.7.2 · p. 24
• 3D geometry consistency remains comparatively stable across reconstruction substrates, whereas state-level understanding and task-level completeness degrade more substantially.This indicates that geometric stability does not guarantee recovery of the correct physical state.Source: §B.7.2 · p. 24
• Reconstructions can preserve internally consistent camera and roundtrip geometry while losing information needed to recover the correct physical state.Generation- and reconstruction-induced errors therefore manifest differently across perceptual, geometric, state-grounded, and execution-oriented dimensions.Source: §B.7.2 · p. 24

B.8. Analysis of RoboPhyscore

• Figure 8 decomposes RoboPhyscore into eight task-aligned source metrics.These source metrics are already normalized to the [0, 1] range.Source: §B.8 · p. 24
• The RoboPhyscore decomposition uses normalized source metrics, enabling the eight task-aligned components to be examined on a common [0, 1] scale.The passage specifies normalization but does not report individual component values.Source: §B.8 · p. 24

B.8.1. Video World Models

• 40.9% average recall and 52.5% nDTW of Ground Truth reveal Wan’s sharper imbalance despite comparatively strong semantic and distributional components.Source: §B.8.1 · p. 24
• CogVideoX and RoboDreamer obtain substantially lower RoboPhyscores than Cosmos.Source: §B.8.1 · p. 24
• RoboDreamer’s relatively high normalized FVD despite its low overall score shows that no single perceptual or distributional metric sufficiently characterizes embodied prediction quality.Source: §B.8.1 · pp. 24, 26
• Table 15 evaluates video world models using metrics without any external reference.Source: Table 15 · p. 25

B.8.2. Reconstruction Methods

• Figure 7 provides detailed RoboPhys-3D benchmark results for four representative reconstruction methods.Source: Figure 7 · p. 25

B.8.1. Video World Models

• 0.6330 RoboPhyscore makes Cosmos the most balanced task-aligned generated model, although its average recall is 0.4773 (67.3%) and nDTW is 0.4297 (72.4%).These deficits show that semantic and physical plausibility do not guarantee accurate state recovery or motion evolution.Source: §B.8.1 · p. 24, Figure 8 · p. 26

• The 50-metric Pearson matrix shows block structure, with strong correlations among related perceptual, geometric, state, and task metrics but heterogeneous relationships across evaluation dimensions.Figure 9 was computed across model–reconstruction configurations and complements sub-dimension-level analysis.Source: §B.9 · p. 26
• 0.965 correlation between SSIM and LPIPS and 0.992 between average precision and recall demonstrate substantial redundancy within image-quality and detection metric families.Trajectory metrics also agree strongly: HSD correlates with trajectory dynamics at r = 0.967 and nDTW at r = 0.912.Source: §B.9 · p. 26
• 0.999 correlation between action planner and data engine indicates near-equivalent task-oriented signals, while their lower correlations with VLM-2 show distinct information.Action planner and data engine correlate with VLM-2 at r = 0.869 and r = 0.876, respectively.Source: §B.9 · pp. 26–27
• −0.936 correlation between roundtrip spatial consistency and normalized score, alongside −0.974 between reprojection photometric consistency and subject action adherence, shows geometric regularity can conflict with task correctness.These inverse relationships indicate that strong reconstruction or reprojection consistency does not guarantee accurate interaction semantics, physical state evolution, or successful execution.Source: §B.9 · p. 27
• The combined redundancy and complementarity demonstrate that perceptual fidelity, geometry, state accuracy, semantic adherence, and task success are not interchangeable, necessitating complementary metrics.Combining measurements distinguishes visually plausible rollouts from physically and behaviorally correct ones.Source: §B.9 · p. 27

B.10. Threshold Sensitivity of RoboPhyscore

• Increasing the threshold progressively compresses the selected set from 19 metrics at 0.5 to 12 at 0.6, eight at 0.7, one at 0.8, and none at 0.9.At 0.9, no metric satisfies the selection criterion, making RoboPhyscore undefined.Source: §B.10 · p. 27
• At 0.7, human evaluation reaches Pearson r = 0.9761, R2 = 0.9528, and MAE = 0.0317, while VLM-2 reaches Pearson r = 0.8696, R2 = 0.7563, and MAE = 0.1611.Threshold 0.8 instead produces the highest human Spearman ρ = 0.9692 and VLM-2 Spearman ρ = 0.9505.Source: §B.10 · p. 27
• At threshold 0.8, JEPA Similarity alone reaches Action Planner r = 0.9665 and Data Engine r = 0.9616, but sacrifices RoboPhyscore’s multidimensional coverage.The same setting yields Action Planner ρ = 0.9717, R2 = 0.9341, and MAE = 0.0575; Data Engine yields ρ = 0.9675, R2 = 0.9246, and MAE = 0.0373.Source: §B.10 · p. 27

C. Supplementary Details about Qualitative Results

• The qualitative supplement compares reconstruction methods, textual conditioning, and inverse dynamics models across Figures 10–12, extending the analysis beyond the primary experimental axes.Figure 10 specifically presents Move Pillbottle Pad examples for Cosmos with different reconstruction methods and a view-aware prompt.Source: C. Supplementary Details about… · p. 27, Figure 10 · p. 30

C.2. Effect of Prompt Specificity

• View-aware prompting produces the strongest Grab Roller rollout, raising Wan’s score from 0.4962 to 0.5474 to 0.5782 across native, Physion-Eval, and view-aware conditions.The view-aware caption supplies explicit gripper laterality, approach direction, and table-contact anchors, yielding a stable two-handed grasp closest to ground truth.Source: §C.2 · p. 29, Figure 11 · p. 30

C.3. Effect of the Inverse Dynamics Model

• DreamGen’s simulator replay most closely matches the Put Object Cabinet demonstration, while J-IDM shows the largest late-episode deviations and MIDM falls between them.All three inverse dynamics models recover the approach, cabinet opening, transport, and insertion phases, but differ in end-effector precision during contact-rich segments.Source: §C.3 · p. 29, Figure 12 · p. 31

D. Supplementary Details about Human Study Protocol

• The human study recruited 70 participants and used blinded, randomized web-based ratings of coherent, physically plausible, and successfully executed robot interactions on a 10-point scale.Participants had normal or corrected-to-normal vision, were unaware of model and reconstruction identities, and evaluated pre-generated videos without sensitive personal data collection.Source: D. Supplementary Details about Human… · p. 31
• Human-evaluation samples were stratified across world models, reconstruction methods, task categories, and low, medium, and high RoboPhyscore ranges, with at least three independent ratings per sample.Ground-truth samples were presented like generated samples without identification, and task assignments were randomized while maintaining approximately balanced annotation coverage.Source: D. Supplementary Details about Human… · p. 31
• Participants completed tutorials and practice examples before blind holistic evaluation, while final human reference scores averaged independent annotator ratings and were compared using Pearson, Spearman, R2, and MAE.Analyses were performed at the sample level, and absolute-error comparisons used linearly rescaled unified scores to a common range.Source: D. Supplementary Details about Human… · pp. 31–32

D.1. Comparison of Different Unified Scores

• AFS, RoboPhyscore, WorldModelBench, EWMBench, and WorldArena were compared after linear rescaling to common ranges, with correlations computed directly because Pearson and Spearman values are scale-invariant.The rescaling maps scores to [0, 10] for human evaluation and [0, 1] for task-success comparison, while R2 and MAE are computed on the corresponding common scales.Source: §D.1 · p. 32

D.1.1. Human Alignment

• RoboPhyscore achieves the highest Pearson correlation with human evaluation (r = 0.9761), while AFS achieves the highest rank correlation (ρ = 0.9715) and lowest MAE (0.0284), outperforming existing unified scores.AFS also attains R2 = 0.9499, whereas RoboPhyscore attains R2 = 0.9528; WorldArena, WorldModelBench, and EWMBench reach Pearson correlations of 0.9263, 0.8090, and 0.6417, respectively.Source: §D.1.1 · p. 32

D.1.2. Task Alignment

• RoboPhyscore achieves the strongest linear agreement with Action Planner (r = 0.9551) and Data Engine (r = 0.9461), while AFS provides the best rank consistency and calibration overall.For VLM-2, AFS is best on all four measures; external scores trail on every target, and the two proposed scores serve complementary roles.Source: §D.1.2 · p. 32, Table 19 · p. 34

E.1.3. Distribution Similarity

• Generative priors frequently attenuate reconstruction-substrate effects, so high scores in camera, motion, consistency, or roundtrip dimensions should not be treated as standalone evidence of accurate physical evolution.This limitation motivates evaluating perceptual and internal-consistency measures together with geometry-, state-, and task-level metrics.Source: §E.1.1 · p. 33, §E.1.4 · p. 40, §E.1.5 · p. 42

E.1.1. Image Quality

• 3.6128/6 makes Cosmos the highest-IQ-1 world model, but perceptual quality, aesthetic appeal, distribution similarity, motion, and content consistency remain imperfect proxies for reference fidelity or physical correctness.Wan leads aesthetic quality (1.6047/4) and motion quality (3.1620), while RoboTwin leads content consistency (3.0863); prompt gains are generally smaller than substrate effects, especially for image and aesthetic quality.Source: §E.1.1 · pp. 33–34, §E.1.2 · pp. 35, 37, §E.1.4 · p. 40

E.1.2. Aesthetic Quality

• Prompt specification improves Wan and Cosmos across image, aesthetic, distribution, motion, and content measures, but the magnitude and components differ substantially by model.Wan is more prompt-sensitive, whereas Cosmos is generally more robust; gains often reflect semantics, motion characteristics, or perceptual plausibility rather than verified physical or task correctness.Source: §E.1.1 · p. 34, §E.1.2 · p. 37, §E.1.3 · p. 39

E.2.1. Reconstruction-Based Geometry

• CogVideoX records the highest reconstruction-based geometry score (2.4253), but its advantage is driven partly by reprojection photometric consistency rather than necessarily superior physical geometry.Wan and Cosmos remain near the RoboTwin reference, while prompt conditioning trades off geometric fidelity against stability and reprojection consistency, especially for Wan.Source: Table 30 · p. 43, Table 31 · p. 43, §E.2.1 · p. 44

E.2.2. Depth Consistency

• Depth consistency is highly reconstruction-sensitive: GT reaches 1.8965, whereas 4C4D and 4DGS retain only 79.2% and 74.7% of the GT score.Prompt specification improves DC by 15.6% for Wan and 7.7% for Cosmos, with the dominant gains coming from Perspectivity rather than Depth Accuracy.Source: §E.2.2 · pp. 44–46, Table 32 · p. 45, Table 33 · p. 45

E.2.3. Camera Trajectory Geometry

• RoboDreamer leads camera-trajectory geometry at 0.9807, but CTG measures recoverable temporal camera consistency rather than overall 3D accuracy and should be interpreted jointly with other geometry metrics.Generation compresses substrate differences, reducing CTG ranges from 0.2310 for RoboTwin to 0.0078 for RoboDreamer.Source: §E.2.3 · pp. 47–48, Table 34 · p. 47, Table 35 · p. 47

E.2.4. Roundtrip Consistency

• Roundtrip consistency responds differently to prompting: Wan decreases from 0.7468 to 0.7108, whereas Cosmos remains nearly stable across prompts at 0.7117–0.7132.Because RC measures internal spatial agreement rather than ground-truth correspondence, changes in scene evolution can lower RC without implying poorer geometric correctness.Source: §E.2.4 · pp. 49–50, Table 36 · p. 49, Table 37 · p. 49

E.3.1. Object Localization

• Cosmos achieves the highest mean object localization at 0.8276, while Wan, RoboDreamer, and CogVideoX reach 0.4497, 0.2581, and 0.1623, respectively.Generated rollouts preserve object presence more reliably than precise localization, as recall exceeds precision for every model.Source: §E.3.1 · p. 50, Table 38 · p. 51

E.3.2. Trajectory Accuracy

• Cosmos leads trajectory accuracy at 1.4702, followed by Wan at 1.2721, CogVideoX at 1.2011, and RoboDreamer at 0.7463, all below RoboTwin’s 3.7295.Reconstruction strongly affects trajectory accuracy: 4C4D retains only 35.7% of the GT score, while generation contracts substrate-dependent variation.Source: Table 40 · p. 52, §E.3.2 · pp. 52–53

E.3.3. Interaction Quality

• Wan and Cosmos obtain the strongest interaction-quality scores, 0.6768 and 0.6762, exceeding RoboTwin’s 0.6216, but visually plausible interactions do not ensure correct placement or motion.Wan reaches 0.7345 under GT conditioning despite substantially lower localization and trajectory accuracy than the reference.Source: Table 42 · p. 54, §E.3.3 · p. 55

E.3.4. Event and Action Adherence

• Cosmos, Wan, and RoboDreamer preserve high event and action adherence, whereas CogVideoX falls to 1.6488; semantic event success nevertheless coexists with weaker localization and trajectory accuracy.EAA ranges remain small for Wan, Cosmos, and RoboDreamer, indicating stronger dependence on high-level action semantics than reconstruction fidelity.Source: §E.3.4 · pp. 55, 57, Table 44 · p. 56

E.3.5. Physical Law Adherence

• Prompt specification raises Wan’s physical law adherence from 0.6131 to 0.6462, while Cosmos remains stable between 0.6469 and 0.6506.These gains indicate improved apparent physical plausibility, not definitive correctness of the underlying dynamics.Source: Table 46 · p. 58, §E.3.5 · pp. 58–59

E.3.6. Semantic Adherence

• Cosmos achieves the highest semantic adherence at 4.7550, retaining 99.0% of RoboTwin’s 4.8047, followed by Wan at 4.6764, while CogVideoX and RoboDreamer degrade further.Semantic agreement can remain high despite low fine-grained textual agreement and substantial state-level errors.Source: Table 48 · p. 59, §E.3.6 · p. 60

E.4.1. Instruction Following

• Cosmos leads instruction following at 1.5422 and Wan follows at 1.5095, both exceeding RoboTwin’s 1.2004, while richer current prompts raise Wan to 1.6600 and Cosmos to 1.6267.Instruction-following scores measure recognizable task compliance rather than exact execution fidelity, so localization and trajectory metrics remain necessary.Source: Table 51 · p. 62, §E.4.1 · pp. 62–63

E.4.2. Collision Safety

• CogVideoX achieves the highest collision safety at 0.9650, above RoboTwin’s 0.9385, but limited motion may reduce collision opportunities without completing the intended interaction.Cosmos reaches 0.9458, whereas Wan and RoboDreamer score 0.8314 and 0.8930; safety should therefore be evaluated jointly with motion and task completion.Source: Table 52 · p. 63, §E.4.2 · p. 64
• CS rises from 0.8333 to 0.8981 for Wan under the proposed current prompt, a 7.8% improvement over the instruction-only baseline.Nearly all of the gain occurs in the first refinement (+0.0640), while the view-aware refinement adds only +0.0008.Source: §E.4.2 · p. 64
• SS increases from 0.9435 to 0.9531 for Cosmos, an approximately 1.0% overall gain, with the proposed current prompt scoring highest.Cosmos is substantially more robust to prompt formulation than Wan.Source: §E.4.2 · p. 64
• Collision safety improvements provide evidence that prompt design affects executable behavior, but should be evaluated jointly with localization, trajectory, instruction-following, and task-completion performance.CS is grounded in collision outcomes rather than visual plausibility alone, and collision avoidance does not establish successful execution.Source: §E.4.2 · p. 64

Task Success

• Moderate reconstruction artifacts are tolerated at the task level, but sufficiently severe degradation causes an abrupt loss of downstream executability.This finding is supported across GT, VGGT, VGGT-Ω, and 4C4D reconstruction conditions.Source: §E.4.3 · p. 66

Prompt Specification

• Wan’s TS rises from 1.4650 to 1.8850 with the proposed current prompt, a 0.4200 (28.7%) gain driven primarily by Action Planner and Data Engine.Action Planner increases from 0.3525 to 0.5850 (+66.0%), while Data Engine rises from 0.2075 to 0.3450 (+66.3%).Source: §E.4.3 · p. 66
• Cosmos’s TS increases from 2.1400 to 2.3275 with the proposed current prompt, an 8.8% improvement as Action Planner, Data Engine, and VLM-2 all reach their highest values.The corresponding component changes are Action Planner 0.7800 to 0.8350, Data Engine 0.4550 to 0.4925, and VLM-2 0.9050 to 1.0000.Source: §E.4.3 · p. 66