PaperlayerView exact revision on arXiv

Source-linked AI summary

GreenWAM: A World Foundation Model for Robotics

Gennady Sidorov, Valerii Kornilov, Denis Grankin, Polina Fedotova, Aleksey Postnikov

arXiv:2610.18774v1cs.RO

TL;DR

Robot world models and policies are usually developed and evaluated separately, while robot datasets use fragmented bodies and action conventions. GreenWAM combines a shared anatomical action space with joint video–action training and a split action expert; it reports strong results across world-model and policy benchmarks, including a real humanoid.

  • Problem

    World models are typically evaluated for prediction and world-action models for policy success, while robot datasets differ in bodies and recording conventions that obscure correspondence across robots.

    Source: §1 · p. 1

  • Method

    GreenWAM maps 20 robot families into shared anatomical joint coordinates, jointly trains visual, action-conditioned, and world-action modes, and derives policies by splitting an action expert from the generator.

    Source: §1 · p. 1

  • Results

    GreenWAM attains top-tier WorldArena 1.0 trajectory, depth, and 3D accuracy, scores among top RoboTwin 2.0 entries, and completes 95.6% of in-distribution instructions on a real humanoid.

    Source: §1 · p. 2, §9 · p. 26

  • Takeaways & Limitations

    Controlled comparisons report that robot specialization improves prediction and control together, an action expert helps when split at adaptation start, and retaining world-model modes improves randomized-scene success.

    Source: §1 · pp. 2–3

    Transfer to a robot absent from the robotics corpus is specifically demonstrated on the single-arm Franka through the shared interface.

    Source: §8 · p. 26

Abstract

from arXiv · show

A robot needs a world model that predicts how its actions will change the scene and a policy that chooses those actions. Models are usually built and evaluated for only one of these roles, and every robot records its actions in its own format. We ask whether one robot world model can serve both roles across different bodies, and what makes this work. We present GreenWAM, a world foundation model for robotics, together with the recipe that produces it: (1) a shared anatomical action space that describes 20 robot families, from bimanual arms to dexterous humanoids, in common joint coordinates; (2) joint training on these families in visual, action-conditioned and world-action modes; and (3) policies that split an action expert off the generator. As a world model, GreenWAM attains top-tier trajectory, depth and 3D accuracy on WorldArena 1.0, ahead of every world-action model and video generator in our comparison. As a policy, it scores among the top entries of the RoboTwin 2.0 leaderboard, reaches the highest zero-shot LIBERO-plus success among the compared methods and is competitive with dedicated vision-language-action and world-action policies on CALVIN and LIBERO. We further adapt GreenWAM to a real humanoid, where it succeeds more often than policies adapted from $\pi_{0.5}$ and from Cosmos 3 Edge, and distill it into a single-step policy for real-time control. Controlled experiments yield three findings: robot specialization improves world prediction and control together; an action expert helps when split off at the start of adaptation; and keeping world-model modes during policy adaptation preserves robustness to scene changes.

1 Introduction

The paper asks whether one robot world model can predict and control across bodies despite fragmented robot data and action conventions. GreenWAM combines a shared anatomical action space, joint training, and a split action expert, with reported results across simulation benchmarks and a real humanoid.

1 Introduction

  • The work targets a gap between world models evaluated for prediction and world-action models evaluated for policy success, amid robot datasets with incompatible bodies and recording conventions.The same tensor dimension can represent different motions on different robots, while existing action representations leave cross-body correspondence implicit or absent.Source: §1 · p. 1
  • GreenWAM maps 20 robot families into a shared anatomical action space, jointly trains visual, action-conditioned, and world-action modes, and derives policies by splitting an action expert from the generator.The shared slots represent common joint coordinates across bodies, while data are prepared as complete subtasks with instructions.Source: §1 · pp. 1–2
  • GreenWAM achieves top-tier trajectory, depth, and 3D accuracy on WorldArena 1.0 and strong policy results across RoboTwin 2.0, LIBERO-plus, CALVIN, and LIBERO.On a real humanoid, it completes 95.6% of in-distribution instructions, versus 88.9% for a policy adapted from its parent and 84.4% for π0.5.Source: §1 · p. 2
  • Controlled comparisons report that robot specialization improves world prediction and control, splitting off an action expert at adaptation start raises policy success by 3.3%, and retaining world-model modes raises randomized-scene success by 8.0%.The specialization comparison reports 3.0 more WorldArena trajectory-accuracy points, 23% lower video prediction error, and 5.4% more RoboTwin success, or 8.4% with the action expert.Source: §1 · pp. 2–3

2 Related Work

Related work spans robot-specialized world foundations, shared action representations across bodies, policies derived from video models, and temporal context for long rollouts. GreenWAM is positioned among these approaches by its shared anatomical projector, joint video–action policy, and history-based continuation.

2 Related Work

  • Prior world foundations include Cosmos 3 [58], which combines visual generation and dynamics but uses six embodiments and embodiment-specific action projections; other routes include human-video proxy actions and generated robot experience [25] [27].Earlier work learned action-conditioned video prediction from diverse datasets [89], while V-JEPA 2 post-trains a latent action-conditioned world model for planning [6].Source: §2 · p. 3
  • Cross-embodiment methods pool robot demonstrations, adapt sensors or action spaces, or map embodiment inputs into shared tokens, prompts, or action coordinates [60] [59] [80] [103] [102] [49].Green-VLA uses a unified embodiment-aware interface across humanoids, mobile manipulators, and fixed-base arms [5].Source: §2 · p. 3
  • GE-Sim 2.0 action maps project arm end-effector poses into calibrated camera images and cannot express finger joints; GreenWAM instead uses unified joint and Cartesian-state projection with 79 anatomical slots and masks.Source: §2 · p. 4
  • Video-derived control ranges from extracting actions from generated plans [23] to joint video–action models such as UVA [43] and UWM [104]; GreenWAM specializes its action stream with a dedicated expert.Other interfaces include joint generation, causal prediction with asynchronous execution, and visual planning followed by inverse dynamics [92] [42] [2].Source: §2 · p. 4
  • For long rollouts, prior methods use semantic-event alignment, sparse history, perturbed history, or refreshed cached predictions [44] [47] [63] [92]; GreenWAM encodes four fixed-position history anchors and reports WorldArena rollouts beyond 20 seconds.World simulators have also been used to compare generated and real executions, evaluate task completion, manage drift, and select policies or checkpoints [28] [63] [96].Source: §2 · p. 4

3 GreenWAM

GreenWAM uses one shared action interface across robot bodies, with layout-specific codecs mapping native states into anatomical coordinates. A single generator is trained across visual, action-conditioned, and world–action modes, while policies use an action expert split off the generator.

3.1 Model Overview

  • GreenWAM maps robot states into shared anatomical coordinates and trains one generator to predict video, motion, or both; policies split an action expert off that generator.The robotics stage initializes from Cosmos 3 Edge, keeps its reasoner and VAE frozen, and trains the generator, shared action projections, and action-modality embeddings.Source: §3 · p. 4, §3.1 · p. 5

3.2 Shared Anatomical Action Space

  • Layout-specific codecs map native robot channels into 79 anatomical coordinates before one shared projector, while validity masks exclude unavailable coordinates from the loss.Forward-dynamics and world–action targets are expressed relative to the block-boundary state for selected joints; grippers, hands, and fingers stay absolute by default, and these relative values are not velocities or accumulated increments.Source: §3.2 · p. 6

3.3 Joint Video–Action Modeling

  • One generator is trained on seven modes that vary which video and state streams are clean conditions and which are noised targets.The fixed mixture assigns three quarters of examples to video prediction, with or without trajectories, and one quarter to inverse dynamics and world–action state prediction.Source: §3.3 · pp. 6–7

3.4 Multi-View Observations and Temporal Context

  • Three-view recordings are composed into a fixed canvas with the head camera above left and right wrist views, while autoregressive modes condition each block on four RGB history anchors.Anchors occupy fixed positions from the start to the end of available history; training uses ground-truth history, while inference uses generated video.Source: §3.4 · p. 8

4 Robot Experience and Training

GreenWAM’s robotics training recipe aligns data from diverse two-armed robots into subtask clips with shared motion targets, then trains a mixture of visual and action modes. The pipeline preserves interval alignment between video, text, and measured robot motion.

4 Robot Experience and Training

  • The pipeline cuts recordings into subtask clips, filters unusable clips, derives separate text for visual prediction and action learning, then trains the robotics stage.It is designed to keep each example’s robot motion aligned with video and language from the same interval.Source: §4 · pp. 8–9

4.1 Cross-Embodiment Corpus

  • The corpus combines public and internal datasets with humanoid data into atoms: subtask clips with synchronized views, measured states, text, and provenance.It spans 20 two-armed robot families, including bimanual platforms and humanoids, in real and simulated scenes.Source: §4.1 · p. 9

4.2 Data Curation and Training Mixture

  • Admission and quality filtering retain 1.43 million atoms and 1,674.7 hours from 1.44 million atoms and 1,688.5 hours.Atoms are bounded by annotated subtask boundaries and length limits; the quality pass removes idle, duplicated, cut, and unreadable clips.Source: §4.2 · pp. 9–10, Figure 6 · p. 9
  • A structured captioner creates visual descriptions and attempt-aware action phases, which a second stage converts into ordered prospective commands.Visual modes use the descriptions, forward dynamics and world–action prediction use derived commands, and inverse dynamics uses a fixed instruction; world–action examples additionally require a positive success label.Source: §4.2 · p. 10
  • Measured states are converted through each robot-layout codec into shared trajectories of realized motion; inverse dynamics predicts absolute positions, while forward dynamics and world–action prediction use displacement targets.Sampling selects a mode, robot cluster, dataset, and window; sources are sampled only in modes their data support, with counts guiding weights for most clusters and datasets.Source: §4.2 · p. 11

4.3 GreenWAM Training

  • The robotics stage trains seven modes with rectified flow and balances video loss with state loss averaged first within anatomical groups, then across groups.Both loss terms are scaled by 10; absent and unresolved coordinates are masked, and the GreenWAM checkpoint is produced after 36,500 optimizer steps from 9.34 million sampled examples.Source: §4.3 · p. 11

5 GreenWAM as a World Model

The robotics checkpoint predicts action-conditioned video across held-out robot families, with generated motion responding to commanded displacement. On WorldArena 1.0, GreenWAM leads the compared world-action models and video generators in trajectory, depth, and 3D accuracy.

5.1 The Robotics Checkpoint on Held-Out Recordings

  • Across held-out recordings of six robot families, generated motion grows with commanded displacement; zero displacement yields under one-fifth of recorded-command motion, while 1.5× yields about a quarter more.The held-out set includes the Green humanoid in simulation and on the real robot.Source: §5.1 · p. 12, Figure 8 · p. 12

5.2 WorldArena Adaptation and Results

  • GreenWAM has the highest trajectory, depth, and 3D accuracy among compared world-action models and video generators (56.0, 98.2, and 93.4); its aggregate score is 63.31, above Motubrain (63.16) and FlowWAM-FiveAges (62.99).The aggregate is computed over 1,000 test episodes with motion metrics capped at the ground truth; the recomputed baseline scores are upper bounds.Source: §5.2 · pp. 12–13, Figure 9 · p. 13
  • Capping dynamic degree, flow score, and motion smoothness at the recorded-video reference prevents rewarding excess motion; GreenWAM loses 0.05 under the cap.112 of 127 board entries exceed the reference in dynamic degree and 113 exceed it in flow score.Source: §5.2 · pp. 12–13
  • Real test videos score 67.23 under the aggregate evaluator, and GreenWAM differs from them by less than four on 13 of 15 metrics.Its two trailing categories are visual quality and controllability, which do not compare generated video directly with the recording.Source: §5.2 · p. 13

6 GreenWAM as a Policy

GreenWAM adapts one shared foundation into policies for simulation benchmarks and a real humanoid, using a split action expert and benchmark-specific post-training. The policies perform strongly across the reported tasks, while a distilled student reduces inference latency for real-time control.

6.1 Policy Architecture

  • A dedicated action transformer splits action tokens from the video generator while joint attention connects both streams; copying the generator initializes the expert without changing predictions.The split adds as many parameters as the generator tower and about 8% to training-step time.Source: §6.1 · pp. 14–15

6 GreenWAM as a Policy; 6.3 Simulation Benchmarks

  • The shared adaptation objective is the WAM mode, while each benchmark retains its own demonstrations, command interface, and inference settings; benchmark data are absent from GreenWAM’s pretraining corpus.The four simulation evaluations are RoboTwin 2.0, CALVIN, RoboDojo, and LIBERO.Source: §6.2 · p. 15, §6.3 · p. 15

6.2 Policy Post-Training

  • Policy post-training predicts future video and state offsets from instructions, camera images, and measured state, then decodes targets for the robot’s native commands and replans after executing a prefix.Color augmentation is used on 80% of samples; simulator-based background replacement is used on RoboTwin.Source: §6.2 · p. 15

6.3.1 RoboTwin 2.0

  • 86.98% clean and 48.10% randomized RoboTwin success, averaging 67.54% over 50 tasks, places GreenWAM among the leaderboard’s top entries.The evaluation used 100 trials per task in each setting and official seeds.Source: §6.3.1 · p. 15

6.3.2 CALVIN; 6.3.3 RoboDojo

  • 4.64 instructions per CALVIN chain exceeds FLOWER’s 4.53; on RoboDojo, GreenWAM scores 16.12 and 10.49% success, higher on both aggregates than π0.5, X-VLA, and X-WAM.RoboDojo weights its five capability dimensions equally.Source: §6.3.2 · p. 16, §6.3.3 · pp. 16–17

6.3.4 LIBERO and LIBERO-plus

  • GreenWAM is close to dedicated policies on LIBERO and achieves the highest compared zero-shot LIBERO-plus success; under changed viewpoints, success is 81.7%.The Franka embodiment is absent from the robotics corpus, and the same policy is evaluated on both benchmarks.Source: §6.3.4 · p. 18, Figure 13 · p. 18

6.4 Distillation for Real-Time Control

  • Distillation cuts policy latency to 35.4 ms, 3.8 times faster than the optimized teacher and 12.3 times faster than the unoptimized policy, at 28.3 Hz.The student replaces four guided denoising steps with one consistency step.Source: §6.4 · p. 19, Figure 14 · p. 19, Figure 15 · p. 19

6.5 Real-World Humanoid Evaluation

  • On the Green humanoid, GreenWAM succeeds in 95.6% of in-distribution and 84.0% of out-of-distribution instructions, ahead of adapted Cosmos 3 Edge and π0.5 policies.The distilled student reaches 91.1% and 80.0%, respectively, under the teacher’s same execution settings.Source: §6.5 · p. 20, Figure 17 · p. 21

7 Empirical Analysis

Controlled comparisons at fixed adaptation budgets test robot specialization, action-expert splitting, and retention of world-model modes. Specialization improves prediction and policy success, the expert raises policy success, and retaining world-model modes chiefly benefits randomized scenes; analyses relate robustness to the accuracy of imagined scenes.

7.1 What Robot Specialization Brings to Each Role

  • Robot-specialized initialization raises WorldArena trajectory accuracy from 44.4 to 47.4 and RoboTwin policy success by 5.4% on average, while prediction is 1.3 dB more accurate on held-out frames.The specialized model has higher first-block and later-block PSNR, and predicts more accurately on 38 of 50 held-out windows.Source: §7.1 · pp. 22–23, Figure 19 · p. 22

7.2 Coupling the World Model to Action

  • Splitting off an action expert at adaptation start raises RoboTwin policy success by 3.3% on average, with gains of 2.8% in clean and 3.7% in randomized scenes.The expert begins as an exact copy of the generator, preserving initial predictions; validation losses are nearly equal.Source: §7.2 · p. 23

7.3 Retaining World-Model Modes during Policy Adaptation

  • Keeping forward-dynamics and image-to-video modes alongside WAM raises randomized RoboTwin success by 8.0%, with no significant change in clean success.Both variants predict video within WAM, so the measured gain is associated with the additional modes.Source: Figure 21 · p. 23, §7.3 · p. 23

7.4 What the Policy Imagines

  • In clean successful episodes, imagined frames exceed a last-frame-repeat baseline by a median 9.6 dB PSNR; randomized-scene success falls from 85% for the most accurate imagination to 27% for the least accurate.Among randomized episodes, those with accurate imagination succeed twice as often as those with poor imagination when judged by the first 100 steps.Source: §7.4 · pp. 24–25, Figure 23 · p. 24
  • Of 20 reviewed clean failures, 19 imagined the executed outcome correctly; in randomized scenes, 13 of 20 failures involved inaccurate imagined scenes or grasping a distractor.The review distinguishes action errors in clean scenes from scene-perception or imagination errors in randomized scenes.Source: Figure 22 · p. 24, §7.4 · p. 25
  • The 8.0% randomized-scene gain from retaining video-prediction modes aligns with failures arising from inaccurate imagination, while clean-scene failures often occur despite accurate predictions.The analysis attributes the modes’ benefit to the setting where failures stem from imagination, not to clean-scene action errors.Source: §7.3 · p. 23, §7.4 · p. 25

7.5 Robustness to Scene Randomization

  • Background textures and table height cause the largest RoboTwin success drops, 12% and 8% below clean; background replacement raises randomized success by 13.1% without changing clean success.Appearance augmentation changes observations but not recorded states or actions, so it does not add motions for altered table heights.Source: §7.5 · p. 25, Figure 24 · p. 25

8 Discussion and Limitations

GreenWAM uses one robotics foundation for world modeling and policy control, with controlled comparisons connecting benefits across the two roles. The discussion also identifies boundaries in geometry robustness, long-horizon prediction, and evaluation scope.

8 Discussion and Limitations

  • Robot specialization improves forward prediction and policy success at the same adaptation budget, linking the two roles in controlled comparisons.The policy that predicts held-out frames more accurately also succeeds more often.Source: §8 · p. 25
  • Keeping forward dynamics and image-to-video during adaptation raises randomized RoboTwin success by 8.0%, and out-of-distribution success tracks imagined-scene accuracy.Both policy variants predict video in WAM mode; the retained modes add training signals beyond that shared prediction.Source: §8 · p. 26
  • Policies remain sensitive to task-geometry changes, long rollouts accumulate prediction error, and the real-robot evaluation covers one humanoid and one packing task.The reported geometry examples include table height on RoboTwin and initial configuration on LIBERO-plus.Source: §8 · p. 26

9 Conclusion

The paper asks whether one robot world model can serve both prediction and policy roles across different bodies. GreenWAM combines a shared action space and joint video–action training, yielding strong world-model and policy results, with controlled comparisons identifying design choices associated with those outcomes.

9 Conclusion

  • GreenWAM adapts Cosmos 3 Edge to 20 robot families through a shared anatomical action space and joint video–action training.Its policy uses an action expert split off from the generator.Source: §9 · p. 26
  • GreenWAM achieves top-tier trajectory, depth, and 3D accuracy on WorldArena 1.0, while its policies rank among top RoboTwin entries, compete on CALVIN and LIBERO, and outperform adapted π0.5 and Cosmos 3 Edge policies on a real humanoid.A distilled one-step student runs at 28.3 Hz and keeps its lead over π0.5 on the robot.Source: §9 · p. 26
  • Controlled comparisons identify robot specialization, an action expert split off at adaptation start, and retained world-model modes as choices associated with improved prediction, control, or scene robustness.The paper notes future work on unseen bodies and more reliable long-horizon prediction and control under geometry changes.Source: §9 · p. 26

A Data Inventory and Anatomical Mappings

The corpus groups recordings by robot body, filters them into admitted training data, and maps registered layouts into shared anatomical slots only when channel correspondences are established.

A.1 Family and Source Inventory

  • The inventory covers 1,217 datasets, grouped into 20 robot families; the largest admitted contributions include AgiBot G2A (256.0 h) and Piper / Cobot Magic (227.2 h).The Green humanoid contributes 214.8 h, Unitree G1 204.9 h, and AgiBot G1 / Genie-1 148.5 h.Source: §A.1 · p. 29

A.2 Corpus Admission and Quality Filtering

  • Source-level filtering retains 1.43 million atoms totaling 1,674.7 h, while admission and quality rules reject overlong, idle, repeated-frame, abruptly cut, or unreadable clips.Atoms without accepted captions or derived commands remain in the quality-admitted pool but do not enter text-dependent modes.Source: §A.1 · p. 30, §A.2 · pp. 30–31, Table 4 · p. 31

A.3 Anatomical Mappings

  • Each of 43 source layouts maps established channels into 79 anatomical slots, while absent or unresolved coordinates are masked from the loss rather than treated as shared.Correspondences are established from sources such as URDFs, manufacturer documentation, named channels, and, for main families, forward-kinematics validation against measured end-effector poses.Source: §A.3 · p. 32, Figure 27 · p. 32

B.1 Robotics Training Recipe

The robotics recipe trains multiple modes with shared noise scheduling and mode-specific batches, preserves complete subtasks and physical timing in prediction targets, and creates separate visual descriptions and action commands.

B.1 Robotics Training Recipe

  • The robotics stage applies a shared noise schedule, packs each micro-batch with examples from one mode, and uses higher learning rates for newly initialized action projections and modality embeddings.The inverse-dynamics and WAM running-median losses fall from 8.3 to 0.29 and from 6.2 to 1.98, respectively.Source: §B.1 · pp. 32–33, Figure 28 · p. 33, Table 5 · p. 33

B.2 Temporal Sampling and Prediction Targets

  • Physical timing is retained while video rate is capped at 30 FPS, and prediction targets use complete subtasks rather than clips that truncate actions.WAM predicts about 1.07 seconds of future video and state within a subtask; autoregressive modes predict whole-subtask blocks using four RGB history anchors.Source: §B.2 · p. 34, Figure 29 · p. 34

B.3 Structured Captions and Commands

  • Each atom receives both a visual description of what happened and a separate imperative command for what the robot should do; 1.24 million atoms have accepted captions and 1.19 million also have accepted commands.Caption fields are generated in sequence and checked against registry facts and attempt records; commands preserve phase order and are never truncated.Source: §B.3 · p. 35, Table 6 · p. 35, Table 7 · p. 36

B.4 Training Mixture

The training mixture balances modes, robot clusters, and datasets rather than sampling in proportion to hours alone; the included policy recipes adapt GreenWAM under benchmark-specific settings.

B.4 Training Mixture

  • The sampler chooses a mode, robot cluster, dataset, and window, with the Worker/ARX5 and Green humanoid clusters assigned 12% and 18% of examples, respectively.Families with little data are sampled more often than their hour shares would warrant, at the cost of repeating their subtasks.Source: §B.4 · p. 36, Table 8 · p. 37

B.5 Policy Adaptation Recipes

  • Policy recipes retain benchmark-specific preprocessing and action representations; RoboTwin adaptation uses a WAM-heavy mixture, while the Green humanoid comparison adapts π0.5, Cosmos 3 Edge, and GreenWAM on the same demonstration split.The RoboTwin policy uses 0.9 WAM, 0.05 forward, and 0.05 image-to-video examples; LIBERO-plus is evaluated zero-shot using the LIBERO-adapted policy.Source: §B.5 · pp. 36–38

C Evaluation Protocols

  • Figure 30 shows representative scenes from RoboTwin 2.0, LIBERO, LIBERO-plus, CALVIN, RoboDojo, and WorldArena 1.0 evaluation settings.The LIBERO-plus examples include a camera turned by −32° and a changed table texture.Source: Figure 30 · p. 38

C.1 Public Benchmarks

The public-benchmark protocols define distinct training and evaluation settings across simulation and real-world tasks, while the humanoid comparison separates in-distribution and out-of-distribution trials.

C.1 Public Benchmarks

  • RoboTwin 2.0 evaluates 50 bimanual tasks using 100 trials per task in clean and randomized scenes, and ranks entries by the average across both settings.Training uses 50 clean demonstrations per task; the benchmark uses the Aloha-AgileX embodiment with ARX5 arm kinematics.Source: §C.1 · p. 38, Table 9 · p. 39
  • The public protocols span WorldArena 1.0, RoboDojo, LIBERO and LIBERO-plus, and CALVIN, with metrics and splits specific to each benchmark.CALVIN trains on environments A, B, and C and evaluates 1,000 five-instruction chains in unseen environment D; WorldArena uses all 1,000 test episodes, and its recomputed aggregate differs from the official leaderboard score.Source: §C.1 · pp. 39–40

C.2 Real-Humanoid Comparison Protocol

  • Humanoid packing-task evaluation uses 45 in-distribution and 25 out-of-distribution instructions per policy across 14 scene configurations, with each instruction attempted once.Out-of-distribution trials change objects or scene conditions excluded from adaptation, validation, and distillation data; success is reported separately for ID and OOD.Source: §C.2 · p. 40

C.3 Latency Benchmark

The appendix specifies GreenWAM’s latency benchmark and optimization sequence, paired-comparison uncertainty, and evaluation details for CALVIN and RoboTwin.

C.3 Latency Benchmark

  • Latency was measured on one NVIDIA RTX 4090 over 100 episodes, with median steady-state latency distinguished from first-request time after a new instruction.The video branch predicts eight frames at 10 Hz from three latent frames; observations and state remain fixed within an episode.Source: §C.3 · p. 40, Table 10 · p. 40
  • Caching, guidance packing, single-frame VAE encoding, compilation, W8A8, and distillation are applied in turn to reduce inference cost.Caching reuses instruction and vision key–value states across denoising steps; distillation replaces four guided denoising steps with one consistency step.Source: §C.3 · p. 40

C.4 Paired Comparisons

  • Paired differences average outcomes over common episode seeds and tasks; hierarchical bootstrap intervals quantify checkpoint-evaluation uncertainty, not variation between training runs.Tasks are resampled first, followed by episode pairs within each task, across 5 000 replicates.Source: §C.4 · p. 41

C.5 Evaluation Details

  • The action variant of WorldArena is sampled without classifier-free guidance when its text prompt is empty; CALVIN test instructions are excluded from augmentation pools.RoboTwin results use 100 trials per task and setting, and controlled comparisons pair episodes on common seeds.Source: §C.5 · p. 41

D.1 Benchmark Tables

The appendix catalogs benchmark-table scopes and comparison sources, then illustrates typical policy failures at final object placement and recovery.

D.1 Benchmark Tables

  • Table 11 reports WorldArena 1.0 categories and fidelity, with EWMScore averaging 15 metrics and capping three metrics at ground truth.For GreenWAM, the cap is applied per video; for other models, it uses leaderboard means, and the final row evaluates recorded test videos.Source: Table 11 · p. 41, §D.1 · p. 41

D.2 Failure Cases

  • Typical failures occur after grasping and transport, when imprecise final placement is not corrected before the step limit.The examples are a tilted cream-cheese placement on a bowl rim and a stapler left partly on its target pad.Source: §D.2 · p. 41
  • The tables cover CALVIN chain completion, RoboDojo score and success across five capability dimensions, and average LIBERO plus zero-shot LIBERO-plus success.The RoboDojo table also reports human teleoperation at a score of 80.42 and success rate of 76.03%.Source: Table 12 · p. 42, Table 13 · p. 42, Table 14 · p. 42
  • LIBERO-plus comparisons use all 10 030 perturbed episodes with one trial each; the LIBERO-plus baselines are evaluated zero-shot after standard LIBERO training.The appendix identifies the source tables for the listed LIBERO and LIBERO-plus comparison methods.Source: §D.2 · p. 42

Related papers

Loading 2610.18774v1…