PaperlayerView exact revision on arXiv

Source-linked AI summary

Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation

PhysicalRSI Team, Danjiao Ma, Enhui Ma, Haohan Liu, Heng Jia, Hui Shan, Jianhua Xu, Jiahuan Zhang, Jiangdi Xu, Kaiwen Guo, Kaicheng Yu, Linwei Zhang, Liyang Jin, Maochun Luo, Pengyao Niu, Shiwen Li, Shuangyu Feng, Tong Zhang, Tianheng Wang, Xin Wang, Xiangru Huang, Yongqiang Huang, Zhaozhi Wang, Zijian Ma

arXiv:2610.02274v1cs.RO

TL;DR

Robot-training data requires interactive, physically valid worlds and repeatable tasks, yet real-world collection is costly and manual simulation preparation limits synthetic-data coverage. Awomo-SimDataEngine connects asset and scene generation to validated, replayable robot demonstrations. On MuJoCo-based LIBERO-Plus, Isaac Sim co-training raises WAM overall success from 77.17% to 89.43%, with more limited gains on long-horizon tasks.

  • Problem

    Robot-training data requires interactive, physically valid scenes and repeatable tasks, while real-world collection is costly and manual simulation preparation limits synthetic-data scale and diversity.

    Source: §1 · pp. 1–2

  • Method

    Awomo-SimDataEngine connects asset generation, image- and text-driven scene construction, and demonstration synthesis through a Graph-Native Harness for validation and repair.

    Source: §1 · p. 2, §3 · p. 6, §9.2 · p. 21

  • Results

    77.17% to 89.43%: Isaac Sim demonstration co-training raises WAM overall success on MuJoCo-based LIBERO-Plus, with gains of 31.66 points on goals and 6.25 on spatial generalization.

    Source: §1 · p. 3, §9.2 · pp. 21–22

  • Takeaways & Limitations

    The results support the utility of the combined data-generation system for cross-simulator policy training under the reported evaluation protocols.

    Source: §9.2 · p. 22

    Ultra’s gain on LIBERO-10 is 1.99 percentage points, smaller than its 31.66-point gain on goals and 6.25-point gain on spatial generalization; richer sequential-task supervision is motivated.

    Source: §9.1 · p. 21

Abstract

from arXiv · show

Generating useful robot-training data requires more than visually plausiblescenes: objects must support interaction, placements must remain physicallyvalid, and tasks must admit repeatable execution. We present\textbf{Awomo-SimDataEngine}, an agentic system that connects asset and scenegeneration to robot demonstration synthesis. Shared asset services providerigid and articulated objects, including structure-grounded part and jointgeneration with ISArt. Scene generation supports two complementary routes:Unravel reconstructs editable scenes from images, while SimForge buildssingle-room and multi-room environments from text. A graph-native harnesscoordinates construction, validation, andbounded repair, routing failures to the responsible module while retainingunaffected scene state. PolicyForge binds validated worlds to tasks and robotembodiments to produce replayable demonstrations. Evaluations cover assetgeometry, scene quality, and downstream policy learning. On MuJoCo-basedLIBERO-Plus, co-training with Isaac Sim demonstrations improves the overallsuccess rate of a World-Action Model (WAM) from $77.17\%$ to $89.43\%$. Goal and spatialsuccess improve by $31.66$ and $6.25$ percentage points, respectively.These results support the utility of the generated data for cross-simulatorpolicy training, with more limited gains on long-horizon tasks.

1. Introduction

Robot-training data needs interactive, physically valid worlds and repeatable tasks, but real-world collection and manual simulation preparation limit coverage. Awomo-SimDataEngine connects asset and scene generation to validated, replayable demonstrations, with a 12.26-point overall WAM success gain from Isaac Sim co-training on LIBERO-Plus.

Motivation

  • Visually plausible scenes can still contain invalid collision geometry, inaccessible interiors, incorrect joints, unstable placements, or unreplayable initial states, affecting data collection and policy learning.Real-world demonstration collection is costly, and manually preparing simulated environments limits synthetic-data scale and diversity.Source: §1 · pp. 1–2

System overview

  • Awomo-SimDataEngine connects asset generation, scene generation, and demonstration synthesis through a Graph-Native Harness for execution and repair.Unravel reconstructs editable scenes from images, while SimForge builds single-room and multi-room environments from text.Source: §1 · p. 2

Execution and repair

  • The harness routes construction failures to responsible modules, preserves unaffected work, and revalidates dependencies before proceeding.Examples include acquiring a missing asset, changing an intersecting object's placement, or revising an incorrect joint.Source: §1 · p. 2, Figure 1 · p. 3

Demonstration synthesis

  • PolicyForge binds accepted worlds to tasks, robot embodiments, and reset specifications, then uses scripted or motion-planned teachers to synthesize demonstrations.The contribution list describes the demonstrations as replayable across scenes, tasks, and robot embodiments.Source: §1 · pp. 2, 4

Experimental evidence

  • 77.17% to 89.43%: Isaac Sim co-training raises WAM overall success on MuJoCo-based LIBERO-Plus by 12.26 points at the best evaluated checkpoints within 13K steps.Goal and spatial generalization improve by 31.66 and 6.25 percentage points, respectively.Source: §1 · p. 3

Contributions

  • ISArt adds structure-grounded articulated-object generation, while shared asset services validate and prepare assets for simulator use.The asset module combines retrieval and generation with common geometry, collision, and physical checks.Source: §1 · pp. 2–3

2. Related Work

The related work spans interactive assets, scene construction, agentic execution, and demonstration synthesis. Awomo-SimDataEngine connects these areas through shared asset services, two scene-generation routes, a repair-oriented execution harness, and PolicyForge.

2. Related Work

  • The paper positions Awomo-SimDataEngine as connecting asset and scene generation with demonstration synthesis, while distinguishing world construction from downstream policy measurements.Its workflow links object identities, coordinate frames, physical representations, task requirements, and validation results.Source: §2 · p. 4

2.1. 3D assets and interactive environments

  • Prior visual asset generators supply candidate geometry, while Awomo-SimDataEngine adds canonical-orientation, extent, collision, and physical-attribute checks; ISArt links structural planning to executable articulation.The cited visual asset methods include Hunyuan3D, TRELLIS, and SAM 3D [Tencent Hunyuan3D Team, 2025a,b; Xiang et al., 2024; SAM 3D Team et al., 2025].Source: §2.1 · p. 4

2.2. Scene synthesis and single-view construction

  • Unravel uses object removal and fixed-camera point evidence for image-grounded reconstruction, while SimForge builds text-driven single-room and multi-room scenes.The single-image route does not imply multi-view reconstruction or uniquely recoverable metric geometry from an unconstrained image.Source: §2.2 · pp. 4–5

2.3. Agentic generation and execution harnesses

  • The execution contract records typed state, evidence, responsible modules, affected work, retry budgets, and checkpoints, reusing preserved state only when dependencies remain valid.These mechanisms support recovery within a run, but policy experiments do not independently establish harness efficiency or learning in the orchestrator.Source: §2.3 · p. 5

2.4. Integrated worlds and policy-data synthesis

  • PolicyForge prepares worlds and task contracts for scripted or motion-planned teachers, retaining replayable resets and synchronized observations and actions; its contribution is a data pipeline, not a new policy optimizer.It complements demonstration expansion and offline-policy work such as MimicGen, RoboMimic, and Diffusion Policy [Mandlekar et al., 2023, 2021; Chi et al., 2023].Source: §2.4 · p. 5

3. Overview

Awomo-SimDataEngine accepts text, a single image, or both, then constructs and validates a world for possible task-conditioned demonstration synthesis. Its asset and scene routes feed a harness that tracks validation and repair state before PolicyForge generates successful teacher episodes.

Input and scene-generation routes

  • The scene input is a description, one image, and an optional task request; at least text or image is required, and without a task request the output is a scene-only world.SimForge uses text, while Unravel reconstructs from a single-view image, optionally accompanied by a description.Source: §3 · p. 6

Asset generation

  • Asset requests use retrieval, image-to-3D models, specialized generation, or procedural modeling, with shared validation, canonical transforms, collision construction, physical augmentation, and provenance.ISArt generates textured part meshes, a kinematic tree, and joint parameters through structural planning, structure-grounded generation, articulation binding, and simulation validation.Source: §3 · p. 6, Figure 2 · p. 7

Scene generation

  • Unravel progressively removes objects, fuses fixed-camera point maps at newly exposed surfaces, and recomposes editable assets in reverse removal order before settling and visual audit.SimForge instead plans room geometry and object placement from text for single-room and multi-room scenes; both routes undergo validation.Source: §3 · p. 6

Graph-Native Harness

  • The harness tracks typed state, validation evidence, responsible modules, repair scope, attempt budgets, and checkpoints, while stage dependencies determine which records become invalid after edits.World acceptance requires hard validation and simulator packaging; a saved render or mesh is insufficient.Source: §3 · p. 6

Policy generation and evaluation

  • PolicyForge binds an accepted world to a task, robot embodiment, and reset specification; successful teacher episodes become demonstrations, and scene, task, initial-state, and language variants diversify training data.The report measures downstream WAM co-training in Isaac Sim-generated environments and evaluation on MuJoCo-based LIBERO benchmarks, distinct from world validity and episode success.Source: §3 · pp. 6–7

4. Asset Generation

Awomo-SimDataEngine turns object requests into validated simulator assets through retrieval, generation, and shared geometry, collision, and physical checks. ISArt extends this process to articulated objects, while standardization preserves reusable geometry and auditable transforms.

4. Asset Generation

  • Asset candidates come from retrieval, image-to-3D or specialized generation, and procedural modeling, then enter shared validation with available semantic, geometric, and physical information.Articulated objects use ISArt; retrieved or procedurally modeled objects remain alternatives when generation fails required checks.Source: §4 · p. 7

4.1. Single-Image Articulated Assets with ISArt

  • Given an object-centric image, ISArt produces textured part meshes, a URDF kinematic tree, and joint origins, axes, and ranges.Its stages plan geometric–kinematic structure, generate structure-grounded parts, bind articulation, and validate in simulation.Source: §4.1 · p. 7
  • ISArt predicts part and joint structure, conditions part-mesh generation on that structure, then refines and simulation-checks joint motion and collisions.Persistent articulation failures can return to structural planning or geometry generation; invalid candidates remain rejected.Source: §4.1 · pp. 7–8
  • Part-geometry Chamfer Distance is 0.163 for ISArt versus 0.268 for TRELLIS.2 + P3-SAM + X-Part on 60 articulated objects.With fixed ground-truth part geometry, URDF binding reports 100% success; this measures binding, not end-to-end asset generation.Source: §4.1 · p. 8

4.2. Standardization, Physical Augmentation, and Reuse

  • Canonical transforms, measured placement extents, convex-decomposition collision proxies, and physical attributes are retained with reusable asset geometry.Visual meshes, collision proxies, link frames, and simulator representations preserve auditable scale and transform relationships.Source: §4.2 · p. 8

5. Scene Generation

Scene generation offers two independent routes: Unravel reconstructs editable scenes from a single image, while SimForge builds single-room and multi-room worlds from text. Both use shared assets and validation, but their construction evidence and evaluations remain distinct.

5.1. Image-to-Scene Generation: Unravel

  • Unravel progressively removes objects from one image, fuses point-map evidence from newly exposed regions, and reassembles independently editable assets.It is training-free and accepts an optional description and task constraints.Source: §5.1 · p. 9, §5.1.1 · p. 9

5.1.1. Verified Layer-Wise Deconstruction

  • Verified deconstruction removes exposed objects while preserving the fixed camera; inpainted hidden surfaces remain generated hypotheses, not new physical measurements.Point-map fusion aligns rounds through stable anchors and retains earlier geometry while adding valid points from regions exposed by edits.Source: §5.1.1 · p. 9

5.1.2. Registration and Support-Aware Recomposition

  • Unravel attributes fused points to object clusters, registers assets to partial observations, and settles movable objects under support, collision, and observation constraints.Reverse-order restoration is an assembly prior, not proof of support; settling and a render-and-inspect audit check the assembled scene.Source: §5.1.1 · p. 9, §5.1.2 · pp. 10–11

5.1.3. Module-Level Reconstruction Evidence

  • Unravel increases mean asset-fidelity scores from 4.7 to 8.2 for VLM ratings and from 4.9 to 8.6 for human ratings across five scenes and 160 objects.Mean scene point count rises from 1.53M to 2.81M, but additional points may come from generated completion rather than independent geometric ground truth.Source: §5.1.3 · p. 11
  • On 20 scenes, Table 2 reports that Unravel beats GPT-6 Astra on all three image-level metrics: CLIP-I, LPIPS, and Edge IoU.Renders use the recovered photo viewpoint; CLIP-I measures semantic content, LPIPS perceptual distance, and Edge IoU layout and composition.Source: §5.1.3 · p. 11, Table 2 · p. 11

5.2. Text-to-Scene Generation: SimForge

  • SimForge plans room geometry, openings, object requests, and spatial relations from text, then places retrieved or generated assets and runs physics checks and critique.It supports multi-room topology and connections; a reference scene image is not required.Source: §5.2 · p. 12, §5.2.1 · p. 12

5.2.2. Reusable Assets and Scene Variants

  • Scene instances retain pose, support, joint state, and task role separately from shared asset data, enabling local substitutions and edits with affected dependencies rechecked.Accepted worlds can be bound to tasks and robots by PolicyForge, but scene generation alone does not imply a successful demonstration.Source: §5.2.2 · p. 12

5.2.3. Module-Level Scene-Quality Evidence

  • SimForge improves count agreement over SceneSmith by 6.5 percentage points at room level and 6.4 at house level, and accessibility by 14.7 and 13.3 points, respectively.Detected collision is 0% and stability is 100% under the stated checks; room and house cohorts are reported separately.Source: §5.2.3 · p. 12

6. Graph-Native Harness for Execution and Repair

The Graph-Native Harness coordinates construction, validation, and recovery across asset, scene, packaging, and demonstration modules. It uses explicit ownership, bounded retries, and checkpoints while distinguishing world acceptance from successful robot episodes.

6. Graph-Native Harness for Execution and Repair

  • The harness connects asset generation, Unravel, SimForge, simulator packaging, and PolicyForge through execution edges that specify inputs, validation requirements, responsible modules, and resume records.The execution graph supports recovery for both text-driven construction and image-grounded recomposition.Source: §6 · p. 12
  • Validation evidence routes failures to the responsible module, while repairs invalidate dependent state and preserve other work only when its inputs remain valid.Examples include returning an incorrect joint to ISArt, a removal edit to its Unravel round, and unstable placement to recomposition and settling.Source: §6 · pp. 13–14
  • Retries are bounded by scope, validated checkpoints can be restored, and exhausting a budget records terminal failure rather than accepting an unmet requirement.Optional-object removal is recorded as degradation and cannot satisfy an unchanged requirement.Source: §6 · pp. 13–14
  • World acceptance requires applicable checks and a valid simulator package, whereas dataset admission additionally requires a successful task rollout.PolicyForge binds accepted worlds to embodiments and reset specifications; failed rollouts return blocker and provenance information, and world revisions invalidate dependent resets and rollouts.Source: §6 · p. 14
  • Section 8 measures co-training data utility, not repair efficiency or the harness’s independent contribution.The harness's execution and recovery mechanisms are not separately evaluated by that measurement.Source: §6 · p. 14

7. Policy Generation via Automated Demonstration Synthesis

PolicyForge turns validated worlds and task contracts into successful, replayable demonstrations across robot embodiments. Its task routing, explicit contract, failure feedback, and scene/task cousins structure the process for reproducible, diversified data generation.

7. Policy Generation via Automated Demonstration Synthesis

  • PolicyForge converts validated worlds and user-level task specifications into multi-embodiment demonstration data for policy learning.It supports Franka, Piper dual-arm, Agibot G1, and Agibot G2 in Isaac Sim.Source: §7 · p. 14

7.1. Pipeline overview

  • The pipeline analyzes and decomposes tasks as needed, routes atomic tasks to execution backends, records synchronized episodes, and exports only task-success-validated episodes.Execution can use inverse kinematics, cuRobo, RL policies, or custom scripts; accepted episodes are exported in LeRobot format.Source: §7.1 · p. 16, Figure 5 · p. 16

7.2. Task contract

  • The task contract records the compiled world, task predicates, robot interface, reset state, randomization ranges, episode horizon, and seed for reproducibility and verification.Its identities refer to exported objects, supports, containers, and articulations rather than hidden generator internals.Source: §7.2 · p. 17

7.3. Multi-embodiment adaptation

  • Embodiment adapters bind robot-specific kinematics, sensors, grippers, and action conventions, allowing a shared scene–task pair to serve heterogeneous robots.Action and proprioceptive representations can be mixed at training time after lightweight convention alignment.Source: §7.3 · p. 17

7.4. Failure feedback

  • When rollout failures identify a world-owned issue, feedback targets the smallest responsible graph edge, and revisions must pass the same validation and export gates.Failure categories include task specification, affordance, geometry/support, physics, reset, and controller.Source: §7.4 · p. 17

7.5. Data cousins: scaling diversity along scene and task axes

  • Data cousins vary scene and task axes separately, while Easy, Lite, and Hard configurations span distinct scene-by-task difficulty combinations.Ultra additionally mixes data sources and uses robot initial-state perturbations and language-instruction augmentation.Source: §7.5 · p. 17

8. Experiments and Results

The experiments test whether Isaac Sim demonstrations improve WAM performance on MuJoCo-based LIBERO-Plus, using each configuration’s best overall checkpoint through 13K steps. SimData-Ultra improves overall and goal, spatial, and object generalization, while the long-horizon gain is smaller; the evidence does not establish real-world transfer or isolate Ultra’s components.

8.1. Setup

  • The study evaluates Isaac Sim co-training on MuJoCo-based LIBERO-Plus using the best overall checkpoint through 13K steps for each configuration.The baseline uses benchmark demonstrations only; Ultra combines SimData-Hard, additional in-house demonstrations, and benchmark data with initial-state and language augmentation, so component effects are not isolated.Source: §8 · pp. 17–18, §8.1 · p. 18

8.2. Cross-Simulator Co-training Improves Overall Success

  • 12.26 percentage points: SimData-Ultra reaches 89.43% overall success versus 77.17% for benchmark-only training at the best evaluated checkpoints through 13K steps.All configurations use the same 1,003-episode evaluation set.Source: Table 4 · p. 18, §8.2 · p. 18, Figure 6 · p. 19

8.3. Goal Generalization Shows the Largest Gain

  • 31.66 points on goals, 6.25 on spatial, and 8.34 on object generalization: SimData-Ultra improves each over the baseline at selected best-overall checkpoints.Long-horizon LIBERO-10 rises by only 1.99 points, from 85.71% to 87.70%; the authors attribute the limited gain to limited long-horizon sequential supervision.Source: §8.3 · pp. 18–19, Figure 7 · p. 19

8.4. Training Progress and Best-Checkpoint Performance

  • 84.75% at 5K steps: Ultra exceeds Hard’s highest listed score of 81.06% at 13K steps, and reaches its 89.43% best within the 13K-step budget.The curves indicate earlier attainment of high success, but do not measure wall-clock savings or data-generation cost.Source: Figure 8 · p. 20, §8.4 · p. 20

8.5. Summary and Scope

  • The reported policy results support cross-simulator data utility, but do not establish physical-world transfer, generator rankings, or harness efficiency.The report also lacks repeated-training-seed confidence intervals, and Ultra is not an ablation of system modules or individual augmentations.Source: §8.5 · p. 20

9. Limitations and Conclusion

Awomo-SimDataEngine combines asset, scene, and demonstration generation through a graph-native harness with explicit evidence, bounded repair, and checkpoints. Its evaluations support geometry, fidelity, and cross-simulator policy gains, while identifying limits in validation, task coverage, and evaluation scope.

9.1. Limitations

  • Hidden geometry and metric scale remain unverifiable from the visual evidence, and the current representations primarily support rigid objects and articulated mechanisms.Deformable interaction requires additional modeling.Source: §9.1 · pp. 20–21
  • Harness acceptance depends on applicable checks and the simulator contract; validators may miss or misattribute errors, and finite repair budgets can leave requests unresolved.The experiments do not quantify repair efficiency, generation cost, or end-to-end accepted-world yield.Source: §9.1 · p. 21
  • PolicyForge demonstrations depend on scripted or motion-planned teachers, feasible resets, and task-success predicates, limiting coverage to interactions those teachers can execute and validate.Contact-rich manipulation, dexterous control, and extended task sequences remain challenging.Source: §9.1 · p. 21
  • The WAM comparison evaluates combined Ultra data without repeated-seed intervals or controlled module and augmentation ablations; real-robot transfer remains unmeasured.Module and policy evaluations use distinct protocols, and gains vary between LIBERO-Plus and LIBERO-Pro.Source: §9.1 · p. 21

9.2. Conclusion

  • ISArt’s geometry and Unravel’s asset-fidelity evaluations improve, while the combined co-training data shows downstream utility under its evaluation protocol.ISArt reports CD 0.163 versus 0.268 for TRELLIS.2 + P3-SAM + X-Part; Unravel’s blind human asset-fidelity scores rise from 4.9 to 8.6 over 160 objects.Source: §9.2 · p. 21
  • Reported results show a 12.26-point overall gain, with improvements of 31.66 points on goals and 6.25 points on spatial generalization under their respective protocols.Source: §9.2 · p. 22

A.1. Transition and Repair Semantics; A.3. Loop-to-Graph Atomicization

  • The Harness turns generation and repair into typed graph transitions with evidence-bearing ownership, bounded re-entry, and explicit checkpoint decisions.A single control semantics applies across domain subgraphs, with executors attached to transitions rather than serving as competing control abstractions.Source: §A.1 · p. 28, §A.3 · p. 29

A.2. Control Plane; A.4. Evidence Routing; A.5. Scoped Re-entry; A.6. Checkpoint, Rollback, and Degradation; A.7. Nested Subgraphs

  • Evidence routes failures to the smallest responsible owner while preserving unaffected validated state; checkpoints restore the last valid candidate when a change violates a hard predicate.When ownership is ambiguous, the Harness preserves that ambiguity rather than assigning a precise owner.Source: §A.4 · p. 29, §A.2 · p. 29, §A.5 · p. 30

A.8. Multimodal Condition and Typed Specification

The harness separates typed conditions, world state, and evidence, and promotes worlds only through a strict export gate. ISArt conditions multi-part geometry generation on structural and visual information.

A.8. Multimodal Condition and Typed Specification

  • Modality enters through input adapters: SimForge accepts text, Unravel accepts a scene image with optional description, and ISArt uses object-centric references.The conceptual condition groups do not prescribe a serialization format.Source: §A.8 · p. 30, Table 6 · p. 30

A.9. Harness State Fields (implementation: PipelineState); A.10. SceneState and Object Fields

  • PipelineState names the implementation state exchanged by graph nodes; partial snapshots may support diagnosis but are not strict success claims.SceneState is a candidate-world checkpoint that distinguishes asset templates from scene instances and preserves lineage for repair and export.Source: §A.9 · p. 30, §A.10 · p. 31, Table 7 · p. 31

A.11. Strict Export Predicate

  • Accepted-scene success requires a conjunctive export gate; a render, intermediate package, or best-effort artifact alone does not qualify.Full success additionally requires experiment-level loadability, visual/collision alignment, task-entity coverage, and reset replay.Source: §A.11 · p. 32

A.12. Evidence Package Organization; A.13. Per-Run Evidence Checklist

  • The evidence package records failures rather than treating absent later-stage evidence as success, and requires condition, state, scene, asset, validation, and rollout records.Policy records join world, task, robot, simulator, seed, checkpoint, terminal status, blocker, and recovery evidence.Source: §A.12 · p. 32, §A.13 · pp. 32–33, Table 9 · p. 32

B.1. Structural Encoding and Geometry Conditioning

  • ISArt injects semantic, spatial, and kinematic priors into a multi-part DiT while retaining visual conditioning from the input image and optional open-state reference.Structural attributes include part semantics and connections, joint type and axis, and part and cavity boxes; the method handles at most one predicted cavity.Source: §B.1 · p. 34
  • Its backbone combines within-part local attention, cross-part global attention, and image cross-attention; structural cross-attention lets geometry tokens retrieve relevant structural information.A shared part-level structural bias is also added at each DiT layer, complementing token-level retrieval.Source: §B.1 · p. 35

B.2. Training and Failure-Driven Data Expansion

This block covers ISArt training and binding, asset and scene records, reconstruction and settling, and PolicyForge’s world-to-policy handoff. Together, the passages specify how candidates and demonstrations retain validation and provenance.

B.2. Training and Failure-Driven Data Expansion

  • The ISArt structural encoder, E/C modules, and attention LoRA parameters are trained with rectified-flow velocity loss.Training totals approximately 2,400 assets and 25,000 steps, taking about 80 hours on four NVIDIA A100 GPUs.Source: §B.2 · p. 35

B.3. Binding Memory and Validation Records

  • ISArt’s binding agent uses motion summaries as priors, binds generated parts to URDF links, and validates motion renders before parameter repair or fallback.Records preserve candidate structure, geometry, motion render, critique, and adoption or rejection reason.Source: §B.3 · p. 36

B.4. Asset Registry Fields

  • The Asset Registry preserves stable identities, lifecycle and provenance, geometry and physics contracts, validation history, and rejected or missing candidates as negative evidence.The registry contract covers adoption as well as rejection, placeholders, and missing collision representations.Source: §B.4 · pp. 36–37, Table 10 · p. 36

C.1. Verified Localization and Deconstruction

  • Unravel verifies target localization through independent local probes and protects newly identified movable objects from accidental deletion.Same-category boxes above IoU 0.85 are treated as duplicates; local agreement thresholds determine whether boxes are confirmed or replaced.Source: §C.1 · p. 38

C.2. Point Fusion, Registration, and Settling

  • Unravel fuses only newly exposed valid points, registers objects using bidirectional median Chamfer distance and truncated similarity ICP, then settles movable objects under physical and observation constraints.Observation-fit tolerances are relative to the observation, not universal bounds on every displacement; audit edits trigger renewed feasibility checks.Source: §C.2 · pp. 38–39

C.3. Support and Coordinate Records

  • Coordinate and support records retain frame transforms, measured local geometry, parent-surface lineage, and accepted pose corrections.Occlusion order alone cannot establish contact.Source: §C.3 · p. 39

D. PolicyForge

  • PolicyForge generates demonstrations from validated worlds and task contracts, while evaluating world acceptance separately from episode success.Source-specific robot, camera, action, and reset adapters preserve trajectory provenance.Source: D. PolicyForge · p. 40

D.1. World-to-Policy Contract; D.2. Rollout Metadata

  • The policy interface admits accepted worlds only after scene, task, simulator, and reset contracts pass; rollout records join each episode to its world, task, policy, robot, simulator, seed, and checkpoint.The evaluated WAM configuration is documented separately from the algorithm-neutral interface.Source: §D.1 · p. 40, §D.2 · p. 40, Table 11 · p. 40

D.3.2. Training Hyperparameters

The evaluation passages describe co-training data and checkpoint protocols alongside separate module-level geometry, binding, and conditioning experiments. They distinguish the conditions each result can support.

D.3.2. Training Hyperparameters

  • LIBERO-Plus experiments scale to four nodes with 32 GPUs, while LIBERO-Pro uses one node with eight GPUs.The LIBERO-Plus learning rate follows square-root scaling to compensate for the sixfold larger global batch size.Source: §D.3.2 · p. 41, Table 14 · p. 41

D.3.3. Co-training Data Configuration

  • SimData-Ultra combines SimData-Hard, in-house complex-scene demonstrations, and LIBERO-Plus data with robot initial-state and language augmentation.Co-training uses a reported 1:1 effective-frame ratio; exact source weights and augmentation settings are not fully enumerated.Source: §D.3.3 · p. 41, Table 15 · p. 42

D.3.4. Evaluation Protocol

  • LIBERO-Plus comparisons use each configuration’s highest-overall-success evaluated checkpoint up to and including 13K steps, and evaluate 1,003 episodes.The reported results preserve checkpoint identity and do not impute unlisted final results.Source: §D.3.4 · p. 42, Table 16 · p. 42

E. Experimental Results

  • Module evaluations measure geometry, binding, and single-view reconstruction separately from downstream policy utility; their results are not pooled into a single system score.Unreported values are not inferred.Source: E. Experimental Results · p. 43

E.1. ISArt Results

  • ISArt geometry is evaluated on 60 objects across 12 categories against corresponding ground-truth part meshes using F@0.05, F@0.1, and Chamfer Distance.Binding uses ground-truth part geometry to isolate kinematic estimation; its success rate does not measure end-to-end asset generation.Source: §E.1 · p. 43, Table 17 · p. 43, Table 18 · p. 43
  • Conditioning ablations find that C mainly improves global part layout and structural consistency, while E provides complementary local-detail refinement.Inference variants share a trained checkpoint; the controlled training comparison uses C-only and E+C variants from the same backbone for 1,000 steps on 100 trashbin samples.Source: §E.1 · p. 43, Figure 11 · p. 44

E.2. Unravel Results

E.2 compares Unravel’s deconstruction process, interprets its coverage measures, and presents fidelity protocols and qualitative examples. E.3 describes LIBERO-Plus and LIBERO-Pro policy-result tables and checkpoint selection.

E.2. Unravel Results

  • The controlled comparison uses five scenes and 160 objects, with Unravel fusing four to eight deconstruction rounds while shared components run off the shelf without fine-tuning.Both variants share segmentation, asset generation, and point estimation; the direct variant uses only the input image.Source: §E.2 · p. 44
  • Objects removed after round one are approximately one third of the test set; their direct mask area averages 77.2% of Unravel’s reference and direct point coverage averages 76.9%.Unravel’s UCR is zero because its own masks and clusters are the reference; the statistics measure relative coverage, not accuracy against independent ground-truth geometry.Source: §E.2 · p. 44, Table 19 · p. 44
  • Blind fidelity scores average judgments of two meshes rendered from four views for each of 160 objects, using the object’s best-observed-round crop as reference.The protocol reports no confidence intervals or rater-agreement statistic, and the reference may include generated completion.Source: §E.2 · p. 44
  • Qualitative examples contrast direct-mask and best-observed-round assets, isolate placement using shared meshes and masks, and show six indoor-photo reconstructions and per-round edit traces.The six-photo examples are excluded from the five-scene quantitative aggregate.Source: Figure 12 · p. 45, §E.2 · p. 45, Figure 13 · p. 46

E.3. Policy Co-training Results

  • LIBERO-Plus tables cover overall success across steps, per-suite rates at 13K and across steps, and baseline-relative gains at 13K; the main-text comparison selects each configuration’s best overall checkpoint through 13K.The overall-success table evaluates 1,003 episodes across four generalization suites.Source: §E.3 · p. 49, Table 20 · p. 49, Table 21 · p. 49
  • LIBERO-Pro tables report overall success across steps and final-step rates by five generalization dimensions and base suite; Ultra’s LIBERO-Pro peak is at 13K, not its final checkpoint.The dimension table uses 200 episodes per dimension per configuration, and the base-suite table uses 250 episodes per suite.Source: Table 24 · p. 51, Table 25 · p. 51, Table 26 · p. 51

Related papers

Loading 2610.02274v1…