PaperlayerView exact revision on arXiv

Source-linked AI summary

SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes

Nicholas Pfaff, Thomas Cohn, Sergey Zakharov, Rick Cory, Russ Tedrake

arXiv:2602.09153v2cs.ROcs.AIcs.CVcs.GR

TL;DR

Existing simulation environments lack the dense clutter, articulated furniture, and physical properties needed to represent real indoor spaces for scalable home-robot training and evaluation. SceneSmith uses hierarchical designer–critic–orchestrator construction with integrated asset generation to create simulation-ready scenes from natural language. It generates denser, physically stable scenes that human evaluators prefer over baselines and supports automatic robot-policy evaluation.

  • Problem

    Existing simulation environments are sparse and poorly matched to real human indoor spaces, while manual environment design is costly, time-consuming, and difficult to scale.

  • Method

    SceneSmith hierarchically constructs scenes from natural language through designer, critic, and orchestrator interactions while routing static objects to text-to-3D synthesis and articulated objects to dataset retrieval.

  • Results

    3-6x more objects, <2% inter-object collisions, and 95.6% static stability, with 92% realism and 91% prompt-faithfulness average win rates against baselines.

  • Takeaways & Limitations

    SceneSmith provides simulation-ready environments for scalable robot training and evaluation, including an end-to-end pipeline from natural-language tasks to automatic success verification.

  • Takeaways & Limitations

    Current text-to-3D methods do not reliably produce articulated structure and kinematics suitable for robotics simulation, so articulated objects are retrieved from a pre-authored library.

Abstract

from arXiv · show

Simulation has become a key tool for training and evaluating home robots at scale, yet existing environments fail to capture the diversity and physical complexity of real indoor spaces. Current scene synthesis methods produce sparsely furnished rooms that lack the dense clutter, articulated furniture, and physical properties essential for robotic manipulation. We introduce SceneSmith, a hierarchical agentic framework that generates simulation-ready indoor environments from natural language prompts. SceneSmith constructs scenes through successive stages$\unicode{x2013}$from architectural layout to furniture placement to small object population$\unicode{x2013}$each implemented as an interaction among VLM agents: designer, critic, and orchestrator. The framework tightly integrates asset generation through text-to-3D synthesis for static objects, dataset retrieval for articulated objects, and physical property estimation. SceneSmith generates 3-6x more objects than prior methods, with <2% inter-object collisions and 96% of objects remaining stable under physics simulation. In a user study with 205 participants, it achieves 92% average realism and 91% average prompt faithfulness win rates against baselines. We further demonstrate that these environments can be used in an end-to-end pipeline for automatic robot policy evaluation.

1. Introduction

SceneSmith addresses the mismatch between simplistic simulated rooms and the dense, diverse, physically complex environments needed for home-robot training and evaluation. It combines hierarchical agentic scene construction with integrated asset generation to produce simulation-ready environments at scale.

  • Motivation: Simulation provides scalable robot training and evaluation through rapid iteration, controlled experiments, and safe testing of failure modes.
  • Motivation: Existing simulation environments are sparse, static, and poorly matched to the dense clutter and articulated furniture found in real homes.This mismatch limits how well simulation reflects real-world indoor scene distributions, while manual design remains costly and difficult to scale.
  • Approach: SceneSmith generates room- or house-level environments from natural-language descriptions through grounded, sequential decisions intended to preserve prompt alignment and physical feasibility.
  • Approach: SceneSmith combines hierarchical layout and furnishing stages with designer, critic, and orchestrator VLM interactions.Its asset pipeline combines text-to-3D synthesis for static objects, articulated-object retrieval, and physical-property estimation.
  • Results: 71.1 vs 11-23 objects per room on average, with 92.2% realism and 91.5% prompt faithfulness win rates against baselines.These results come from evaluations across 210 room- and house-level prompts and a user study with 205 participants.
  • Contribution: SceneSmith integrates asset generation and routing with scene construction to support dense, articulated, physically valid environments for scalable robot training and evaluation.

2. Related Work

Prior indoor scene synthesis methods use procedural rules, learned scene distributions, or language-guided generation, but agentic approaches increasingly add iterative refinement. Existing methods still face limitations in semantic flexibility, fine-grained spatial reasoning, and physical feasibility.

  • Indoor Scene Synthesis: Procedural methods scale through hand-crafted object-relationship rules but have limited semantic expressivity and extensibility across scene types.
  • Indoor Scene Synthesis: Data-driven methods learn spatial patterns from 3D scene datasets, while language-guided methods add open-vocabulary semantic priors and natural-language controllability.
  • Indoor Scene Synthesis: Language-guided approaches often struggle with fine-grained spatial reasoning and physical feasibility, producing implausible placements or object interpenetrations.
  • Agentic Scene Synthesis: Agentic scene synthesis shifts from single-shot generation toward iterative refinement involving planning, action, reflection, or designer–critic interaction.SceneSmith adapts the designer–critic refinement pattern for scene-level generation.

3. Agentic Scene Construction for Simulation-Ready Environments

SceneSmith constructs simulation-ready indoor scenes as a hierarchical tree of agentic stages, progressively adding architecture, room furnishings, and manipulable objects. Its routing pipeline combines generated and retrieved assets, validation, and physical-property augmentation, then connects scenes to automated robot-policy evaluation.

  • Agentic Construction: SceneSmith decomposes scene creation into layout, furnishing, object-population, and refinement decisions, with each stage using designer, critic, and orchestrator agents.
  • Hierarchical Scene Representation: Each room is represented by geometry and objects whose assets include visual geometry, collision geometry, physical properties, and joint definitions when articulated.
  • Hierarchical Scene Representation: The hierarchy begins with architectural geometry, independently populates each room with furniture and fixtures, and branches from supporting entities to add small manipulable objects.
  • Agentic Construction: Agents propose, evaluate, and control structured scene modifications using observations, visual checks, relational placement tools, and persistent stage-specific memory.
  • Asset Generation and Routing: The asset router selects text-to-3D synthesis, articulated-object retrieval, or thin-covering strategies, then validates candidate assets and reroutes or regenerates failures.
  • Robot Policy Evaluation: An end-to-end pipeline converts natural-language robot tasks into generated environments, robot execution, and automated verification without manual environment or evaluation-predicate design.The evaluator reasons jointly over symbolic scene state and rendered visual observations, supporting open-ended tasks at the cost of determinism.

4. Evaluation

SceneSmith is evaluated against external baselines, ablations, automated metrics, and robot-policy evaluation settings. It produces denser, more physically feasible scenes, receives stronger human preferences, and supports policy discrimination and interactive simulation.

  • Experimental setup: The evaluation compares SceneSmith with five external baselines across 210 prompts spanning room types, object density, themed scenes, and house-level environments.The study also includes six ablations, pairwise human judgments, SceneEval metrics, and physics-based collision and stability measures.
  • Human evaluation: 92.2% realism and 91.5% prompt-faithfulness win rates are achieved against room-level baselines, while house-level comparisons against Holodeck reach 80.3% and 84.7%.The room-level results are significant at p < 0.001; the smallest room-level margins are against HSM.
  • Ablations: NotGenerated, NoAssetValidation, and NoObserveScene produce the largest ablation effects, with realism scores of 63.8%, 63.0%, and 61.5%, respectively.NoCritic, NoSpecializedTools, and NoAgentMemory show smaller, nonsignificant effects under the study power.
  • Automated evaluation: SceneSmith generates 71.1 objects per room versus 11–23 for baselines, with 1.2% collision rate and 95.6% stability versus 3–29% and 8–61%.Its lower accessibility and navigability scores are attributed to the substantially higher object density; house-level scenes contain 214 versus 81 objects, with 0.9% collisions and 79.8% stability.
  • Automated evaluation: SceneSmith achieves the best performance on CNT, ATR, OOR, OAR, COL, and STB, including a 2.2x improvement on OOR over the best baseline.Remaining collisions average 3.8mm penetration depth, 3–12x shallower than baselines.
  • Robot policy evaluation: The standard policy reaches 16% success versus 12% for a degraded variant, while manual verification finds 99.7% agreement between automated evaluator judgments and human labels.The evaluation covers 100 generated scenes and four pick-and-place tasks.
  • Robot simulation demonstrations: Teleoperation and zero-shot rollouts show an RB-Y1 manipulating an articulated cabinet and a prior policy moving an apple from a bowl to a cutting board.The zero-shot policy predates SceneSmith and has no SceneSmith-specific training data.

5. Conclusion

SceneSmith generates dense, physically valid indoor scenes through hierarchical agentic construction and an integrated asset pipeline. The reported results show higher object density, low collisions, strong stability, favorable human preferences, and suitability for interactive robot simulation.

  • Conclusion: SceneSmith combines hierarchical designer-critic-orchestrator interactions with generative and retrieved assets to produce simulation-ready scenes from natural-language prompts.The asset pipeline supports static-object text-to-3D synthesis and articulated-object retrieval.
  • Conclusion: Teleoperation and zero-shot policy rollouts indicate that SceneSmith scenes support interactive robot simulation beyond static scene-quality metrics.The conclusion frames environment generation as no longer the primary bottleneck for scalable simulation-based robot training and evaluation.

Impact Statement

The work aims to improve simulation environments for robot development and safer evaluation before real-world deployment, and reports no anticipated direct negative societal impacts.

  • Impact Statement: The work aims to advance robot development through improved simulation environments, enabling safer evaluation before real-world deployment.
  • Impact Statement: The authors do not foresee direct negative societal impacts from this research.

A. System Architecture

SceneSmith builds scenes through five sequential stages, each coordinated by designer, critic, and orchestrator agents. Specialized asset acquisition, observation, validation, and checkpointed refinement support physically plausible scene construction.

  • Five stages construct layouts, furniture, wall-mounted objects, ceiling-mounted objects, and manipulands.
  • The designer proposes scene modifications, the critic scores quality and suggests improvements, and the orchestrator manages iteration and termination.The critic uses categorical 0–10 scores, while the orchestrator coordinates the critique-and-improve loop.
  • Agents combine structured state queries with visual renders to reason about object metadata, room layouts, and spatial relationships.Structured queries expose poses, dimensions, assignments, and bounding boxes; renders provide annotated multi-view observations.
  • Checkpoint rollback restores the previous scene when refinement decreases quality, while preserving agent context for an alternative attempt.
  • Batch request pipelining overlaps asset generation with mesh post-processing so text-to-3D does not become a throughput bottleneck.The system generated 25 scenes concurrently across 8 NVIDIA L40S GPUs with nearly constant per-scene generation time as parallel scene count increased.
  • Assets are routed among text-to-3D generation, articulated-object retrieval, and procedural thin-covering generation according to object type and stage.Composite requests can be decomposed into individually manipulable assets before acquisition.

B.7. Asset Validation

SceneSmith validates generated and retrieved assets against object identity, appearance, structure, and suitability requirements before scene placement. Validation remains permissive where current generation or limited libraries cannot support exact specifications.

  • The validator checks object type, style consistency, single-object structure, completeness, proportions, and closed articulated states.Examples include rejecting attached manipulands, missing parts, and generated assets with open doors or drawers.
  • Text-to-3D assets are accepted with leniency toward text, labels, metallic materials, and transparent materials that current generation cannot reliably reproduce.
  • Limited articulated-asset libraries require accepting close matches rather than exact specifications, including minor structural variations.Retrieved assets use a more lenient validator focused on object type and basic structure.
  • Failed assets are regenerated or re-retrieved within a configurable retry budget; persistent failures return feedback that lets the designer request alternatives or adjust the scene.
  • Layout generation: The layout agent derives room-specific prompts from the global prompt and propagates them to downstream agents for consistent furnishing.
  • Layout generation: Room placement uses best-first backtracking over candidate edge positions, but optimality is limited to the fixed room ordering and discretized position space.As an anytime algorithm, it returns the best layout found within the timeout.

C.5. Critic Evaluation

The layout and furniture critics evaluate semantic, spatial, functional, and physical feasibility through structured observations and specialized checks. These tools identify collisions, blocked passages, disconnected walkable regions, and other placement failures for targeted correction.

  • Layout criticism scores room proportions, spatial flow, natural lighting, material consistency, and prompt following on 0–10 scales.
  • Observation tools combine top-down or perspective renders with ASCII layouts, coordinate grids, bounding boxes, labels, and direction arrows.
  • Facing checks determine whether an object points toward or away from a target and compute the yaw rotation needed for alignment.The procedure uses object yaw, forward direction, target selection, ray–AABB intersection, and an optimal rotation.
  • Snap-to-object resolves overlap, aligns orientation, and removes gaps through collision resolution, optional rotation, and incremental motion toward the target.
  • Reachability models the robot footprint through erosion and dilation, then verifies that the remaining walkable area is a single connected region.Blocking furniture can be identified and returned to the designer for repositioning.

D.5. Critic Evaluation

Furniture, wall-mounted objects, and architectural surfaces are inspected with multi-view observations and stage-specific feasibility checks. Critic criteria emphasize realism, functionality, spatial organization, prompt adherence, and traversability.

  • Critic evaluation: The furniture critic scores realism, functionality, layout, holistic completeness, prompt following, and reachability across six categories.
  • Furniture observations include one top-down view and four corner views, with occluding walls hidden in corner views.
  • Annotated top-down views provide coordinate grids, bounding boxes, object identifiers, direction arrows, and architectural labels for spatial reasoning.
  • Wall placement: Wall-mounted objects use wall-local SE(2) coordinates and are checked for collisions, boundary violations, and overlap with doors, windows, or open connections.

E.2. Critic Evaluation

Critics evaluate wall, ceiling, and manipuland placement using category-specific realism, functionality, layout, completeness, and prompt-following criteria. Agents receive structured and visual observations, then use physics checks and targeted collision reports to refine placements.

  • Wall Object Placement: Wall placement is scored for realism, functionality, layout, holistic completeness, and prompt following, with each category rated 0–10.Wall views combine a top-down room context view with per-wall orthographic views showing grids, openings, identifiers, labels, and nearby furniture.
  • Ceiling Object Placement: Ceiling fixture placement is scored for realism, functionality, layout, and prompt following, including spacing and clearance from tall furniture.Ceiling observations provide top-down and side views with coordinate markers and labeled fixtures.
  • Manipuland Placement: Physics checks report manipuland–manipuland and manipuland–furniture collisions, filtering violations to those addressable during the current placement session.Reported violations include object identifiers for targeted corrections, while observations expose surfaces, bounding boxes, coordinate markers, and articulated interiors.
  • Manipuland Placement: Manipuland placement is evaluated for realism, functionality, layout, holistic completeness, and prompt following, each on a 0–10 scale.The manipuland agent uses entity-specific item, constraint, and style outputs, with nearby furniture guiding orientation and arrangement.

L.4. Experimental Results

The evaluation includes pick-and-place policy tests and comparisons with external scene-generation baselines. SceneSmith’s automatic evaluator closely matches human judgments and distinguishes standard from degraded policy performance.

  • Robot Policy Evaluation: The robot-policy evaluation uses four pick-and-place tasks across 100 generated scenes, with 25 scenes per task.Tasks involve moving fruit, a coke can, a cup, or a water bottle to specified destinations.
  • Robot Policy Evaluation: 299/300 evaluations agreed with human judgments, yielding 99.7% evaluator accuracy.The only disagreement concerned whether fruit landing on a plate edge counted as success.
  • Robot Policy Evaluation: 16% overall success for the standard policy versus 12% for the degraded variant confirms that the evaluation detects their intended performance difference.The comparison is reported across the four pick-and-place tasks.
  • Baseline Comparison: SceneSmith is compared with five external LLM- or VLM-based baselines using their open-source codebases.Holodeck is the only baseline supporting house-level generation, while LayoutVLM is evaluated with two asset libraries.

M.1.2. ABLATIONS

The ablation study removes critics, generated assets, asset validation, specialized tools, visual observations, or agent memory. Evaluation spans 210 prompts and combines semantic scene metrics with physics-based collision and stability measures.

  • Ablation Design: Six ablations test the contributions of critic iterations, generated assets, asset validation, specialized tools, visual observations, and agent memory.The variants are NoCritic, NotGenerated, NoAssetValidation, NoSpecializedTools, NoObserveScene, and NoAgentMemory.
  • Evaluation Setup: 210 prompts are organized into five categories, including room-level benchmark prompts, underrepresented room types, and object-density stress tests.Room-level and house-level prompts are evaluated separately because most baselines support only room-level generation.
  • Evaluation Metrics: Semantic metrics include object count, attributes, object-object relationships, object-architecture relationships, accessibility, and navigability.These metrics are drawn from the SceneEval benchmark.
  • Evaluation Metrics: SceneSmith adds Drake-based physics metrics for collision rate and static equilibrium to assess simulation readiness.Collision rate measures objects penetrating beyond 1 mm, while static equilibrium uses a 5-second gravity simulation with displacement and rotation thresholds.
  • Evaluation Metrics: Baseline scenes are made simulator-compatible by adding collision geometries and estimating mass, center of mass, and inertia with uniform density of 1000 kg/m3.Collision geometries use V-HACD convex decomposition with up to 128 hulls.

N. Human Evaluation

The human study compares SceneSmith with baselines and ablations through blinded pairwise judgments of realism and prompt faithfulness. Across prompt categories, SceneSmith achieves consistently high win rates, especially for dense and stylistically constrained scenes.

  • Analysis: The analysis uses two-sided binomial tests against a 50% win-rate null with Benjamini–Hochberg correction for 28 simultaneous tests.Effect sizes are reported using Cohen’s h, and the study was powered to detect win rates of at least 63% with 95% power.
  • Study Design: 205 participants with complete data evaluated 179 room-level and 31 house-level prompts through blinded pairwise scene comparisons.The study collected 3,051 valid responses, with 15 scene pairs per participant.
  • Study Measures: Participants judged realism by comparing room plausibility, object placement, sizing, and layout coherence.Realism was a forced choice between Scene A and Scene B.
  • Study Measures: Participants judged prompt faithfulness by comparing requested objects, room type, colors, quantities, arrangements, and other described details.Additional objects fitting the room were explicitly allowed, and participants could select Equal.
  • Study Design: The study randomized scene position, comparison order, and question order, and assigned each baseline to an equal number of participants.Each participant saw each prompt at most once.
  • Results by Prompt Category: 94.7% realism win rate was highest for Object Density prompts, while 94.9% faithfulness win rate was highest for Themed Scenes.All room-level prompt categories exceeded 89% on both metrics.

O.1. SceneEval Benchmark Results

SceneSmith is evaluated on SceneEval room prompts using semantic, physical, stability, collision, and generation-cost metrics. Results show benefits from generated assets and agent components, alongside trade-offs between scene density, cost, and latency.

  • Benchmark setup: 179 room-level scenes are evaluated with semantic, navigation, accessibility, collision, stability, and out-of-bounds metrics.Values are reported as means with 95% t-distribution confidence intervals.
  • Ablations: Generated assets improve CNT, OOR, and STB over the NotGenerated ablation, with 82.9% vs 75.7% CNT, 67.6% vs 53.7% OOR, and 95.6% vs 88.4% STB.These differences have non-overlapping confidence intervals.
  • Ablations: The NotGenerated ablation preserves SceneSmith’s construction pipeline while replacing generated assets with retrieved HSSD assets, isolating asset-library effects.Its improvement over HSM and gap to full SceneSmith indicate that both the pipeline and generated assets contribute to results.
  • Physics stability: 12.3 ± 4.5mm mean displacement shows that SceneSmith objects reliably settle within approximately 1cm, while baselines exhibit 382–932mm displacement.The baseline displacement is reported as 30–75x higher, with non-overlapping confidence intervals.
  • Collision depth: 1.2% collision rate and 3.77 ± 1.28mm mean penetration depth indicate rare, shallow overlaps, compared with 10.87–47.05mm baseline penetrations.The reported overlaps are consistent with slight manipuland packing and do not cause simulator instability in the described examples.
  • Generation cost and time: NoCritic reduces cost by 70% and generation time by 51% but lowers average object count from 71.1 to 54.0, illustrating a cost-density trade-off.NoObserveScene is also cheaper and faster, while NoAgentMemory increases cost and latency through repeated failed approaches.
  • Generation cost and time: Designer calls account for 75.9% of SceneSmith’s cost, making the refinement loop the main target for further reduction.The paper identifies smaller or open-weight models as a possible direction for reducing cost and time.

P. Evaluation Prompts

The evaluation spans 210 prompts covering room types, object diversity, density, themes, and house-level layouts. A multi-modal user-study interface supports interactive, click-through, and grid-based scene comparison.

  • P. Evaluation Prompts: The evaluation prompt set contains 210 prompts organized by category.The categories include room-level, type-diversity, object-density, themed, and house-level prompts.
  • P.1. SceneEval-100 (Room): Room-level prompts specify furniture, object placement, spatial relations, and room arrangements across bedrooms, dining rooms, gyms, and other spaces.Examples include beds and nightstands, dining tables and chairs, exercise equipment, desks, and wardrobes.
  • P.2. Type Diversity: Type-diversity prompts cover varied commercial, public, and themed environments such as pet stores, bookstores, grocery stores, pharmacies, museums, and escape rooms.They test whether scenes can represent distinct object compositions and environment types.
  • P.3. Object Density: Object-density prompts require many repeated manipulands, including cans, pasta packs, potatoes, fruits, plates, bowls, cups, and spice jars.These prompts explicitly specify minimum object counts on shelves, tables, and racks.
  • P.4. Themed Scenes: Themed-scene prompts request recognizable styles and franchises, including Star Wars, The Incredibles, and Art Deco.Theme requirements apply to furniture, bedding, rugs, lamps, and seating.
  • P.5. SceneEval-100 (House) and P.6. House-Level: House-level prompts specify multi-room layouts such as bedrooms with walk-in closets, master suites, studio apartments, hotels, apartments, and dental offices.They test room connectivity and coordinated furnishings across multiple spaces.
  • Q. User Study Interface Screenshots: The user study provides 3D, click-through, and grid viewing modes, with controls for independent camera navigation, object hiding, and focused inspection.Alternative modes use nine pre-rendered camera angles or display all angles simultaneously.
  • Q. User Study Interface Screenshots: Participants answer forced-choice realism questions and faithfulness questions that include an Equal option with confirmation prompting.The interface supports side-by-side comparison and reconsideration when participants cannot distinguish which scene better matches the prompt.

R. Additional Qualitative Results

Additional results show SceneSmith across multi-room layouts, diverse room prompts, manipuland arrangements, and highly specific themes. The paper also documents asset, reasoning, and hierarchical-construction failure modes.

  • Qualitative comparisons: SceneSmith and Holodeck are compared on hotel-room connectivity, while further figures compare house-level and room-level generations against baselines.The comparisons cover room connectivity, multi-room scenes, and four diverse room prompts.
  • Generation examples: Additional examples span diverse room types and complexity levels, including a dental office and a boutique hotel suite.The scenes include coordinated multi-room areas such as reception, exam rooms, bedrooms, closets, and bathrooms.
  • Manipuland placement: Manipuland examples populate furniture with individually manipulable objects such as fruit, products, menus, bottles, glasses, books, and gaming-desk items.Each object is represented as a separate simulation asset, including individual books in stacks.
  • Themed room generation: Themed-room examples match requested visual styles including Jurassic Park, Art Deco, steampunk, Star Wars, Frozen, and The Incredibles.Generated assets and retrieved materials are adapted to the requested themes.
  • Asset-level failures: Retrieved articulated assets can be semantically close but incorrect, while static assets may have inaccurate transparent or metallic materials because SAM3D lacks full PBR channels.The paper gives a mini-fridge/storage-cabinet mismatch and opaque glass as representative examples.
  • VLM reasoning failures: VLM processing can misorient wall-mounted lamps and place utensils with identical rather than mirrored orientations on opposite sides of a table.These errors arise during canonicalization or spatial reasoning in asset processing and placement.
  • Hierarchical construction failures: Sequential construction can prevent later stages from revising earlier choices, causing furniture to absorb manipulands or leaving insufficient support space.Examples include a TV embedded in a TV stand and a plant placed beside, rather than on, a missed small surface.

S.2. System Limitations

SceneSmith’s high-fidelity generation prioritizes scene quality over latency and remains bounded by asset coverage, physical-property estimation, and supported simulated phenomena. The reported failure cases include both asset-processing errors and hierarchical reasoning mistakes.

  • Runtime and cost: The NoCritic ablation lowers cost but produces sparser scenes, exposing a quality-cost trade-off.The full system prioritizes high-fidelity, simulation-ready generation over single-scene latency, with cost dominated by the Designer and refinement loop.
  • Articulated-object diversity: Articulated-object diversity is bounded by the retrieval library, with text-to-3D fallback producing static rather than articulated objects.This limits coverage of novel or out-of-distribution mechanisms.
  • Estimated physical properties: Automatically estimated mass, friction, center of mass, and inertia support broad simulation but do not replace measured parameters for contact-rich tasks.The system represents mass as a range suitable for domain randomization and maps estimated material categories to friction coefficients.
  • Scope of simulated phenomena: SceneSmith currently focuses on rigid and articulated objects, leaving deformable, fluid, breakable, and detailed material phenomena outside scope.A formal sim-to-real transfer study with SceneSmith-generated scenes is also future work.
  • Representative failure cases: Representative failures span incorrect asset retrieval, missing material properties, orientation errors, and mistakes in hierarchical placement reasoning.The cases include a cabinet retrieved for a mini fridge, opaque reconstructed glass, unmirrored utensils, and misplaced or incorrectly assembled objects.
Loading 2602.09153v2…