PaperlayerView exact revision on arXiv

Source-linked AI summary

SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learning

Haoqiang Kang, Xiaokang Ye, Yuhan Liu, Siddhant Hitesh Mantri, Lingjun Mao, James Fleming, Drishti Regmi, Lianhui Qin

arXiv:2605.09423v2cs.AI

TL;DR

Embodied agents lack abundant, diverse, automatically generated interactive 3D environments with verifiable tasks and standard learning interfaces. SimWorld Studio uses a self-evolving coding agent to generate Gymnasium-compatible worlds and adaptive curricula, yielding an 18-point Success Rate gain over fixed-environment training and a 40-point gain over an untrained agent.

  • Problem

    Embodied learning lacks abundant, diverse, automatically generated interactive 3D environments with verifiable tasks and standard learning interfaces.

  • Method

    SimWorld Studio uses a self-evolving coding agent with tools, skills, verifiers, and learner feedback to generate and adapt Gymnasium-compatible embodied environments.

  • Results

    18-point Success Rate gain over fixed-environment training and a 40-point gain over an untrained agent were achieved through co-evolution across three embodied navigation case studies.

  • Takeaways & Limitations

    Closed-loop co-evolution makes generated environments more effective for embodied learning and supports transfer to unseen navigation benchmarks.

  • Takeaways & Limitations

    Effectiveness remains bounded by the coding agent’s spatial reasoning, with failures possible in fine-grained spatial planning and precise multi-object arrangement.

Abstract

from arXiv · show

LLM/VLM-based digital agents have advanced rapidly thanks to scalable sandboxes for coding, web navigation, and computer use, which provide rich interactive training grounds. In contrast, embodied agents still lack abundant, diverse, and automatically generated 3D environments for interactive learning. Existing embodied simulators rely on manually crafted scenes or procedural templates, while recent LLM-based 3D generation systems mainly produce static scenes rather than deployable environments with verifiable tasks and standard learning interfaces. We introduce SimWorld Studio, an open-source platform built on Unreal Engine 5 for generating evolving embodied learning environments. At its core is SimCoder, a tool/skill-augmented coding agent that writes and executes engine-level code to construct physically grounded 3D worlds from language/image instructions. SimCoder self-evolves by using verifier feedback (e.g., compilation errors, physics checks, VLM critiques) to revise environments and autonomously add reusable tools and skills to its library. Generated worlds are exported as Gym-style environments for embodied agent learning. SimWorld Studio further enables co-evolution between environment generation and embodied learning: agent performance feedback guides SimCoder to generate adaptive curricula near the learner's capability frontier, so that environments become increasingly challenging as the embodied agent improves. Three case studies on embodied navigation show that self-evolution improves generation reliability, generated environments substantially improve embodied agent performance that generalizes to unseen benchmarks, and co-evolution yields an 18-point success-rate gain over fixed-environment learning and a 40-point gain over an untrained agent.

1 Introduction

SimWorld Studio addresses the shortage of scalable embodied-learning environments with an Unreal Engine 5 platform that automatically generates interactive, physically grounded worlds and exports them through standard learning interfaces. Its SimCoder agent self-evolves through verifier feedback and supports co-evolution with embodied learners by adapting environment difficulty.

  • Motivation: Existing embodied platforms provide infrastructure but largely depend on manually crafted scenes, limiting scalable simulation of physically grounded environments for embodied-agent training and evaluation.Embodied environments must support deployment, action, consequences, observation, and task feedback—not merely visual plausibility.
  • Platform: SimWorld Studio is an open-source Unreal Engine 5 platform whose SimCoder agent writes and executes engine-level code to construct physically grounded environments from language, images, and editing requests.The system generates diverse interactive environments rather than merely placing static assets.
  • Learning Interface: Generated environments export as Gymnasium-style interfaces with reset(), step(), observations, actions, and rewards, while navigation tasks derive automatically from traversable regions, obstacles, goals, and spatial relations.This supports direct deployment and training on verifiable downstream tasks.
  • Self-Evolution and Co-Evolution: SimCoder self-evolves by writing reusable tools and skills, reusing them across iterations, and refining scenes with verifier feedback such as solvability checks.Embodied-agent performance signals, including success, failure modes, and exploration coverage, steer generation toward the learner’s current ability frontier.
  • Results: Three case studies show reliable physically valid and prompt-aligned generation, transfer improvements to unseen navigation benchmarks, and diversity-driven generalization.The introduction attributes measurable quality contributions to structured tools, verification, and self-evolution, and identifies environment diversity as directly driving generalization.

2 SIMWORLD STUDIO

SimWorld Studio is an Unreal Engine 5 platform in which SimCoder generates, verifies, and self-evolves executable 3D environments, then exports them as Gymnasium-compatible tasks. Its co-evolution loop uses embodied-agent performance to adapt environment difficulty into an increasingly challenging curriculum.

  • Automatic environment generation: SimWorld Studio combines SimCoder, reusable tool and skill libraries, and rule- and VLM-based verifiers to construct and revise scenes.SimCoder accesses Python tools and reusable procedures through an MCP bridge, while verifiers provide feedback for scene construction and revision.
  • Self-evolution: Verifier feedback drives SimCoder to continue building, revise scenes, or add reusable tools and skills for future generations.When generated scenes pass verification, SimCoder derives tasks and exports them as Gymnasium environments for embodied-agent interaction.
  • Gymnasium compilation: Generated environments expose standard reset() and step(action) interfaces returning RGB-D observations, agent pose, and reward, supporting unmodified RL or training-free LLM policies.Tasks include point navigation and object navigation, with solvability guaranteed through scene-structure queries such as NavMesh inspection.
  • Co-evolution: Co-evolution alternates embodied-agent training on generated environments with SimCoder updates based on agent performance, turning generation into an adaptive curriculum.SimCoder adjusts difficulty, lowers it when agents stall, and oversamples structural features they have not mastered without modifying the underlying LLM weights.
  • Co-evolution: The resulting adaptive curriculum outperforms fixed-environment training in navigation tasks.Section 3.3 uses outcome-level agent feedback to adapt SimCoder’s difficulty schedule while the agent improves through incremental rule accumulation.

3 Experiments and Analysis

The experiments evaluate SimWorld Studio through three case studies spanning environment generation, embodied navigation learning, and co-evolution. Results show that SimCoder’s tools and self-evolution improve generation reliability, generated environments transfer navigation gains, and adaptive curricula drive continued improvement.

  • Experimental scope: Three case studies assess scene generation quality, embodied navigation in generated environments, and co-evolution between SimCoder and the embodied agent.The studies increase in scope from evaluating components separately to closing the loop between environment generation and agent learning.
  • Environment generation: Near-perfect physical validity holds across settings, while Opus 4.6 leads semantic quality and image guidance improves Qwen3.5-27B from 0.59 to 0.67.Opus 4.6 achieves semantic scores of 0.77, 0.79, and 0.75 in S1, S2, and S3, respectively; Opus and Sonnet collision-free rates remain ≥0.98.
  • Environment generation: Adding customized MCP tools raises held-out scene-generation quality from 0.16 to 0.45, a +0.29 improvement over the vanilla coding agent.The tools provide the structured action space needed for reliable asset interaction; the ablation also evaluates the verification loop and self-evolution.
  • Embodied navigation: +14.62% SR on ObjectNav is achieved by Qwen3.5-9B over the no-training baseline in held-out generated environments, while transfer to SimWorld-MMNav reaches +12.0pp on 2B.Test success also rises with environment diversity, including a +5.5pp gain on Qwen3.5-27B when training uses more distinct generated environments under a fixed budget.
  • Co-evolution: 18-point gain over fixed-environment learning is achieved by closed-loop co-evolution, which ultimately reaches a 90% test Success rate on SimWorld-MMNav.Adaptive curricula introduce progressively harder environments; success dips after difficulty increases and recovers as the agent transfers distilled strategies.

4 Related Work

Prior embodied simulators trade off realism, scalability, and extensibility: hand-built platforms use costly fixed scene catalogs, while procedural generators remain constrained by hand-designed structures. Agent–environment co-evolution methods adapt difficulty or task parameters from agent feedback but generally do not construct new scenes.

  • Embodied simulation platforms: Hand-built simulators support diverse embodied tasks but depend on manually authored scene catalogs that are fixed and expensive to extend.These platforms cover navigation, manipulation, driving, urban robotics, and language-grounded games.
  • Embodied simulation platforms: Procedural generators increase environment count but remain constrained by hand-designed structures.
  • Agent–environment co-evolution: Prior co-evolution methods keep difficulty near the agent’s frontier, while LLM-based extensions revise rewards, configurations, or task programs from agent feedback.These methods include unsupervised environment design and systems such as EnvGen and Eureka.
  • Agent–environment co-evolution: Existing LLM-based co-evolution methods generally tune parameters of pre-built simulators rather than construct scenes.

5 Conclusion

SIMWORLD STUDIO addresses static scene-generation bottlenecks by synthesizing scalable, interactive 3D environments for embodied learning. Its self-evolving coding agent translates prompts into Gymnasium-compatible worlds and adapts difficulty according to embodied-agent performance.

  • Platform contribution: SIMWORLD STUDIO synthesizes scalable, interactive 3D environments to overcome the bottleneck of static scene generation.The platform targets embodied learning environments rather than static scenes.
  • Platform contribution: A self-evolving coding agent automatically translates prompts into Gymnasium-compatible worlds.This connects language-driven generation with a standard environment interface for embodied learning.
  • Co-evolution: Environment difficulty adapts based on embodied-agent performance through closed-loop co-evolution.The adaptation is intended to prevent training saturation and support continued learning.
  • Results: The reported results show that closed-loop co-evolution prevents training saturation and boosts zero-shot generalization.These outcomes summarize the platform’s demonstrated learning benefits.

A Limitation … C.1 MCP Tool Reference

The paper identifies limitations in SimWorld Studio’s dependence on coding-agent spatial reasoning and discusses broader impacts, including workflow changes and potential misuse. It also documents 14 MCP tools organized across four functional groups.

  • A Limitation: SimWorld Studio remains bounded by the underlying coding agent’s capability, especially for complex 3D environment generation.Required abilities include spatial reasoning about object placement, geometric constraints, physical plausibility, navigability, and long-range layout consistency.
  • A Limitation: Failures may still arise when environments require fine-grained spatial planning or long-range layout consistency.The current system can produce useful scenes but remains vulnerable to demanding spatial-planning tasks.
  • B Broader Impact: SimWorld Studio can reduce repetitive engineering effort by letting coding agents directly edit, validate, and reuse scene-building skills.This is particularly relevant to complex engines such as Unreal Engine, where coding-agent support remains limited.
  • B Broader Impact: Automated scene generation may shift human roles in game development, simulation design, and digital-content production toward supervision, specification, and quality control.The paper frames such systems as assistive tools intended to augment human creativity and engineering productivity.
  • B Broader Impact: Potential misuse includes restricted or unsafe environment generation, surveillance-like or tactical planning, and unauthorized recombination of licensed assets.The release is intended for research use and excludes trained harmful policies, scraped personal data, and human-subject datasets.
  • C.1 MCP Tool Reference: 14 MCP tools are exposed by the SimWorld Studio server across actor management, environment and asset management, scene evaluation, and a Python escape hatch.The Python interface supports operations not covered by the predefined API.

C.2 SIMCODER Generation Pipeline … E Case Study 1

SIMCODER generates embodied-learning scenes through tool-driven planning, iterative construction, verification, and self-authored skills, while a container-maze case demonstrates feedback-guided complexity growth, collision repair, and skill reuse.

  • C.2 SIMCODER Generation Pipeline: SIMCODER acquires current assets, retrieves applicable skills, initializes Unreal scenes, plans spatial layouts, incrementally spawns actors, and verifies collisions, support, and semantic alignment.Its pipeline uses list_assets, SkillRegistry, setup_environment, layout plans, per-batch checks, and VLM judgments to guide corrections.
  • C.3 Skill Library Structure: Verifier feedback can trigger corrective skill authoring, and built-in or learned Markdown skills with optional Python utilities are indexed for reuse across future generations.The five built-in skills cover building placement, city layout, street furniture, weather and mood, and screenshot tours.
  • C.4 Detailed Running Case of SIMCODER: The maze case uses a fixed loop that injects prior scene, collision, and VLM feedback, applies fixes, plans edits, spawns actors, evaluates screenshots, and outputs new skills.The full run reports four shown rounds, six skills learned, and a final recorded VLM score of 6.5/10 at Round 3.
  • Evidence of Pipeline Components: Learned scene-editing patterns persist across rounds, while VLM instructions become hard prompt context and collision detections mandate deleting or moving overlapping actors.The run increases geometric complexity through 0°, 45°, 90°, and later 135° yaw placements.
  • C.5 SIMWORLD STUDIO’s Graphic User Interface: SIMWORLD STUDIO integrates user–agent interaction, Unreal rendering, asset and backend services, Gym APIs, embodied-agent monitoring, skill management, tools, and direct interactive control.The interface supports browsing, creating, inspecting, and reusing skills alongside static and dynamically learned tools.
  • C.6 SIMWORLD STUDIO Running Configuration: The released system relies on specified software, model APIs, assets, and minimal requirements; users must comply with underlying engine, provider, and asset-library licenses.The project states that it neither scrapes personal data nor releases human-subject data.
  • D Additional Related Work: Related work frames SIMWORLD STUDIO at the intersection of generative 3D worlds, tool-augmented coding agents, self-evolving agents, and agent–environment co-evolution.These research lines emphasize explicit scene generation, executable tool use, feedback-driven improvement, and curricula near an agent’s capability frontier.

E.1 Full Results by Difficulty Level

Table 7 presents scene-generation quality across easy, medium, and hard difficulty levels for four LLM backbones and three generation settings, using multiple metrics and difficulty averages.

  • Difficulty-level comparison: Table 7 breaks down per-metric scene-generation performance by easy, medium, and hard difficulty levels across four LLM backbones and three generation settings.Metrics are reported on [0, 1] scales, with higher values indicating better performance.
  • Difficulty-level comparison: The evaluation covers quantity, physical validity, semantic, and aesthetic quality, plus the mean across difficulties.Bold values denote the best result per column within each setting.

E.2 Detailed Metric Specifications

The section specifies the complete metric inventory and details how VLM-based and rule-based metrics are computed. VLM scores use anchored 0–10 judgments normalized to [0, 1], while rule-based checks evaluate collisions, gravity, and spatial bounds.

  • Metric inventory: Table 8 provides the complete metric inventory across all three evaluation settings, distinguishing rule-based metrics from VLM-as-Judge metrics.Bold formatting identifies metrics novel to this work.
  • VLM scoring rubric: VLM metrics use an anchored 0–10 integer rubric and normalize scores to [0, 1] by dividing by 10 for aggregation.The rubric defines 10 as perfect realization, 7 as mostly correct with minor deviations, 5 as a partial match, 3 as vaguely related, and 0 as unrelated; the VLM receives the prompt, six multi-angle screenshots, and scene graph.
  • Rule-based metric details: Rule-based metrics test AABB collisions, gravity alignment, and whether actors remain within the 190m × 190m ground plane.Collision tests cover non-environment actor pairs with both extents >100 units; gravity uses a ±200-unit ground-plane tolerance; bounds require |x|, |y| ≤9500 units.

E.3 Ablation Results · F Case Study 2: Experimental Details · F.1 Compute Setup for Case Study 2

The ablation study evaluates verification and self-evolving skill accumulation under a minimal prompt, while Case Study 2 uses parallel SIMWORLD STUDIO instances on an 8×H100 server for embodied rollouts.

  • E.3 Ablation Results: Table 9 ablates the verification loop and self-evolving skill accumulation across four models.All methods use the same minimal system prompt without domain-specific hints.
  • E.3 Ablation Results: The ablation metrics are CNT, DIV, COL, PF, SRF, and LAES, with all values in [0, 1].Tools denotes total MCP tool calls across all rounds.
  • E.3 Ablation Results: Self-evolving runs process tasks sequentially from easy to mid to hard, accumulating skills between tasks.
  • E.3 Ablation Results: Verification uses up to 3 harness-driven fix rounds per task.
  • F.1 Compute Setup for Case Study 2: Case Study 2 runs the embodied-agent rollout pipeline on an 8×H100 server hosting eight SIMWORLD STUDIO instances in parallel.Each instance executes a subset of navigation episodes and synchronizes with agent-inference workers for observations, actions, rewards, and trajectory logging.
  • F.1 Compute Setup for Case Study 2: Processing the 1.2K-episode training set requires approximately 8 hours per full pass.

F.2 Experiment Design · F.3 Metric Definitions · F.4 Hierarchical Memory Design

The experiments generate Gymnasium navigation environments and episodes from diverse UE5 scenes, evaluate agents with complementary success, efficiency, progress, and trajectory-fidelity metrics, and train them through hierarchical episodic memory. The memory hierarchy supports within-episode correction, retrieval of relevant trajectories, and cross-environment navigation principles.

  • F.2 Experiment Design: Training environments span five urban archetypes on fixed 190m × 190m planes and are automatically exported as Gymnasium environments.The archetypes are downtown intersections, residential neighborhoods, industrial districts, commercial avenues, and mixed-use blocks.
  • F.2 Experiment Design: Episodes sample walkable starts and goals, compute geodesic shortest paths with NavMesh A*, and retain reachable paths of 3m–20m with visible ObjectNav targets.PointNav goals are 2D coordinates, whereas ObjectNav goals are semantic categories from scene objects.
  • F.2 Experiment Design: The study uses 1,200 training episodes and 329 held-out episodes, with RGB, goal-bearing, geodesic-distance, and step-count observations and four discrete movement actions.Episodes terminate on stop or after 40 steps, and evaluation also includes the unseen three-tier SimWorld-MMNav benchmark to test transfer.
  • F.4 Hierarchical Memory Design: After each training episode, the system flushes step records, summarizes the trajectory into L2 memory, and periodically distills recurring patterns into L3 principles.L3 distillation occurs every 10 episodes and replaces outdated principles.
  • F.3 Metric Definitions: The evaluation reports four complementary metrics covering task success, path efficiency, partial progress, and trajectory fidelity.Success is the primary binary task-completion metric; SPL penalizes inefficient successful paths, the continuous relaxation rewards partial progress, and nDTW measures alignment with the reference path.
  • F.4 Hierarchical Memory Design: The hierarchical memory stores experience at step, trajectory, and task granularities so strategies can be retrieved at the appropriate inference scope.Step-level records support self-correction, trajectory-level records summarize strategies and outcomes indexed by environment features, and task-level records distill general navigation principles.
  • F.4 Hierarchical Memory Design: Before each inference episode, the agent retrieves the top-k relevant trajectory records and current task-level knowledge into a structured context block before acting.Trajectory retrieval is based on environment similarity, giving the agent a head start grounded in prior experience.

F.5 Observation Modality Ablation … G.2 Difficulty Curriculum Parameterization

The ablations show that RGB-D observations provide the strongest navigation performance, while SimCoder’s curriculum varies path length, heading offset, and dynamic obstacle density and advances by rolling-success mastery gates. Case Study 3 evolves navigation prompts through prioritized, failure-targeted rules that are pruned and capped to control prompt growth.

  • F.5 Observation Modality Ablation: RGB-D consistently outperforms depth-only and text-only inputs across model scales, combining complementary geometric and semantic cues for navigation.Text-only bearing-and-distance inputs remain competitive for larger models but degrade sharply at smaller scales.
  • G Case Study 3: Experimental Details: Case Study 3 optimizes navigation with a structured GEPA variant that preserves an ordered list of prioritized decision rules instead of rewriting the entire system prompt.Rules are injected verbatim in fixed priority order, reducing the risk of overwriting effective strategies.
  • G.1 Structured Prompt Evolution: More-specific reactive rules take precedence over general rules, with later rules firing only when no earlier rule applies.Examples address forward movement, loop escape, and stopping when the goal is within 1.5 m.
  • G.1 Structured Prompt Evolution: After each episode, a rule-synthesis LLM converts trajectory failure modes into new non-contradictory rules appended to the active list.Detected failures include directional errors, loops, goal-proximity failures, obstacle avoidance failures, and timeouts; each failure yields one to three targeted rules.
  • G.1 Structured Prompt Evolution: The rule list removes rules inactive for 20 episodes and caps active rules at 30 to prevent prompt bloat.Newly synthesized rules are appended at the end of the list.
  • G.2 Difficulty Curriculum Parameterization: SimCoder parameterizes episode difficulty along geodesic path length, initial heading offset, and dynamic obstacle density across eight levels, L0–L7.The three axes increase monotonically so longer paths demand extended planning, larger offsets require reorientation, and denser obstacles increase avoidance demands.
  • G.2 Difficulty Curriculum Parameterization: 0.80, 0.75, 0.70, 0.65, 0.60, 0.55, 0.50, 0.45 are the mastery thresholds τ0 through τ7 for advancing levels.The rolling success rate is averaged over the most recent 5 epochs, and thresholds decrease as difficulty rises.

G.3 Training and Evaluation Protocol … I.4 Iterative Scene Development

The paper evaluates co-evolving curriculum generation against fixed-difficulty and no-learning baselines on a held-out navigation benchmark, while illustrating text-, image-, and language-guided iterative scene construction in UE5. Qualitative examples show progressively enriched environments, but are excluded from quantitative comparisons.

  • G.3 Training and Evaluation Protocol: Each co-evolution epoch generates 20 episodes, rolls out the embodied agent, updates rules from failures, and advances difficulty when rolling success reaches the threshold.SIMCODER generates episodes at the current difficulty, while policy updates use extracted failure modes.
  • G.3 Training and Evaluation Protocol: The evaluation compares adaptive co-evolution, fixed L3 difficulty, and no learning, with all conditions using Qwen3.5-9B and the same initially empty rule list.The fixed-difficulty condition isolates adaptive curriculum effects, while no learning is the zero-shot LLM baseline.
  • G.3 Training and Evaluation Protocol: Final performance uses Success Rate on 329 held-out SimWorld-MMNav episodes spanning easy, medium, and hard tiers in independently authored unseen UE5 environments.Success Rate is the primary reported metric.
  • H Prompt Examples; I.1 Text-to-Scene; I.2 Image+Text-to-Scene; I.3 Scene Editing: Prompt examples specify hard text-to-scene neighborhoods, image-conditioned urban blocks, and scene edits adding buildings, roads, vegetation, marketplaces, and construction objects.The examples demonstrate language-, image-, and editing-based environment generation.
  • I Qualitative Examples: Qualitative appendix examples are visual demonstrations from available model snapshots and are excluded from the quantitative comparisons in Table 1.This limitation applies to the qualitative examples accompanying the prompt sections.
  • Prompt P1; I.1 Text-to-Scene: The downtown-intersection prompt requires a clean 4-way UE5 scene with non-overlapping roads, sidewalks, buildings, vehicles, and pedestrians, generated in one script call before one screenshot.The prompt leaves aesthetic choices open while fixing geometric and execution constraints.
  • Prompt P1; I.2 Image+Text-to-Scene; I.3 Scene Editing; Prompt P2: The desert-village and residential-street prompts constrain large-scale layout, house counts or building types, scene inspection, removal and retention operations, sidewalks, and outward construction.The associated figures show rendered outputs for text- or image-guided construction and editing across model backbones.
  • I.4 Iterative Scene Development: Iterative scene development enriches a bare intersection over six steps, adding downtown buildings, streetside dressing, pedestrians, vehicles and signals, then dusk lighting.Figure 15 documents the sequence from Iter-1 through Iter-6.
Loading 2605.09423v2…