PaperlayerView exact revision on arXiv

Source-linked AI summary

Recursive Code World Models: Building Complex Worlds through Recursive Scene Programs

Zhiqi Li, Yuxuan Liao, Bo Zhu

arXiv:2609.11499v1cs.CV

TL;DR

Complex executable-world reconstruction requires organizing computation across multiple scales while maintaining relationships among parts. RCWM uses Recursive Scene Programs and a recursively self-calling visual coding solver that establishes, decomposes, and revisits scene structure. It outperforms prior code-based reconstruction methods, while ablations suggest deeper recursive solves can improve fidelity.

  • Problem

    Complex-world reconstruction must resolve structures at multiple scales while maintaining relationships that a single reconstruction trajectory may not capture.

  • Method

    RCWM represents worlds as Recursive Scene Programs and recursively applies the same vision-language construction solver to establish wholes, reconstruct unresolved subworlds, and refine their composition.

  • Results

    RCWM obtains the highest PSNR and lowest LPIPS on every evaluated reference and achieves higher fidelity with deeper recursion on medieval-village.

  • Takeaways & Limitations

    Recursive construction gives fine-scale structures complete perception-and-editing loops while preserving scene-wide geometry and relationships.

  • Takeaways & Limitations

    Single-view input leaves hidden geometry and absolute scale ambiguous, while evaluation uses a limited set of scenes, single runs, and ablations on two scenes.

Abstract

from arXiv · show

Code world models represent worlds as executable programs, but this representation alone does not determine how to construct a complex world. We introduce Recursive Code World Models (RCWM), a framework for reconstructing complex 3D worlds in code from a single reference image. RCWM couples a Recursive Scene Program (RSP) representation with a construction solver that recursively calls itself. An RSP represents the executable world as compositional scene code, while each solver call follows the same complete process: establish the whole, recursively reconstruct unresolved parts, and revisit the whole to refine their composition. This global-local-global recursion gives fine-scale structures their own perception-and-editing loops while preserving scene-wide geometry and relationships. Reference-aligned views propagate a shared camera projection across levels, while parent revisitation addresses boundaries, spatial relations, and shared errors that emerge after local refinement. A vision-language coding agent directly compares reference images with scene renders to guide refinement, recursive descent, and return. Across complex scenes, RCWM outperforms prior code-based image-to-scene reconstruction methods. Ablation studies further support the benefits of recursive construction and suggest that deeper calls can improve finer-scale reconstruction. RCWM provides a recursive construction principle for building complex executable worlds from visual evidence.

1 INTRODUCTION

RCWM argues that complex executable worlds should be constructed recursively, because local detail requires focused reconstruction while preserving scene-wide relationships. Its solver repeatedly establishes a whole, recurses into unresolved subworlds, and revisits the composition after their results return.

  • 1 INTRODUCTION: RCWM addresses the tension between scene-wide coherence and fine-scale detail by giving unresolved subworlds focused reconstruction while preserving their relationships to the whole.Local edits can create boundary and spatial conflicts that become visible only when refined parts are inspected together.
  • 1 INTRODUCTION: Each solver call establishes a whole, recursively reconstructs unresolved parts, and revisits the assembled composition before returning to its parent.The same complete cycle applies within scenes, buildings, facades, and terrain regions rather than using a fixed scene–object pipeline.
  • 1 INTRODUCTION: RCWM combines Recursive Scene Programs, executable compositional subworld code, with a vision-language solver that performs recursive construction and parent-level refinement.The representation includes explicit child references, editable structure, and shared dependencies.
  • 1 INTRODUCTION: The framework evaluates whole-scene and local reconstruction against image-to-scene-program baselines and studies construction order, recursive depth, and parent revisitation through ablations.These evaluations use a parameterized Three.js compiler and the same base model.
  • 1 INTRODUCTION: Reference-aligned crops and camera views provide focused evidence at each level, while inherited projection and parent revisitation support consistency across nested subworlds.Figure 2 illustrates a scene calling solvers for the west village and then the tall house before revisiting the parent composition.

2 RECURSIVE CODE WORLD MODELS

RCWM reconstructs executable 3D worlds as nested Recursive Scene Programs, using a solver that repeatedly establishes a whole, recursively solves unresolved parts, and refines their composition on return.

  • 2.1 EXECUTABLE RECURSIVE SCENE PROGRAMS: RCWM represents an executable world as nested subworld programs with editable component code, parameters, references, placement rules, and shared dependencies.The root program forms the executable world, while child programs represent objects, assemblies, or continuous surface regions.
  • 2.2 RECURSIVE WORLD CONSTRUCTION: Each recursive solver call establishes the current whole, recursively constructs selected unresolved children, then composes and refines the assembled result before returning.The same complete cycle applies at every depth rather than using a fixed scene-to-object pipeline followed by cleanup.
  • 2.2.1 CONSTRUCTION STATE AND INITIALIZATION: Root initialization establishes world-coordinate and unit conventions and calibrates the reference camera; later camera revisions refresh affected child states before descent continues.Child calls derive their cameras from the root projection, while the root controls camera revisions.
  • 2.2.2 ESTABLISH THE WHOLE: The solver uses rendered scene-reference comparisons to revise layout, shared structures, component interfaces, and local code throughout construction.A child is rendered within inherited surrounding geometry and placement rules, and repeated observation-edit-render passes guide refinement.
  • 2.2.3 PREPARE AND RECURSIVELY CONSTRUCT THE PARTS: Reference-aligned crops preserve coordinates, magnification, and inherited camera pose, while overlapping windows inspect details, spatial relationships, and contact boundaries.Each child receives focused reference evidence and a corresponding camera view of the scene region.

3 RELATED WORK

Prior code-based world reconstruction exposes executable structure, but image-to-code systems still struggle to preserve fine details and spatial relationships across complex worlds. RCWM frames recursion as a general construction principle rather than a domain-specific generation procedure.

  • Code-based representations make geometry, materials, and spatial relationships explicit, enabling editing, reuse, and downstream simulation.
  • Image-to-code reconstruction must reproduce visible shapes, arrangements, occlusions, and details under a consistent camera despite ambiguous 3D geometry.
  • RCWM applies recursive inference to an evolving executable world, where child subprograms alter the scene that parents later inspect.
  • Unlike room-specific recursive systems, RCWM uses the same complete visual solver across objects, assemblies, buildings, towns, and terrain without a prescribed semantic hierarchy.

4 EXPERIMENTS

Experiments compare RCWM with code-based reconstruction baselines and evaluate recursive construction through controlled workflow ablations. RCWM leads reference-view metrics broadly, while deeper recursion improves fidelity on one evaluated scene.

  • 4. EXPERIMENTS: The evaluation covers ten references, combining five whole-scene images with five local crops and matched visual comparisons at full and magnified scales.
  • 4.2 RECONSTRUCTION RESULTS: RCWM achieves the highest PSNR and lowest LPIPS on every reference, plus the highest SSIM on nine of ten references.On valley-village, VIGA reaches SSIM 0.41 versus RCWM’s 0.38.
  • 4.2 RECONSTRUCTION RESULTS: Additional views render reconstructed programs from rotated, elevated, and close-up cameras without re-optimization, exposing encoded 3D geometry beyond the reference projection.
  • 4.3 ABLATION STUDY: The ablation varies construction order, post-composition refinement, and recursive depth while holding the model, representation, tools, resolution, and crop access constant.
  • 4.3 ABLATION STUDY: On medieval-village, free-depth recursion raises whole-frame PSNR from 16.8 to 19.0 and local SSIM from 0.52 to 0.60 versus fixed two-level recursion.The free-depth run expands to three levels with 33 nodes and outperforms the other workflows on both measures.

5 CONCLUSION AND LIMITATIONS

RCWM combines Recursive Scene Programs with recursive whole–parts–whole construction to address local detail and scene-wide consistency. The paper reports promising reconstruction results but identifies substantial ambiguity, computation, and evaluation limitations.

  • 5 CONCLUSION AND LIMITATIONS: RCWM combines executable Recursive Scene Programs with recursive construction that repeatedly establishes wholes, solves parts, and refines their composition.
  • 5 CONCLUSION AND LIMITATIONS: Reference-aligned views support local inspection, while parent revisitation addresses spatial relationships among returned components.
  • 5 CONCLUSION AND LIMITATIONS: Single-view input leaves hidden geometry and absolute scale ambiguous, while visual inspection can miss errors and repeated refinement may require substantial computation.
  • 5 CONCLUSION AND LIMITATIONS: The evaluation uses a limited scene set, single runs on one base model, and ablations on only two scenes.The authors call for larger benchmarks, repeated runs, and human evaluation of spatial relationships.
  • 5 CONCLUSION AND LIMITATIONS: Additional views and recorded call trees show encoded 3D structure and repeated solver execution across multiple recursive levels.

A REPRODUCIBILITY DETAILS

The appendix documents the solver instruction and execution settings used in the reported experiments. Each solver call receives reference and camera context plus the inherited parent state, with recursion capped at five levels.

  • A REPRODUCIBILITY DETAILS: Each solver call receives a reference crop, matched camera specification, task brief, and inherited parent snapshot.
  • A REPRODUCIBILITY DETAILS: When children are requested, the runner starts their solver calls and resumes the parent after their components return.
  • A REPRODUCIBILITY DETAILS: The runner permits five total recursion levels, while the fixed two-level ablation limits recursion to two levels.

A.1 THE SOLVER INSTRUCTION

The solver applies one recursive work order at every node: inspect the whole, repair or descend into unresolved parts, then continue refinement under inherited scene context.

  • A.1 THE SOLVER INSTRUCTION: The same instruction governs every recursive node, with inherited reference, camera, and parent-snapshot materials supplied through the node’s inputs.Runtime responsibilities include resource limits, schema checks, and evidence recording, while the model retains discretion over reconstruction choices.
  • A.1 THE SOLVER INSTRUCTION: Each solver call first renders its viewport beside the matched target, inspects conspicuous residuals, and checks rotated views for flat or missing thickness.The instruction repeats this whole-scene comparison as needed before editing or descent.
  • A.1 THE SOLVER INSTRUCTION: The agent descends when a discrepancy involves unresolved object groups, layout relations, or complex internal structure, while simpler differences are repaired at the current level.A typical level creates two to four children, and most scenes close in two or three levels.

B DESIGN PHILOSOPHY

RCWM treats every subworld as a complete reconstruction problem while preserving a shared visual frame and revisiting composition to correct cross-part inconsistencies.

  • B DESIGN PHILOSOPHY: Each subworld receives its own reference view, editable code, and complete reconstruction loop, whether it is a facade, building, interacting assembly, or terrain region.The loop establishes the subworld, reconstructs details through further calls, and inspects the assembled result.
  • B DESIGN PHILOSOPHY: Inherited camera pose, crop coordinates, and magnification keep local comparisons aligned with surrounding-scene shape, scale, placement, and occlusion.The shared projection lets local inspection retain the child’s relationship to the broader scene.
  • B DESIGN PHILOSOPHY: Single-image reconstruction requires assumptions about hidden surfaces and out-of-frame continuations, which are checked through rotated views and again from the reference view.The checks look for changes in visible silhouettes, occlusions, and shading after completing unobserved structure.
  • B DESIGN PHILOSOPHY: After child edits return, the parent renders the assembled result and revisits placement, contact, occlusion, and shared geometry that local reviews may miss.This targets overlaps and boundary changes caused by components edited against earlier surroundings.
  • B DESIGN PHILOSOPHY: Executable code exposes component procedures, parameters, references, and spatial arrangement, allowing parents to continue editing and composing returned subprograms.The representation preserves procedures and component references for further editing, reuse, and rendering from new viewpoints.
Loading 2609.11499v1…