PaperlayerView exact revision on arXiv

Source-linked AI summary

3DCodeVerse: Exploring 3D Worlds as Code

Yipeng Gao, Ziyao Zeng, Jinfa Huang, Yijiang Li, Haolin Xiong, Wang Qin, Xinlei Yu, Youheng Yao, Jie Zhang, Kun-Yu Lin, Rong Liu, Yinyuan Zhao, Yajie Zhao, Meiqi Guo, Jiebo Luo, Lei Shu, Yunhao Ge, Xinyu Hu, Laurent Itti

arXiv:2610.22341v1cs.CVcs.AI

TL;DR

3D coding resources remain fragmented across tasks and environments, despite the value of executable, editable 3D content. 3DCodeVerse combines a broad executable-code corpus with an automated generation and refinement harness; experiments report a 90% human-evaluation win rate against Claude Design and improved execution after fine-tuning.

  • Problem

    Shared infrastructure for developing and evaluating 3D coding systems across environments remains limited, and existing datasets and harnesses are fragmented across tasks and capabilities.

    Source: §1 · p. 2

  • Method

    3DCodeVerse combines a unified corpus of executable 3D programs with a DAG-based harness that uses execution, spatial checks, and visual judgments to guide revisions.

    Source: §1 · p. 2, §4 · p. 5

  • Results

    3DCodeVerse Harness with Claude Opus 5.5 is preferred in 90% of human comparisons against Claude Design, while fine-tuning raises Qwen3.5-9B execution on 3DCodeBench to 84.9%.

    Source: §5.1 · p. 6, §5.2 · p. 8

  • Takeaways & Limitations

    The ecosystem connects reusable 3D coding infrastructure with training data, models, and reproducible recipes intended to support further research on 3D generation through code.

    Source: §7 · p. 9

Abstract

from arXiv · show

We introduce 3DCodeVerse, a research ecosystem that enables coding agents to create editable and verifiable 3D content at scale. 3DCodeVerse comprises large-scale, high-quality datasets of executable 3D code, a coding harness for spatial reasoning and automated verification, and practical guidance for 3D coding. First, we construct 3DCodeVerse-2M, a large-scale curated dataset of standalone 3D executable code. We build this corpus by collecting open-source data, converting specialized representations, and generating new programs with coding agents. Each program is packaged with the source files and associated assets required for execution in its declared environment, along with available descriptions and verification records. Second, we develop the 3DCodeVerse Harness, a multi-agent directed acyclic graph framework for generating and refining 3D content. It uses execution checks, spatial measurements, and visual feedback to automatically evaluate and revise generated programs without human feedback during generation. Third, we conduct systematic empirical studies of 3D coding, establishing baselines with fine-tuned open models and deriving practical guidance through component-level analyses of our harness. Extensive experiments demonstrate that fine-tuning on our dataset significantly improves the 3D coding capabilities of open-source models. Notably, our 3DCodeVerse harness achieves a 90% win rate against Claude Design (3D Object) in pairwise human evaluations, when both are utilized by Claude Opus 5.5.

1 INTRODUCTION

3D coding creates editable, executable 3D content, but resources for developing and evaluating systems across environments remain fragmented. 3DCodeVerse combines a broad executable-code corpus with an automated generation and refinement harness, and reports gains from both fine-tuning and harness use.

1 INTRODUCTION

  • Existing 3D coding resources are fragmented across tasks and environments, with datasets focused mainly on static or articulated objects and harnesses differing in visual assessment and automated corpus construction.The cited resources cover CAD construction, Blender scripting, visual refinement, and articulated asset generation.Source: §1 · p. 2
  • 3DCodeVerse-2M unifies executable programs for static objects, articulated assets, scenes, and graphics, converting heterogeneous representations into reusable code and removing specialized or unavailable library dependencies.Programs are paired with assets, descriptions, provenance, and available verification signals.Source: §1 · p. 2
  • The harness decomposes complex projects into a DAG of coding and assembly tasks, then uses execution, rendering, independent judging, and spatial checks to guide localized repairs without human intervention.Iterative refinement supports inference-time scaling to projects containing thousands of lines of code.Source: §1 · pp. 2–3
  • Fine-tuning on 3DCodeVerse-2M reaches 84.9% and 92.9% execution rates for 9B and 27B models on 3DCodeBench, while the harness gains 15.5–29.1 points over one-shot generation.The harness also serves as a data-generation engine, recording successful programs, failures, repairs, and preference comparisons for supervised fine-tuning and preference learning.Source: §1 · p. 3

2 RELATED WORK

Prior work provides supervision, procedural resources, benchmarks, and feedback systems for particular aspects of 3D coding. 3DCodeVerse focuses on scaling executable supervision and providing a reusable verification and data-curation framework across environments.

Datasets and Benchmarks for 3D Coding

  • Construction-sequence datasets and natural-language-linked CAD supervision have expanded into executable CadQuery and Blender programs for reconstruction and language-conditioned generation.Examples include DeepCAD, Fusion 360 Gallery, Text2CAD, BlendNet, CAD-Recode, CAD-Coder, and MeshCoder.Source: §2 · p. 3
  • Procedural resources broaden code-based content to assets, scenes, and shader programs, while existing benchmarks assess code generation, editing, and physics capabilities.The passage names Infinigen and its extensions, Shaders21k, and benchmarks including 3DCodeBench.Source: §2 · p. 3

Harness Engineering for 3D Coding

  • Earlier harnesses explore planning, rendered-visual refinement, structured asset validation, graph-based quality control, and agent observation interfaces, whereas 3DCodeVerse combines these functions with corpus curation.Its framework coordinates execution, rendering, spatial measurements, and structural checks across objects, scenes, and visual effects.Source: §2 · p. 3

3 3DCODEVERSE-2M: A LARGE-SCALE 3D CODING DATASET

3DCodeVerse-2M treats 3D coding as producing standalone programs that construct 3D content in declared environments, with quality assessed beyond mere execution. Its corpus combines collected, converted, and harness-generated programs, retaining associated files and available checks.

Task Definition of 3D Coding

  • 3D coding is defined as writing a standalone program that constructs 3D content for a condition comprising a text request and optional reference images.Programs bundle the source files and assets needed to run in a declared environment such as Blender or CadQuery.Source: §3 · p. 3
  • Successful 3D coding requires both execution and high-quality content, assessed against a reference when available or otherwise through programmatic measurements and rendered output.Execution yields an artifact and feedback; failure is represented by no artifact.Source: §3 · pp. 3–4

Data Sources and Composition

  • Coding agents use an execution-and-rendering compile–debug loop, revising programs based on feedback and renders; the harness supplies these tools and final programs share the same evaluation protocol.The corpus draws on public datasets and platforms, converted CAD and articulated-asset representations, scene reconstructions, agent-generated subsets, and newly synthesized programs.Source: §3 · p. 4

Agentic Data Curation and Toolkits

  • Agentic curation converts programs that depend on specialized APIs into standalone programs by translating APIs, inlining routines, and bundling required sources and assets.DeepCAD sequences are instead converted with a deterministic script.Source: §3 · p. 4
  • Inspection tools check part inventories, bounding boxes, contact, collisions, joint poses, support, renders, and temporal defects, returning execution errors or failed checks for targeted repair.Conversion fidelity is measured with voxel IoU for DeepCAD and Chamfer distance for Articraft.Source: §3 · p. 4

Data Sources and Composition

  • Each record stores provenance, license, and available verification results alongside code, environment, conditions, and rendered output; meshes and videos are linked when available.Coverage varies by source: collected and converted programs receive execution and rendering checks where runtimes exist, while harness-generated programs also carry gate and judge results.Source: §3 · p. 5

4 3DCODEVERSE HARNESS: A DATA ENGINE FOR 3D CODING

3DCodeVerse Harness decomposes projects into a DAG of coding and assembly tasks, then iteratively evaluates and revises programs using execution, measured, and visual feedback. Its development loop alternates between refining the judge and the generation workflow.

Multi-Agent Directed Acyclic Graph (DAG) Framework

  • A planner turns the condition and target environment into a component plan, which becomes a prerequisite-linked DAG with independently owned coding tasks and assembly tasks.Assembly integrates the completed code into an initial program for evaluation and refinement.Source: §4 · p. 5

Automated Feedback and Iterative Refinement

  • Each round collects execution logs, component-level measurement violations with numerical correction hints, and VLM judgments of multi-view renders against the condition and acceptance criteria.Execution failure produces no artifact and leaves logs as the only feedback.Source: §4 · pp. 5–6
  • A score combines weighted criteria and defect penalties, is capped by measured violations and unmet requirements, and guides prioritized revision tasks for a fix agent.The fix agent applies the tasks to produce the next program.Source: §4 · p. 6

Adversarial Loop Engineering for Harness Optimization

  • During development, judge refinement uses a fixed artifact bank to reduce judgment variability and measurement contradictions before the revised judge guides workflow refinement.Candidate workflows are compared on matched prompts, balancing judged output quality against observable workflow failures.Source: §4 · p. 6
  • The harness objective includes a penalty for failures such as invalid plans, retried tool calls, and failed builds, with weights expressing trade-offs.The judged score is obtained by executing and judging the selected program.Source: §4 · p. 6

5 EXPERIMENTS

Across the evaluated tasks, 3DCodeVerse Harness generally outperforms native harnesses, while fine-tuning on 3DCodeVerse-2M improves open models’ execution rates. Qualitative examples show structural and appearance defects corrected through automated feedback and refinement.

5.1 RESULTS OF 3DCODEVERSE HARNESS

  • Evaluation compares models with native harnesses and Claude Design on 100 text-to-3D object prompts, and native harnesses on articulated objects, 50 scene prompts, and 50 graphics prompts.The reported metrics include human preference, VLM preference, Judge Score, and four-view SigLIP-2 text–image similarity.Source: §5.1 · p. 6
  • 3DCodeVerse Harness wins most comparisons across backbones and tasks; with Claude Opus 5.5, it is preferred over Claude Design in 90% of human comparisons and scores 88 versus 37.On 3D objects, human preference ranges from 69% to 89%, and Judge Score improves by 11–25 points; human and rubric-score agreement suggests gains are not specific to the refinement rubric.Source: §5.1 · p. 6, Figure 4 · p. 7

5.3 QUALITATIVE ANALYSIS

  • For matched prompts and backbones, harness outputs show more complete structures and finer details, while successive refinement rounds correct structural and appearance defects.Examples include attaching stool legs to a seat and reducing block artifacts in a neon shader, without additional human instructions.Source: Figure 5 · p. 8, §5.3 · p. 8

5.2 FINE-TUNING OPEN MODELS ON 3DCODEVERSE-2M

  • Qwen3.8-27B execution on 3DCodeBench rises from 51.9% to 92.9% (+41.0 points) after fine-tuning on 3DCodeVerse-2M.Qwen3.5-9B rises from no successful executions to 84.9% on 3DCodeBench; Table 2 reports execution rates before and after fine-tuning.Source: Table 2 · p. 7, §5.2 · p. 8

6 APPLICATIONS OF 3D CODING

The applications illustrate 3D coding for robotics simulation and video generation. Programs provide editable control over geometry, physical constraints, scene layout, motion, and camera trajectories.

Robotics

  • Robotics programs encode rigid-link geometry, joint axes, and joint limits as editable parameters, supporting checks and controlled variation in dimensions, articulation, and initial poses.The resulting task instances can be executed in simulation to generate robot interactions and observations.Source: §6 · p. 9
  • In a preliminary 12-case evaluation, code renders and UWM VAE reconstructions pass the door-and-handle geometry check in 12/12 outputs, versus 7/36 UWM predictions.The cases span two assets, two contact states, and three control sequences.Source: §6 · p. 9

Video generation

  • For video generation, programs specify scene layout, object articulation and motion, and camera trajectories; rendered sequences condition video models while those models refine visual appearance.The paper examines paired frames for preservation of layout, motion, and camera trajectories.Source: §6 · p. 9

7 CONCLUSION

3DCodeVerse presents an open ecosystem of datasets, a reusable harness, and open model baselines as a shared foundation for 3D coding.

7 CONCLUSION

  • 3DCodeVerse combines datasets, a reusable harness, and open model baselines in an open ecosystem for 3D coding.The paper presents this as a shared foundation for the field.Source: §7 · p. 9
  • The dataset packages programs with descriptions, assets, and available quality records, while the harness uses execution, spatial checks, and visual judgment without human feedback during a run.The harness retains resulting programs and revisions for data curation.Source: §7 · p. 9

APPENDIX

  • The harness modularly configures a planner, judge, and coding agent, and orchestrates stage progression, refinement, and candidate selection.It also supplies deterministic spatial tools, verification gates, multi-view renderers, and rubric aggregation.Source: Figure 7 · p. 15

A THE 3DCODEVERSE HARNESS: DESIGN AND IMPLEMENTATION

The harness coordinates planning, code synthesis, verification, and iterative refinement across multiple 3D tracks, using both deterministic checks and visual judgment to evaluate generated content.

A.1 COMPONENTS AND MODEL ROLES; A.2 PLANNING AND THE JOB GRAPH

  • The harness separates planner, coding agent, and judge roles across isolated runtimes, then turns prompts into metric plans and file-scoped DAG sessions.The plan specifies parts, dimensions, materials, and acceptance checks; disjoint sessions bound context and prevent cross-part regressions.Source: §A.1 · p. 15, §A.2 · pp. 15–16

A.3 SPATIAL TOOLS AND DETERMINISTIC GATES

  • Deterministic gates return structured failures and prioritized numerical fixes, while enforcing connectivity, part attachments, and plan adherence.Corrective vectors are projected into each runtime’s native authoring frame.Source: §A.3 · pp. 16–17

A.4 AUTOMATED JUDGMENT

  • 0.030 mean score standard deviation for Gemini 3.1 Pro across repeated artifact evaluations supports a view-order-robust scoring signal.The judge receives measurements, gate findings, telemetry, and labelled multi-view renders before scoring.Source: §A.4 · pp. 17–18

A.5 REFINEMENT ROUNDS, HISTORY, AND OUTPUT SELECTION

  • Refinement prioritizes gate errors and mandatory checklist items before perceptual suggestions, and output selection ranks judged rounds by score with tie-break rules.The standard profile allows up to four rounds and a 60-minute wall-clock ceiling.Source: §A.5 · pp. 18–19

A.6 A RECORDED RUN

  • 0.410→0.844 after structural repairs and 0.933 after geometric refinement at 11,052 triangles; the harness selects Round 2 for the stool.The complete trajectory takes 4.0 active minutes and costs $0.74.Source: Figure 11 · p. 19, §A.6 · p. 19

A.7 ARTICULATED OBJECTS: GEOMETRY, KINEMATICS, AND JOINT SWEEPS

  • The articulated track checks rest-pose geometry against kinematic frames, then sweeps joint limits and seeded poses for collisions, attachment, and motion direction.Dynamic collisions or unattached links cap the score at 0.60.Source: §A.7 · p. 20

A.8 SCENES AND GRAPHICS

  • Scene generation stages asset and zone assembly before refinement, while graphics generation samples temporal frames and checks frame statistics and multi-frame montages.Scene gates also check grounding, framing, animation, and draw-call budgets.Source: §A.8 · pp. 20–21, Figure 13 · p. 21

A.9 SKILLS

  • Seventeen skill bundles are routed by track, runtime, session role, planned features, and prior gate findings.The routing table selects up to five bundles per session.Source: §A.9 · p. 21, Table 4 · p. 22

B.1 DATASET HIERARCHY AND COUNTING UNITS

3DCodeVerse-2M organizes executable 3D programs and training exports across several representations, while its conversion and training studies report verified outputs and stable optimization across data scales.

B.1 DATASET HIERARCHY AND COUNTING UNITS

  • The catalogue indexes 269,597 executable programs, while the training pairs and unified export use distinct counting units and representation views.The training split contains 768,596 prompt–code pairs; the unified text export contains 2,028,286 rows.Source: §B.1 · p. 22

B.2 SOURCES AND CONVERSION PIPELINES

  • DeepCAD conversion produces 128,201 unique executable CadQuery programs, with mean voxel IoU of 0.985 against original solids.The compiler converts 98.2% of raw sequences, and deduplication removes cross-split leakage across the CAD lineage.Source: §B.2 · p. 22
  • Shader syntax and uniform repair raises flattened-set headless compilation from 84.6% to 99.0%, while 6,878 of 6,882 converted Articraft mesh assemblies pass verification.The collection also includes execution-verified procedural, community, and agent-authored programs.Source: §B.2 · pp. 22–23

C.1 TRAINING DETAILS

  • 13× higher step throughput for 27B LoRA than ZeRO-3 is reported under standard DDP, at approximately 6.2 K versus 460 tokens/s.Across four H100 GPUs, 9B LoRA sustains approximately 17 K tokens/s.Source: §C.1 · p. 24
  • Training strictly holds out all 212 3DCodeBench factories, their 3,904 re-seeded instances, and per-dialect evaluation splits to prevent train–test leakage.Examples exceeding the context window are excluded rather than truncated.Source: §C.1 · p. 24

C.2 TRAINING CURVES OF THE 27B DATA-SCALE RUNS

  • 0.274 terminal smoothed loss is achieved by Arm A; its full 1.99 M-row run also reaches 0.275 versus 0.316 for Arm B at step 2,490.Arm C uses 293 K unique instruction–program pairs and 647 steps, offering a higher-sample-efficiency regime than Arm B.Source: §C.2 · pp. 24–25, Figure 14 · p. 25
  • Across three representation-scale runs, optimization is reported as smooth and stable, with no logged step exceeding 1.3× the trailing 20-step average.The runs share an optimization recipe and differ in the representation views they consume.Source: §C.2 · p. 24

C.3 EVALUATION PROTOCOL

The evaluation compares harness-based generation with native workflows and external outputs using executable-artifact checks, rubric scores, and preference measures. The harness outperforms several baselines on object prompts, with gains also reported on scene and graphics tracks.

C.3 EVALUATION PROTOCOL

  • The single-request protocol allows autonomous tool use but no human edits or candidate selection, and the paired design compares a harness workflow with the same backbone’s native baseline.Human and VLM preferences use anonymized matched-view renders; VLM preferences require agreement across display orders.Source: §C.3 · pp. 25–26
  • The independent evaluator rebuilds each artifact, scores failed builds zero, and combines blind prompt-based judging with deterministic caps and multi-metric checks.The evaluation uses Judge Scores alongside SigLIP-2 similarity and blind human pairwise preferences.Source: §C.3 · p. 26
  • +25.6 points on the uncapped, scale-free visual score separates 3DCodeVerse Harness from Claude Design (88.1 vs. 62.4; 58 of 60 prompts won).This score excludes proportions-and-scale and applies raw criterion means before defect penalties and caps.Source: §C.3 · p. 26
  • On objects, Harness Judge Score is 86.6 versus 61.3 for Gemini 3.8 Flash and its W/L/T against Claude Design is 59–1; scene and graphics gains also appear across backbones.The paired comparisons are reported in the object and cross-track tables.Source: Table 7 · p. 27, Table 9 · p. 27, §C.3 · p. 27
  • 93.9% of harness input tokens are served from prompt cache, while wrapping Gemini 3.8 Flash raises execution reliability from 93.3% to 100.0%.At an average cost of $3.31 per object, the harnessed Flash system achieves an 86.6 Judge Score.Source: Table 8 · p. 27, §C.3 · p. 28
  • Fixed-artifact judging has standard deviation 0.030 for Gemini 3.1 Pro, while independent generation runs can produce structurally diverse outputs with substantial score variance.The paper uses best-of-N initialization and multi-round refinement to hedge against single-rollout topological variance.Source: §C.3 · p. 28, Table 10 · p. 28

C.4 ADDITIONAL OPEN-MODEL RESULTS

Fine-tuning improves open models’ execution and geometric fidelity across 3D coding tasks, while preference optimization and mixture design further target reliability, shape quality, and cross-dialect performance.

Fine-tuning

  • 0/212 zero-shot Qwen3.5-9B scripts execute on 3DCodeBench; patching the deprecated scene-clearing call raises execution only to 4/212.The passage attributes the remaining failures to wider Blender 5.0 API drift.Source: §C.4 · p. 28
  • 93.2% held-out Blender execution and F@0.05 of 0.328 on 3DCodeBench are reached by fine-tuned Qwen3.5-9B, up from 0.000.The 104 K mixture further lifts GLSL compilation to 73.0% (+16.0 pp) and Three.js execution to 92.5% (+20.0 pp).Source: §C.4 · p. 28, Table 11 · p. 29

Preference optimization

  • 75.0% to 96.2% greedy 3DCodeBench execution and 91.0% to 99.1% four-seed union pass@4 follow two rounds of execution-feedback DPO on the 9B checkpoint.Geometry-feedback DPO raises conditional F@0.05 from 0.387 to 0.403; termination-feedback DPO shortens mean OpenSCAD output from 30,163 to 2,884 tokens.Source: §C.4 · p. 29, Table 13 · p. 30

Mixture balancing

  • 98.0% CadQuery execution under joint multi-dialect training exceeds 73.0% under single-dialect training.Increasing Blender’s mixture share from 15% to 34% raises held-out Blender F@0.05 from 0.705 to 0.816 and greedy OpenSCAD execution from 4% to 56%.Source: §C.4 · p. 30

C.5 ADDITIONAL HARNESS RESULTS

The harness improves multi-round generation across model backbones, while targeted planning and infrastructure changes address articulation completeness, reliability, and compute costs.

Multi-round refinement and backbone scaling

  • Across five configurations, the harness gains +15.5 to +52.3 points over one-shot generation and achieves 100% execution reliability versus 38%–92% for one-shot baselines.These comparisons cover the challenging object, articulated-mechanism, and shader prompts.Source: §C.5 · p. 30, Table 15 · p. 31, Table 16 · p. 31

Backbone scaling

  • 0.758 is GPT-6 Sol’s score with the harness, +15.5 points over Sol one-shot and winning 8/8 prompts against Flash plus the harness.Flash plus the harness scores 0.625, matching the GPT-6 Sol one-shot baseline at 0.603 and winning 5 of 8 head-to-head prompts.Source: §C.5 · p. 30, Table 16 · p. 31

Kinematic joint coupling

  • 10/10 coupled-linkage prompts receive valid URDF mimic declarations with an explicit planner coupling field, versus 3/10 without it.Eight mechanisms reduce to one independent degree of freedom, while two retain secondary degrees of freedom for wheel rotation or mirror tilt.Source: §C.5 · p. 30

Targeted infrastructure ablations

  • 6.7× fewer plan-stage failures result from resetting context with the violated schema rule rather than repairing a degenerate hierarchy in-context (13/277 to 2/276).A reset triggered specifically on dangling link references eliminates that failure mode (0/276 vs. 3/140).Source: §C.5 · p. 31, Table 17 · p. 32
  • 20× fewer redundant retries, 65% fewer uncached prompt tokens per request, and 40% lower median generator cost per round follow verdict–transport decoupling and bounded diagnostics.Across 1,063 sessions and 30,401 turns, halving turn depth cuts cumulative tokens by 68% and net API cost by 55% after prompt-cache discounts.Source: §C.5 · p. 32

D EXECUTABLE ARTICULATION AND COMPARISON WITH UWM

The paper compares simulator-ready articulated code with UWM video rollouts under matched robot actions, finding that code-based simulation preserves assessable door-and-handle geometry more consistently.

Evaluation setup

  • The evaluation covers 12 cases across two initial contact states and three control sequences, with 16 matched robot commands at 20 Hz per rollout.A calibrated door-and-handle contour detector was checked on 24 held-out static renders and recovered hinge angle within 1° on all 20 accepted detections.Source: D EXECUTABLE ARTICULATION AND… · p. 32

Quantitative structural preservation

  • 12/12 code-simulation and 12/12 VAE-reconstructed renders preserve the geometric criterion, compared with 7/36 UWM rollouts (19.4%).The table reports that 29 of 36 UWM rollouts fail to preserve assessable door-and-handle contours.Source: D EXECUTABLE ARTICULATION AND… · pp. 32–33

Interpretation

  • Code simulation combines crisp, view-consistent geometry with simulator-enforced rigid transformations, joint axes, and articulated state transitions.Passing code renders through UWM’s frozen VAE retains the 12/12 pass rate, indicating that UWM rollout contour degradation stems from latent action dynamics rather than VAE reconstruction limits.Source: D EXECUTABLE ARTICULATION AND… · p. 33

Related papers

Loading 2610.22341v1…