LL3M: Large Language 3D Modelers

1. Introduction

• The method treats shape generation as code writing, using reusable high-level structure to generate comprehensive, open-vocabulary assets with detailed geometry and appearance.This contrasts with directly outputting unordered mesh vertices and faces, which do not capture high-level shared structure.Source: §1 · p. 2
• LL3M uses a team of specialized LLM agents to plan, retrieve code, debug scripts, and evaluate rendered assets while generating complete 3D assets from text without specialized datasets or finetuning.The agents coordinate through structured communication and multiple modalities and interfaces.Source: §1 · pp. 2–3
• Shared modular Blender code lets users refine an existing asset through follow-up prompts that target relevant parts while preserving the rest, rather than regenerating from scratch.The same interpretable code exposes tunable parameters for fine-grained adjustments.Source: §1 · p. 2
• BlenderRAG retrieves Blender documentation and concrete examples, supporting more sophisticated code for advanced operations without compromising executable code.Source: §1 · p. 2
• The results gallery spans diverse shapes, intricate arrangements, and rich materials, with each mesh generated through interpretable, editable Blender code.Examples include lantern tassels, windmill features, piano keys, skateboard gradients, and a glossy lamp base.Source: Figure 4 · p. 3

2. Related work

• Generative 3D methods use implicit representations such as NeRFs and Gaussian splatting, while explicit meshes are compatible with graphics pipelines and human workflows [40].The passage also identifies methods that learn latent spaces decoded into implicit representations [29] [32] [42] [65].Source: §2 · p. 3
• Mesh-generation methods predict vertices and faces [43] [49] [6], while editing methods modify input meshes through neural optimization or multiview reconstruction from 2D diffusion priors [4] [5] [12] [13].The paper notes that low-level mesh elements lack high-level shared structure and do not support interpretable parameter changes [37] [51] [57].Source: §2 · pp. 3, 5
• LL3M uses pretrained LLMs and BlenderRAG to generate explicit Blender shapes without additional training or neural optimization, and its code supports artist edits [33].Unlike BlenderMCP, which uses a single LLM, LL3M's multi-agent system is reported to produce higher-quality assets with more detailed geometry.Source: §2 · pp. 4–5
• Procedural representations such as GeoCode, PyTorchGeoNodes [50], and ShapeAssembly [25] offer modular programs but can be category-specific or assume particular geometric primitives.Prior methods also train models, discover decompositions, and reconstruct shapes in ShapeAssembly's language [14] [26] [27] [28].Source: §2 · p. 4
• LLM mesh-token methods use low-level elements [37] [51] [57], while earlier Blender-code systems target narrower tasks or predefined programs [63] [19] [10] [68].L3GO generates simple primitives without appearance; BlenderAlchemy generates materials for existing geometry; 3D-GPT supplies parameters to Infinigen; SceneCraft arranges retrieved assets.Source: §2 · pp. 4–5

3. Method

• LL3M coordinates specialized agents to write Blender code for incrementally constructing, refining, and editing 3D assets.Agents share code context, with roles spanning planning, retrieval, coding, critique, verification, and processing user feedback.Source: §3 · pp. 5–6

3.1. Initial Creation Phase

• The planner decomposes the prompt into subtasks; retrieval supplies Blender documentation, and coding agents execute code using those summaries and prior subtask code.Execution errors trigger retrieval-assisted correction until the code runs without error and produces an asset roughly matching the prompt.Source: §3.1 · p. 6

3.2. Automatic Refinement Phase

• Visual critique identifies mismatches and proposes fixes, while coding agents refine the existing script and verification checks whether proposed changes were applied.The initial asset may contain disconnected geometry or visual attributes that do not match the prompt; refinement edits existing code rather than rewriting from scratch.Source: §3.2 · p. 7

3.3. User Guided Refinement Phase

• Users can request targeted edits to an existing mesh, with verification checking the changes; they can also modify modular Blender code or shader-node features directly.The edit cycle continues until the user ends it, and complex meshes may require multiple prompts.Source: §3.3 · pp. 7–8

4.1. Implementation details

• Experiments used Blender 4.4, and BlenderRAG was built by converting 1,729 official Blender 4.4 documentation HTML files into PDFs and adding them to a RAGFlow knowledge base.This enabled the retrieval agent to access version-specific knowledge for the Blender Python module.Source: §4.1 · p. 8

4.2. Properties of LL3M

• Across diverse categories, LL3M generates varied geometry, textures, and materials, and auto-refinement improves asset plausibility and prompt adherence.The same generic tree prompt produces substantially different trees, while the refinement examples show changes to shape and material.Source: §4.2 · pp. 8–9

4.3. User-guided Refinement

• User-guided refinement supports targeted geometry and material changes, stylistic adaptations, and successive semantic edits while preserving unaffected asset parts.Edits include changing a knife blade’s material, adapting hats to a steampunk prompt, and adding attributes or changing a character’s pose.Source: §4.3 · pp. 9–10

4.4. Evaluation

• Compared with BlenderMCP, LL3M produces higher-quality, more plausible shapes with richer detail and closer alignment to the same text prompts.Examples include correctly oriented, connected cat whiskers and strawberry seeds.Source: §4.4 · p. 10, Figure 14 · p. 10
• With BlenderRAG, complex operations increase 5× versus without it; BlenderRAG reduces the total error rate across the mesh generation process by 26%, while Table 1 reports greater complex-function variety without increased execution errors.Source: §4.4 · p. 11, Table 1 · p. 11
• About 59% of presented user edits use one instruction; others need 3 to 4 follow-ups, which take about 38 seconds each rather than about 10 minutes for full regeneration.The paper reports an average of about 38 seconds per edit instruction.Source: §4.4 · p. 11
• Agent ablations show progressively improved mesh quality, while shared context helps refinement preserve the initial asset rather than produce a substantially different one.Without shared code context, the refined UFO and glasses differ substantially from their initial versions.Source: §4.4 · pp. 11–12, Figure 16 · p. 12, Figure 17 · p. 12
• Automatic refinement may leave flaws such as disconnected or incorrectly posed watering-can handles, which users can remedy with further instructions.The side handle moves closer to the body but remains disconnected after automatic refinement.Source: §4.4 · p. 12

5. Conclusion

• LL3M handles varied shape classes and visual styles, generating geometry alongside textures, colors, and material properties.The framework represents assets with interpretable, modular Python code in Blender.Source: §5 · p. 12
• Sequential natural-language directives let users correct errors, add components, and adjust visual attributes without regenerating the asset from scratch.The iterative loop is described as preserving coherence and user intent.Source: §5 · p. 12
• Interpretable, human-readable code lets users inspect or modify asset logic, connecting automated generation with user authorship.The code is described as well-structured, with modular variable names and informative comments.Source: §5 · pp. 12–13
• VLMs may miss spatial artifacts during automatic visual feedback, but users can supply corrections to the system.Source: Figure 18 · p. 13

Supplementary Material

• The appendices cover agent implementation, Blender version adaptability, quantitative-result details, BlenderRAG, and guidance for additional mesh-editing inputs.Source: Supplementary Material · p. 16

A.1. Multi-Agent Coordination

• The orchestrator broadcasts agent outputs into shared context and selects agent order using configured conditions, including retrieval after code execution errors.It manages turn-taking, agent communication, and session termination.Source: §A.1 · p. 16

A.2.1. Planner Agent

• The planner breaks high-level user instructions into manageable subtasks and delegates them to the appropriate agents.It plans and delegates rather than executing tasks itself.Source: §A.2.1 · p. 16

A.2.2. Retrieval Agent

• The retrieval agent uses BlenderRAG to find relevant Blender API information, typically when code encounters an error or needs API clarification.The coding agent is instructed to consult retrieval for error solutions.Source: §A.2.2 · p. 16, §A.2.1 · p. 16

A.2.3. Coding Agent

• The coding agent edits and executes Blender bpy scripts, and the workflow diagram shows a retrieval-coding loop when code execution produces an error.The coding agent uses Claude 3.7 Sonnet.Source: §A.2.3 · pp. 16–17, Figure 19 · p. 17

A.2.4. Critic Agent

• The critic agent uses multi-angle renders and a vision-language model to identify prompt mismatches or disconnected components.The critic agent uses GPT-4o and the critique scene tool.Source: §A.2.4 · p. 17

A.2.5. Verification Agent

• The verification agent re-renders the scene and uses a vision-language model to assess it against the critic’s saved critique list.It determines whether the updated scene satisfies the critiques.Source: §A.2.5 · p. 17

A.2.6. User Proxy Agent

• The user proxy accepts direct human instructions, passes them to other agents, and executes updated code so users can continue refining from the latest visual output.The interface agent does not use a language model and supports geometric edits such as changing outlines, adding surface details, or adding parts.Source: §A.2.6 · p. 17, Figure 20 · p. 18

B. Future Blender Versions

• BlenderRAG can incorporate documentation for future Blender versions or custom knowledge bases, independent of the code agent’s training-cutoff knowledge.The work uses BlenderRAG for a specific Blender version.Source: B. Future Blender Versions · p. 18

C. Additional Quantitative Analysis Details

• 34 bpy scripts spanning 17 diverse objects were examined to evaluate BlenderRAG, with GPT-4o prompted to count simple and complex operations.The objects included examples such as a Chinese Lantern, Crocodile, and Windmill.Source: C. Additional Quantitative Analysis… · p. 18
• Complex operations include multi-step mesh editing, shader-node construction, geometry-node setups, and procedural geometry manipulation.The passage contrasts these with common API calls and basic transformations categorized as simple operations.Source: C. Additional Quantitative Analysis… · p. 18
• Table 2 covers 30 scripts with and without RAG and counts simple operations, complex operations, and errors using GPT-4o evaluation.The table caption describes the evaluation scope but does not provide cell values.Source: Table 2 · p. 18

• BlenderRAG reduces the execution error rate, but the reduction is relatively mild.Source: D. Further discussion on BlenderRAG · p. 18
• Without BlenderRAG, the coding agent may delete buggy lines; retrieved examples help fix issues while enabling more sophisticated code without increasing error rates.Source: D. Further discussion on BlenderRAG · pp. 18–19
• When the prompt explicitly requests parenting, the agent can create semantically named parent-child relationships that support intuitive control and structural coherence.In Blender, transformations applied to a parent propagate to its children.Source: D. Further discussion on BlenderRAG · p. 19, Figure 21 · p. 19

E. Additional Details for User Guided Refinement

• Complex meshes may require multiple user prompts to reach high-quality results.The authors attribute this in part to LLM hallucination, imprecise prompting, and imperfect VLM spatial awareness.Source: E. Additional Details for User Guided… · p. 19

1. Create a mini cartoon character

• The first character edits move the pupils outward and the hands backward to attach them to the arms.Source: §1 · p. 19

2. Add a blonde wig on the head

• The wig is scaled smaller and moved upward onto the character’s head.Source: §2 · p. 19

3. Add a pair of glasses to the face

• The glasses are rotated and repositioned along the x- and z-axes to align with the character’s eyes.Source: §3 · p. 19

4. Add a sprinkled ice cream to the left hand

• The ice cream is repositioned into the character’s left hand, with larger, denser sprinkles added.Source: §4 · p. 19

5. Make the character sit down and eat the ice cream with both hands

• The final prompts separate overlapping legs, pose both hands to hold the ice cream, and turn its sprinkles toward the camera.The critic evaluates five-angle Eevee renders using an external VLM; a verification agent may require another coding iteration when a critique remains unimplemented.Source: §5 · p. 19, Figure 22 · p. 20, Figure 23 · p. 20