Source-linked AI summary
BlenderRAG: High-Fidelity 3D Object Generation via Retrieval-Augmented Code Synthesis
Massimo Rondelli, Francesco Pivi, Maurizio Gabbrielli
TL;DR
LLMs often generate Blender code with syntactic and geometric problems. BlenderRAG retrieves semantically similar examples from an expert-curated multimodal dataset to provide context during generation, improving compilation and semantic alignment across four LLMs without fine-tuning. Its retrieval-based, Blender-integrated design supports deployment without specialized hardware.
Problem
LLM-generated Blender code frequently contains syntactic errors, inconsistent proportions, and poor geometric coherence.
Method
BlenderRAG retrieves semantically similar descriptions and executable code from a 500-example multimodal dataset and injects them into LLM prompts for Blender code generation.
Results
40.8% to 70.0% compilation success and 0.409 to 0.774 normalized semantic alignment were achieved across models with BlenderRAG.
Takeaways & Limitations
BlenderRAG improves 3D object generation quality without fine-tuning and enables immediate deployment through a Blender plugin.
Takeaways & Limitations
Fine-tuning approaches require multi-GPU infrastructure, substantial computational budgets, and expertise in training-pipeline configuration, motivating BlenderRAG’s zero-training scope.
Abstract
from arXiv · showhide
Automatic generation of executable Blender code from natural language remains challenging, with state-of-the-art LLMs producing frequent syntactic errors and geometrically inconsistent objects. We present BlenderRAG, a retrieval-augmented generation system that operates on a curated multimodal dataset of 500 expert-validated examples (text, code, image) across 50 object categories. By retrieving semantically similar examples during generation, BlenderRAG improves compilation success rates from 40.8% to 70.0% and semantic normalized alignment from 0.41 to 0.77 (CLIP similarity) across four state-of-the-art LLMs, without requiring fine-tuning or specialized hardware, making it immediately accessible for deployment. The dataset and code will be available at https://github.com/MaxRondelli/BlenderRAG.
1 Introduction
BlenderRAG addresses frequent syntactic errors, inconsistent proportions, and poor geometric coherence in LLM-generated Blender code by retrieving semantically similar expert examples as context.
- LLM-generated Blender code often contains syntactic errors, inconsistent proportions, and poor geometric coherence.
- BlenderRAG provides a complementary deployment-ready approach that avoids fine-tuning and specialized training infrastructure.
- RAG outputs exhibit superior geometric coherence, realistic proportions, and structural detail compared with baseline models in the qualitative comparison.
- BlenderRAG uses semantically similar expert-validated examples to guide diverse LLMs toward higher-quality 3D objects with minimal computational overhead.
2 Methodology
BlenderRAG combines an expert-validated multimodal dataset, semantic retrieval, LLM code generation, and Blender integration to produce executable 3D objects without fine-tuning.
- Dataset: The dataset contains 500 examples across 50 object categories, with 25 indoor and 25 outdoor categories and 10 design variations per category.
- Dataset: Each dataset instance pairs a detailed textual description with executable Blender Python code and a rendered 2D image.
- Dataset: Expert users manually validated and refined initial Claude Opus 4.1 code drafts for geometric accuracy and visual realism under standardized rendering conditions.
- Dataset analysis: Indoor code lengths range from 4,000–15,000 characters, while outdoor objects span 3,000–24,000 characters, reflecting diverse structural complexity.
- Retrieval-Augmented Generation: Semantic retrieval matches embedded queries against Qdrant and injects retrieved descriptions and object code into the LLM prompt as structural context.
- Blender add-on integration: The integrated Blender plugin supports multiple LLM backends, executes generated Python directly, and enables immediate 3D object creation.
- Zero-training deployment: BlenderRAG avoids fine-tuning and runs on CPU with off-the-shelf LLM APIs, enabling deployment on consumer laptops without specialized hardware.
- Blender add-on integration: The pipeline retrieves three similar examples, generates executable code, runs it in the current Blender scene, and displays the resulting rendering.
3 Experimental Evaluation
BlenderRAG was evaluated on 30 out-of-distribution object prompts using compilation success and CLIP-based semantic alignment, improving both metrics across models and producing more plausible geometry.
- Setup: The evaluation used 30 out-of-distribution prompts describing novel objects absent from the dataset.
- Setup: Compilation Success Rate measures the proportion of generated code executing without errors, while Semantic Alignment uses normalized CLIP cosine similarity between prompts and rendered images.
- Results: 40.8% to 70.0% compilation success and 0.409 to 0.774 normalized semantic alignment were achieved across models with BlenderRAG.
- Results: Without retrieval, models achieve 10–56% compilation success; with RAG, success rates reach 80% across the four models.
- Results: RAG outputs show realistic articulation, proper dimensional relationships, and structurally sound elements, unlike baselines with floating components and inconsistent scales.
4 Conclusion
BlenderRAG substantially improves 3D object generation quality through retrieval over expert-curated multimodal examples without fine-tuning, while offering practical deployment through a Blender plugin.
- BlenderRAG substantially improves 3D object generation quality through retrieval-augmented generation over expert-curated multimodal examples without fine-tuning.
- Gemini 3 Flash reaches an 80.0% practical compilation rate, while normalized semantic alignment is strong across all evaluated LLMs.
- The Blender plugin enables immediate deployment for content creation workflows.
- Future work targets multi-object scene composition, active learning for dataset expansion, and text-to-image retrieval for image-based queries.