BlenderLLM: Training Large Language Models for Computer-Aided Design with Self-improvement

• CAD design demands specialized skills and training, and tasks such as parameter adjustment and model validation require substantial human effort.These demands make design labor-intensive and time-consuming, increasing project costs and slowing iteration and innovation.Source: §1 · p. 1
• Prior LLM applications to CAD face adoption challenges, including input forms that can be too complex to use.Source: §1 · p. 1
• The framework combines 8k-sample BlendNet, a CAD script generation model, and CADBench, with fine-tuning and iterative self-improvement.BlendNet samples map natural-language instructions to bpy scripts.Source: §1 · p. 2
• BlenderLLM outperforms all baseline models across multiple dimensions on CADBench.The introduction reports empirical evaluations and points to Table 1 for examples of model performance.Source: §1 · p. 2

2.1 Computer-Aided Design (CAD)

• Earlier CAD methods included rule-based systems and machine learning, while CNNs were later used to convert 2D sketches into 3D models.The passage says rule-based systems lack flexibility and machine learning requires extensive labeled data and is limited by training-data scope.Source: §2.1 · p. 2

2.2 Large Language Models for CAD

• LLM-based CAD work includes natural-language-to-command or script generation, sketch completion, and parametric sketch generation from text and images.Examples include CADGPT, BlenderGPT, 3D-PREMISE, CAD-LLM, and CadVLM.Source: §2.2 · pp. 2–3

2.3 Blender

• Blender is used to render CAD scripts, connecting LLM outputs with visual results.Its Python API supports bpy scripts, and the paper cites its lower learning curve and broader user base as advantages over other CAD software.Source: §2.3 · p. 3

3.1 Data Construction

• A three-module pipeline generates instruction–script pairs, renders scripts in Blender, and verifies that the resulting images align with the instructions.The Text, Image, and Verification Modules construct training data for supervised fine-tuning.Source: §3.1 · p. 3

3.1.1 Text Module

• The authors manually created 135 diverse seed instructions, then used iterative Self-Instruct distillation, sampling seeds to generate new instructions and yielding approximately 50k; GPT-4o generated a corresponding script for each instruction.Source: §3.1.1 · p. 3

3.1.2 Image Module

• Blender executes each generated script, producing four images from different angles for each 3D object.The images are stored as a set paired with the instruction.Source: §3.1.2 · p. 3

3.1.3 Verification Module

• GPT-4o checks whether rendered images match instructions; manual cross-validation found 89.7% consistency, and manual verification yielded 2k pairs alongside 6k GPT-4o-validated pairs.Together, these sets form BlendNet.Source: §3.1.3 · pp. 3–4

3.2 Model Optimization

• BlenderLLM uses two optimization phases: supervised fine-tuning followed by self-improvement.The paper describes self-improvement as further optimization using model-generated data.Source: §3.2 · p. 4, §3.2.2 · p. 4

3.2.1 Step I: Supervised Fine-tuning

• Fine-tuning Qwen2.5-Coder-7B-Instruct on the constructed data produces BlenderLLM-base, the starting model M0 for the next optimization step.Source: §3.2.1 · p. 4

3.2.2 Step II: Self-improvement

• Self-improvement alternates model-generated instruction–script pairs, Cascade Filter selection, and fine-tuning until validation loss stops decreasing.The Cascade Filter combines a Qwen2-VL-7B-Instruct coarse filter with GPT-4o as the fine filter; selected data updates the previous model.Source: §3.2.2 · p. 4

4 Benchmarking CAD

• CADBench provides 700 instructions for quantitative CAD-script evaluation, using predefined criteria rather than fixed ground-truth outputs for this open-ended task.Source: §4 · pp. 4–5

4.1 Design Principles

• CADBench is designed for user-centricity, comprehensiveness, granularity, and reliability, with broad category coverage, three evaluation dimensions, and manually annotated grading criteria.Its design also aims for evaluation consistency and alignment with human preferences.Source: §4.1 · p. 5

4.2.1 Part I: CADBench-Sim

• CADBench-Sim contains 500 synthetic samples, with category and instruction-type distributions designed to be uniform and instruction lengths broadly varied.Source: §4.2.1 · p. 5, Figure 3 · p. 5

4.2.2 Part II: CADBench-Wild

• CADBench-Wild contains 200 real-world questions from online forums, representing scenarios described as more challenging than synthetic tasks and out-of-distribution relative to BlenderLLM training.The benchmark uses these questions to assess generalization beyond synthetic environments.Source: §4.2.2 · pp. 5–6

4.3 Criteria

• CADBench criteria span three major and eight minor dimensions; GPT-4o drafts sample-specific criteria that are then manually verified.The criteria are provided to GPT-4o to support detailed assessment without fixed ground truth.Source: §4.3 · p. 6

4.4 Evaluation Protocol

• Scripts are generated, executed in Blender for four-angle screenshots, and scored by GPT-4o; on 200 outputs, human–human κ=0.883 and LLM–human κ=0.791.The kappa values are reported as agreement results for the evaluation framework.Source: §4.4 · p. 6

4.5 Evaluation Metrics

• The final score averages outputs across all criteria, with image or script input omitted for criteria that do not use that input.Source: §4.5 · p. 6

5.1 Training Details

• BlenderLLM-base is obtained by fine-tuning Qwen2.5-Coder-7B-Instruct on BlendNet-Human, with later rounds fixed at 2k samples.The 2k size is used to prevent training-data saturation and overfitting; full-parameter fine-tuning is applied.Source: §5.1 · p. 6

5.3 Main Results

• BlenderLLM achieves state-of-the-art performance across all dimensions on CADBench-Sim and CADBench-Wild, significantly outperforming second-place o1-Preview.Table 2 covers Attr., Spat., Inst., Avg., and Esyntax; higher scores indicate better performance.Source: Table 2 · p. 7, §5.3 · p. 7

5.2 Baselines

• The comparisons use a one-shot context and include proprietary, open-source, and CAD-focused models, including o1-Preview, GPT-4o, BlenderGPT, and Qwen2.5-Coder-7B-Instruct.The full baseline list also includes GPT-4 turbo, Claude3.5-sonnet, Gemini-1.5-pro, DeepSeek-V2.5, Qwen-2.5, LLaMA3.1-8B-Instruct, Mistral-7B-Instruct-V0.3, and CodeLLaMa-7B-Instruct.Source: §5.2 · p. 7

5.3 Main Results

• The syntax error rate is significantly lower than other models’ and barely increases on CADBench-Wild.The authors interpret this as evidence of proficiency in understanding CAD script syntax.Source: §5.3 · p. 7
• Table 3 traces BlenderLLM’s progression from limited instruction following to understanding instructions, spatial reasoning, and successful object modeling.The examples show the progression during Self-improvement.Source: §5.3 · p. 7, Table 3 · p. 8

5.4 Analysis and Discussion

• The analysis attributes advantages in attributes, spatial reasoning, instruction following, and syntax to BlendNet’s varied instructions and Self-improvement training.The authors connect BlendNet to learning attribute relationships and Self-improvement to progressively enhanced spatial reasoning.Source: §5.4 · p. 8

6 Ablation

• Epoch accumulation fine-tunes Qwen2.5-Coder-7B-Instruct on fixed BlendNet-Human data, adding one training epoch at each iteration.Source: §6 · p. 8
• Predefined incremental training begins with BlendNet-Human and adds 2k unused BlendNet-GPT examples in later iterations.Source: §6 · p. 8
• At the same number of training iterations, Self-improvement consistently outperforms epoch accumulation and predefined incremental training on CADBench-Sim and CADBench-Wild.The authors also report stronger instruction-following and spatial reasoning in visualizations.Source: §6 · p. 8, Table 4 · p. 8

7 Conclusion

• The framework combines data construction, self-improvement-based SFT training, and benchmark testing; BlenderLLM demonstrates superior performance across various metrics compared with mainstream models.The conclusion highlights combining Self-improvement with a high-quality dataset as effective.Source: §7 · p. 8

Limitation

• The work focuses on basic CAD modeling and instruction-to-script generation, without material properties, surface treatments, internal complexity, multimodal inputs, direct model generation, or multi-turn dialogue.The authors note that these boundaries may affect advanced-task performance and limit complex interactive conversations.Source: Limitation · p. 9

Ethics Statement

• The study involved no human subjects or personally identifiable information, followed data-privacy and intellectual-property guidelines, declared no conflicts of interest, and licenses its datasets and models under CC-BY 4.0.Source: Ethics Statement · p. 9

D.2 Example for Criteria

• The process filters instruction-render pairs, collects 2,000 accepted pairs per iteration, trains and evaluates successive models, and stops when validation performance degrades.The preceding model is retained if the score degrades; otherwise, the new model seeds the next iteration.Source: §D.2 · p. 16
• Example criteria assess proportions and textures, spatial support and contact, and whether the generated object accurately follows all specified attributes.Examples include proportionate seat-to-leg dimensions, correctly positioned and supported parts, and armrests fixed to the backrest and seat.Source: §D.2 · p. 16

D.3 Average Number of Criteria across Dimensions

• Figure 8 presents the average number of criteria for each subdimension, but the supplied passage gives no numerical values.The prose identifies the figure's scope without reporting its cell values.Source: §D.3 · p. 17, Figure 8 · p. 17

D.4 Evaluation Metrics

• Sub-dimension scores average criterion scores, dimension scores average across sub-dimensions, and the overall score averages across dimensions.The metric descriptions define the hierarchy from criteria to sub-dimensions, dimensions, and the model's overall score.Source: §D.4 · p. 17
• Syntax error rate measures the share of generated scripts that cannot successfully produce an image.The metric uses the number of samples with syntax errors and the total number of samples.Source: §D.4 · p. 17
• Standard deviation across the k dimensions is used to assess the consistency of a model's scores.Source: §D.4 · p. 17

E Benchmark Evaluation Process

• Figure 9 provides a detailed description of the benchmark scoring process.Source: E Benchmark Evaluation Process · p. 17

F Details of The Data Generation Pipeline

• The data-generation example uses a bpy script to construct a table scene, then evaluates whether its rendered images satisfy object and placement criteria.The example includes a table, a book, and a mug; the illustrated assessment identifies a missing mug opening and incorrect book placement.Source: F Details of The Data Generation… · p. 18

G.1.1 Objective

• BlendNet-Human review checks image-instruction alignment for positions and spatial relations, and script-instruction alignment for attributes such as color, size, and materials.Source: §G.1.1 · p. 19, §G.1.2 · p. 19

G.1.3 Annotation Workflow

• Two annotators independently pass or fail each entry, a third resolves disagreements, and quality control reviews 30% of the data.Source: §G.1.3 · p. 19

G.1.4 Team and Results

• Over 10k entries were reviewed, yielding 2k entries for BlendNet-Human.Source: §G.1.4 · p. 19

G.2.2 Annotation Guidelines Instruction Filtering

• CADBench criteria curation excludes infeasible or out-of-scope instructions and simplifies instructions with multiple material, surface, or complexity constraints.Criteria validation also requires coverage of dimensions, measurable specificity, and defaults for unspecified properties.Source: §G.2.2 · p. 19, Criteria Validation · p. 19

G.2.3 Annotation Workflow

• For instruction-criteria pairs, two annotators review each entry, a third resolves disagreements, and quality control reviews 30% of the data.Source: §G.2.3 · p. 19

G.2.4 Team and Results

• From 2.5k initial instruction-criteria pairs, annotators curated 500 high-quality pairs for CADBench.Source: §G.2.4 · p. 19

G.3.1 Objective

• Human evaluation scores 200 model responses, using images for most criteria and scripts for color, size, texture, and surface detail.Criteria receive 1 point when satisfied and 0 when not; a quality-control team reviews 30% of the data.Source: §G.3.1 · p. 19, §1 · p. 20, §2 · p. 20

G.3.4 Team and Results

• The human-evaluator consistency measure is kappa 0.883.Source: §G.3.4 · p. 20

H Baseline LLMs

• The listed baselines include general-purpose and code-focused LLMs such as Qwen-2.5, Mistral-7B-Instruct-V0.3, CodeLLaMa-7B-Instruct, and Qwen-2.5-Coder-7B-Instruct.Source: H Baseline LLMs · pp. 20–21

I Performance on Sub-Dimensions

• Figure 10 presents different models' performance on sub-dimensions.Source: I Performance on Sub-Dimensions · p. 21

J Visual Performance of Different Models

• Table 9 presents visual examples of different models' performance.Source: J Visual Performance of Different Models · p. 21

K Visual Performance of Different Training strategy

• Table 10 presents visual examples of different training strategies' performance.Source: K Visual Performance of Different… · p. 21

L Characteristics of Annotators

• Annotators held bachelor's degrees in specified fields and had received full English instruction during their academic education.Source: L Characteristics of Annotators · p. 21