From Idea to CAD: A Language Model-Driven Multi-Agent System for Collaborative Design

1 Introduction

• CAD/E supports product development with digital representations, while traditional CAD tools typically rely on highly skilled engineers.Digital geometry serves uses including aesthetic discussions, design-quality evaluations, and feasibility checks.Source: §1 · p. 1
• The proposed human-AI team creates CAD models from sketches, images, or text by planning, generating code, and checking requirements.Agents can identify missing information, ask users to fill gaps, and exchange information to improve the design and model quality.Source: §1 · pp. 1–2
• The paper presents a VLM-based MAS that mirrors development phases and introduces visual self-feedback and human validation to address VLM spatial-reasoning limits.The authors describe potential applications from engineers generating models during design sessions to hobbyists creating models for consumer-grade 3D printing.Source: §1 · p. 2

2.1 Design processes, CAD/E & its Automation

• Engineering design processes clarify requirements, develop system designs, check compliance iteratively, and hand over products; CAD digitizes design but often requires skilled operators.Parametric, code-based design such as CadQuery can automate and customize design while producing models usable in downstream tasks.Source: §2.1 · p. 2

2.2 Language Models in Engineering

• Language-model engineering work includes text-to-3D generation and parametric CAD-code approaches, with reported constraints in task scope, flexibility, or iterative improvement.Prior CAD-code studies include a single drawing in one case, restricted prompt guidance in another, and incomplete specifications or weak performance on some component types in another.Source: §2.2 · p. 3

2.3 Agents & Multi Agent Systems

• Multi-agent systems coordinate autonomous agents, and engineering examples include agents checking design-code compliance and communicating violations to users.LLM-based agents have also been used to mimic groups of people and specialized teams.Source: §2.3 · p. 4

2.4 Research Gap

• The authors identify a gap: autonomous LLM-based MAS had not yet been used to create CAD models while explicitly mimicking established development-process steps.The V-model is given as an example of such a process.Source: §2.4 · p. 4

3.1 Architecture Overview

• Three specialized agents divide the process: the RequirementsEngineer clarifies specifications, the CadEngineer creates the model, and the QualityAssuranceEngineer verifies it before user validation.The user may request further changes in an outer validation loop.Source: §3.1 · p. 4

3.2 Requirement Specification

• The RequirementsEngineer iteratively checks sketch and text inputs and asks about geometric features such as shapes, orientations, and dimensions until the specification is sufficient.Users may let the VLM make assumptions, reducing their effort but potentially increasing deviations from the intended result.Source: §3.2 · p. 4

3.3 Model Design

• The CadEngineer turns the specification into a textual plan and CadQuery Python code, checks executability, and can revise code using errors or feedback.The process retrieves documentation and may require human confirmation before executing generated code.Source: §3.3 · pp. 4–5

3.4 Verification

• The QualityAssuranceEngineer renders the model from seven default views and compares them with the specification, returning textual suggestions when it finds deviations.This verification loop is intended to address VLM spatial-reasoning limitations and support specification-to-code translation.Source: §3.4 · p. 5

3.5 Validation

• In the outer validation loop, users confirm the model or provide feedback when dissatisfied, and that feedback is used to regenerate it.The loop addresses cases where the specification is incorrect or the system cannot converge on the desired model.Source: §3.5 · p. 5

4.1 Implementation

• The prototype uses CadQuery for parametric modeling, Python's AST module for pre-execution code checks, and rendered STL views for quality assurance.The implementation uses gpt-4o-2024-11-20 as its default VLM and retrieves CadQuery documentation for model generation.Source: §4.1 · p. 6

4.2 Experiments

• Experiments used hand-drawn sketches and photos, with geometric-feature count serving as an indication of complexity.Inputs included a rectangular block, a block with two holes, a simplified toy car, a plastic cap, and an angle bracket; Figure 3 contrasts 0-shot VLM results with CAD-team iterations.Source: §4.2 · p. 6, Figure 3 · p. 7
• The comparison uses a 0-shot VLM baseline without requirements clarification, verification, or validation, alongside CAD-team designs that add requirements support and further iterations.User feedback was provided during validation steps, including in the toy-car design process.Source: §4.2 · pp. 6–7, Table 2 · p. 7

4.3 Discussion

• The authors report that the multi-agent system produces designs with significantly higher readiness than vanilla VLM CAD generation.They also describe results as variable and note failures on more complex components, spatial-orientation difficulties, and limits of the CAD library.Source: Figure 3 · p. 7, §4.3 · p. 8

• The combined architecture generated designs with significantly higher readiness than vanilla VLM CAD generation.Source: §5 · p. 8
• Code generation may benefit from finer-grained plan-to-code iterations, stepwise visual confirmation, and renderings that include coordinate systems and dimensions.Source: §5 · p. 8
• Expert users may benefit from the ability to edit generated code interactively.Source: §5 · p. 8
• The authors suggest that the individual agents, especially the CadEngineer, may benefit from VLM fine-tuning on a dedicated training corpus.Source: §5 · p. 8

• The requirements-agent prompt asks users to clarify underspecified dimensions and positions before returning a specification addendum.Source: Appendix · p. 10
• The CAD-agent prompt generates CadQuery Python code from textual or image descriptions, while the planning prompt asks for modeling steps including planes, sketches, and extrusions.Source: Appendix · p. 10
• The documentation-retrieval prompt uses code, feedback, and documentation to suggest fixes when applicable.Source: Appendix · p. 10
• The quality-assurance prompt compares seven model views with the specification and requests the two most relevant geometry issues or an empty response if acceptable.Source: Appendix · p. 11
• The example code creates a 10 × 10 × 2 block with two diameter-2 through-holes centered at (2, 8) and (8, 2).Source: Appendix · pp. 10–11