Source-linked AI summary
AgenticCADedit: A Stateful, Tool-Mediated Agentic Approach to Multimodal 3D CAD Editing
Saptarshi Neil Sinha, Mika Silvan Goschke, Paul Julius Kühn, Arjan Kuijper, Michael Weinmann
TL;DR
Existing multimodal CAD editing baselines refine complete programs from the original model, while practical editing involves cumulative geometry and requests combining speech, sketches, and model interaction. AgenticCADedit uses persistent, tool-mediated incremental edits, and improves every reported metric across evaluated LLMs on the neuralCAD-Edit benchmark. The human baseline remains ahead, and long composite requests and sketch-based intent remain only partially solved.
Problem
Prior multimodal CAD editing repeatedly refines complete programs from the original model, despite editing requests combining speech, sketches, and model interaction and operations that build on evolving geometry.
Source: §1 · p. 2Method
AgenticCADedit applies incremental code steps to a persistent CAD session, using geometry inspection, rendered verification, and undo to guide edits.
Source: §1 · p. 3, §3 · p. 5, §3.2.2 · p. 5Results
The approach improves validity, instruction following, model quality, and acceptance for every evaluated LLM on the 192-task neuralCAD-Edit benchmark.
Source: §6.2 · p. 7, §7 · p. 9Takeaways & Limitations
With model, iteration budget, and evaluation protocol held fixed, pairwise comparison isolates interaction-paradigm differences from those of the foundation model.
Source: §7 · p. 9The human baseline remains ahead, and long composite requests and sketch-based intent remain only partially solved.
Source: §7 · p. 9
Abstract
from arXiv · showhide
Computer-aided design is central to industrial manufacturing, and much of a designer's daily work consists of editing existing models from multimodal requests involving speech, sketches, and model interaction. Existing neural CAD approaches focus predominantly on unconditional or text-conditioned generation. The neuralCAD-Edit approach formalizes expert multimodal editing requests, but its iterative baseline refines a complete CAD program across attempts, executing each attempt from the original model in a stateless CAD environment. Every attempt must therefore reconstruct the entire edit from scratch, so partially correct progress is discarded rather than accumulated, and the model can neither inspect the geometry it has just produced nor selectively revert a single faulty operation. We present AgenticCADedit, which turns editing into a sequence of small, verifiable actions on a persistent CAD state instead of a single regenerated program. Rather than emitting one complete program, it applies incremental code steps that each commit to the session, inspects the resulting faces and edges, renders highlighted selections to verify that the intended region was addressed, and reverts individual operations when it was not. Subsequent actions therefore build on the geometry produced by earlier ones. Our approach improves on all metrics for all three evaluated LLMs (open-weight: qwen3.6-27b, gemma4-31b; proprietary: gpt-5.6-luna), with the largest gains for the weakest baseline model, qwen3.6-27b, whose validity rises from 51.0% to 94.8% and acceptance from 1.6% to 12.0%. A token-cost analysis with gpt-5.6-luna further shows $66.7$% fewer output tokens than neuralCAD-Edit, while $94.8$% of input tokens are served from the prompt cache.
1 Introduction
Multimodal CAD edits are cumulative and require recovering intent across speech, sketches, and model interaction, but prior editing repeatedly regenerated complete programs from the original model. AgenticCADedit instead makes incremental edits to persistent state, with tools for inspecting and verifying geometry; it improves every metric for each evaluated model.
1 Introduction
- Existing editing methods are predominantly text-conditioned, and neuralCAD-Edit refines complete programs from the original model without persistent geometric effects between attempts.Designers often communicate through speech, pointing, sketches, and references to prior actions, making intent recovery multimodal and interleaved.Source: §1 · pp. 1–2
- Multimodal editing of existing solids becomes stateful incremental interaction, so successful edits persist instead of requiring complete regeneration from the original model.The requests combine modalities, while later operations act on evolving geometry that may need inspection or revision.Source: §1 · pp. 2–3
- The editing-oriented MCP interface combines incremental code execution, numerical geometry queries, rendered selection checks, and undo operations.The overview shows the model editing a persistent state and exporting the result after declaring completion.Source: Figure 1 · p. 2, §1 · p. 3
- AgenticCADedit improves every metric for all evaluated LLMs while emitting fewer output tokens, with the largest gains for qwen3.6-27b.The comparison keeps the benchmark and evaluation protocol fixed against stateless script refinement.Source: §1 · p. 3
2 Related works
Related work spans CAD representations and generation, agentic CAD modeling, and multimodal editing benchmarks. These lines of work leave open the study of persistent, tool-mediated editing of existing 3D solids under reference-based evaluation.
2.1 Neural CAD generation and representation
- B-Reps expose faces, edges, and vertices for direct geometric editing, unlike sampled or learned shape representations that generally do not produce editable B-Reps.CAD datasets variously preserve final geometry, sketches, or construction history; several sequence resources remain restricted to sketch-and-extrude operations.Source: §2.1 · pp. 3–4
- Generative CAD methods produce B-Reps, construction sequences, or executable scripts, and this work follows the script view to cover operations such as fillets, shells, and Booleans.The passage notes that direct B-Rep generation yields valid solids but no construction history, while sequence methods can inherit narrow operation vocabularies.Source: §2.1 · p. 4
2.2 Agentic CAD modelling
- Prior agentic CAD systems use execution feedback or persistent sessions, but address other tasks or differ in training, limiting isolation of the interface's contribution to editing.This work applies persistent interaction to multimodal edits while holding the model, sampling parameters, and benchmark fixed.Source: §2.2 · p. 4
2.3 Multimodal CAD editing and benchmarks
- Multimodal CAD editing remains limited: mrCAD targets 2D CAD, while neuralCAD-Edit collects speech, pointing, and drawing for 3D solids and supplies reference edits.The paper reuses neuralCAD-Edit's evaluation protocol to compare interaction paradigms.Source: §2.3 · p. 4
3 Methodology
AgenticCADedit gives the LLM incremental control of a persistent CAD session through an MCP interface. A tool-mediated loop executes, inspects, and manages edits until the current model is exported.
3 Methodology
- The agentic editing arm replaces repeated complete-program generation with small, verifiable tool actions on a persistent CAD session.The harness mediates between the LLM and CAD session through a Model Context Protocol interface.Source: §3 · pp. 4–5
3.1 LLM context
- The LLM receives tool schemas and task information, and is instructed to inspect, render, and verify geometry before and after edits.Requests support text and videos of designers speaking, pointing, rotating, and drawing on the model.Source: §3.1 · p. 5
3.2 Agentic CAD editing loop
- Each loop iteration supplies the full conversation history; requested tools run in order, and their results are appended for the LLM's next step.Multiple tools can run in one turn, and each call has a time limit.Source: §3.2 · p. 5
3.2.1 Stateful execution environment
- A persistent namespace retains variables between calls; failed code restores the prior state, while undo reverts successful but unintended edits.The evolving geometry is stored in result.Source: §3.2.1 · p. 5
3.2.2 MCP tools
- MCP tools support code execution, numerical geometry queries, visual rendering, and session management, including visual checks of selected edges.run_step executes CadQuery code in the persistent session; finish signals completion.Source: §3.2.2 · pp. 5–6, Figure 2 · p. 6
3.2.3 LLM interface
- The OpenAI tool-calling harness appends model responses and textual tool results unchanged, while collected renders are delivered in a separate user message.It records token counts, transcripts, and images, including failed runs.Source: §3.2.3 · p. 6
3.2.4 Exit criteria
- The loop ends when the LLM calls finish, returns without a tool call, or exhausts the 100-iteration budget; the current model is then exported for evaluation.Both approaches use the same maximum of 100 iterations per task.Source: §3.2.4 · p. 6
4 Dataset
The evaluation benchmark contains 192 expert multimodal CAD-editing requests spanning different input modalities, difficulty levels, and model categories.
4 Dataset
- The neuralCAD-Edit benchmark provides 192 expert multimodal CAD-editing requests.Source: §4 · p. 6
- Requests use text, interactive speech and model interaction, temporary drawings, or persistent drawings, with videos preserving their temporal alignment.Drawings in draw-static requests remain visible throughout the request or until manually removed.Source: §4 · pp. 6–7
- The benchmark covers three difficulty levels and four model categories combining parametric or non-parametric models with single-body or assembly construction.Source: §4 · p. 7
5 Implementation details
Implementation used two self-hosted open-weight models and one proprietary model accessed through an API.
5 Implementation details
- The evaluated models were qwen3.6-27b, gemma4-31b, and gpt-5.6-luna.Source: §5 · p. 7
- The open-weight models were self-hosted on a cluster with four NVIDIA A100 GPUs, each with 40 GB of memory.Source: §5 · p. 7
- The proprietary gpt-5.6-luna model was accessed through its public API with reasoning effort set to high.Source: §5 · p. 7
6 Evaluation
The evaluation compares stateful editing with neuralCAD-Edit’s stateless refinement across three LLMs, then examines quantitative, qualitative, ablation, and token-cost results. AgenticCADedit improves every metric for every evaluated model, with especially large gains for qwen3.6-27b.
6 Evaluation
- Evaluation follows baseline metrics, including Chamfer distance, voxel IoU, DINOv2 similarity on isometric renders, validity, and VLM ratings for instruction following, model quality, and acceptance.Instruction and quality ratings use a 1–7 scale and are normalized to [0, 1]; acceptance requires both scores to be at least 5.Source: §6 · p. 7
6.1 Comparison with Baseline Approaches
- The primary baseline is neuralCAD-Edit’s iterative refinement; each LLM runs in both harnesses to separate the interaction paradigm from model choice.Source: §6.1 · p. 7
6.2 Quantitative results
- Across all 192 tasks, AgenticCADedit improves every metric for every LLM over neuralCAD-Edit; qwen3.6-27b validity nearly doubles and acceptance increases by more than sevenfold.Valid-output counts rise substantially for open-weight models; on tasks both systems solve validly, mean Chamfer distance is the sole exception to the otherwise similar improvement pattern, and the human baseline remains ahead, especially on acceptance.Source: §6.2 · pp. 7–8
6.3 Qualitative results
- Easy edits preserve dimensions and symmetry across models, but medium merges and hard multi-step wheel edits remain error-prone, especially for open-weight models.The qualitative comparison covers easy, medium, and hard requests across the baseline and all three models.Source: Figure 5 · p. 7, §6.3 · pp. 8–9
6.4 Ablation studies
- With the same LLM, the tabulated comparison isolates the interaction paradigm; Figure 6 shows incremental valid steps producing a better-following fastener assembly than the baseline.For the example, the agentic output scores 5/7 for instruction and quality, versus 2/7 and 3/7 for full-program refinement.Source: Figure 6 · p. 8, §6.4 · p. 9
6.5 Token cost
- For gpt-5.6-luna, output tokens fall 66.7% and API cost falls 58.2%, despite 301.37 million input tokens versus 99.39 million for the baseline.94.8% of the loop’s input tokens are prompt-cached, compared with 0.2% for the baseline.Source: §6.5 · p. 9
7 Conclusion
AgenticCADedit turns multimodal CAD editing into stateful, tool-mediated interaction with an evolving model. It improves every evaluated LLM’s results, but long composite requests and sketch-based intent remain only partially solved.
7 Conclusion
- On 192 expert multimodal requests, the approach increases validity, instruction following, model quality, and acceptance for every evaluated LLM, with the largest gains for open-weight models.The human baseline remains ahead; long composite requests and sketch-based intent are still only partially solved.Source: §7 · p. 9
- AgenticCADedit replaces repeated reconstruction from the original geometry with actions that inspect, verify, and revise a persistent geometric state.Source: §7 · p. 9
- Long composite requests and sketch-based intent remain only partially solved, and the human baseline stays ahead.Source: §7 · p. 9