PaperlayerView exact revision on arXiv

Source-linked AI summary

ExpConCAD: Experience-Guided Text-to-CAD Generation from Shape Descriptions with Implicit Spatial Constraints

Jingyao Liu, Jinkang Tang, Chen Huang, Wenqiang Lei, See-Kiong Ng

arXiv:2608.24760v1cs.CL

TL;DR

Text-to-CAD systems often require detailed spatial instructions, although realistic descriptions leave essential construction constraints implicit. ExpConCAD recovers construction structure and scopes, retrieves reusable experience, completes the missing constraints, and generates executable CadQuery programs. On CADFUSION-HARD, it improves VLM-Score over the strongest baseline by 22.5% on Qwen3.5-27B and 27.9% on GPT-5.

  • Problem

    Existing Text-to-CAD methods typically require detailed geometric, modeling, and spatial instructions, while realistic descriptions omit constraints needed for valid CAD construction.

  • Method

    ExpConCAD recovers construction structure and constraint scopes, retrieves reusable scoped experience, completes missing spatial constraints, and generates executable CadQuery programs.

  • Results

    22.5% and 27.9%: ExpConCAD improves VLM-Score over the strongest baseline on Qwen3.5-27B and GPT-5, respectively.

  • Takeaways & Limitations

    Construction-aware reasoning and reusable experience memory support spatial constraint completion from underspecified descriptions.

  • Takeaways & Limitations

    The work does not study how experience memory should be updated, deleted, merged, or validated over time.

Abstract

from arXiv · show

Text-to-CAD aims to generate executable CAD programs from natural-language descriptions. However, real-world descriptions are often underspecified and omit critical spatial constraints required for valid CAD construction, a challenge that has been largely overlooked by existing methods. In this paper, we argue that missing spatial constraints should be inferred with respect to the underlying construction structure and informed by reusable design experience. Based on this insight, we propose ExpConCAD, an experience-enhanced framework for implicit spatial constraint completion. ExpConCAD first recovers the intended construction structure and constraint scopes, then retrieves relevant constraint-completion experience for similar scopes to complete the missing spatial constraints, and finally generates executable CadQuery programs. Extensive experiments demonstrate the effectiveness of ExpConCAD and provide insights into the role of construction structure understanding and experience memory in spatial constraint completion. Our code is available at: https://github.com/Hotjiashell/ExpConCAD.

1 Introduction

Text-to-CAD methods often assume detailed geometric and spatial instructions, while realistic shape descriptions leave essential constraints implicit. ExpConCAD addresses this gap by recovering construction structure, completing scoped constraints with reusable experience, and improving benchmark performance.

  • Motivation: Realistic shape descriptions often omit spatial relations needed for valid CAD construction, despite existing methods requiring detailed inputs.Such detailed instructions impose a substantial burden on users.
  • Approach: ExpConCAD infers missing spatial relations from construction structure and constraint scopes rather than surface descriptions alone.The framework uses construction-aware reasoning before generating executable CadQuery code.
  • Approach: Prior CAD cases provide transferable patterns for completing recurring scopes such as cutter containment, cutter separation, and through-cut depth.Different geometries can share comparable constraint scopes, enabling experience retrieval across cases.
  • Results: 22.5% and 27.9%: ExpConCAD improves VLM-Score over the strongest baseline on Qwen3.5-27B and GPT-5, respectively.The evaluation uses CADFUSION-HARD, a benchmark with underspecified spatial constraints.
  • Contributions: The paper identifies implicit spatial constraints as a critical Text-to-CAD challenge and validates construction-aware reasoning with reusable experience memory.Its broader aim is to infer unstated design knowledge for more human-centric CAD agents or copilots.

2 Related Work

Related Text-to-CAD work increasingly generates executable CAD programs, but most methods assume descriptions explicitly provide construction information. ExpConCAD instead targets underspecified shape descriptions with implicit spatial constraints.

  • Background: CAD modeling traditionally requires domain expertise and familiarity with professional modeling software, motivating natural-language Text-to-CAD systems.Text-to-CAD converts natural-language descriptions into structured CAD representations.
  • Comparison: Table 1 distinguishes methods by implicit-spatial-constraint coverage, description type, and output representation, including CED, USD, Seq., and CQ.ISC indicates whether a method explicitly addresses implicit spatial constraint completion.
  • Executable CAD Generation: Recent methods generate executable CAD programs such as CadQuery scripts from natural-language descriptions.These programs are executed to create CAD models.
  • Research Gap: Most existing methods assume inputs explicitly specify modeling operations, geometric parameters, and spatial relations required for construction.This assumption covers both task-specific models and recent LLM-based approaches.

3 EXPCONCAD

ExpConCAD transforms underspecified descriptions into executable CadQuery programs through construction-structure recovery, scoped spatial constraint completion, and experience-guided generation. Its memory converts diagnosed CAD failures into reusable, retrievable constraint patterns.

  • Problem Formulation and Overview: Given an underspecified input x, ExpConCAD completes omitted spatial constraints required for valid CAD construction before producing executable program y.The framework combines construction-aware reasoning with an experience memory B.
  • Construction Structure Understanding: Construction Structure Understanding recovers CAD elements, modeling operations, and constraint scopes implied by the input.For a plate with two holes, it identifies the plate, cylindrical cutters, Boolean cuts, and relevant placement scopes.
  • Spatial Constraint Completion: Spatial Constraint Completion instantiates coordinate conditions, distance bounds, size relations, and operation parameters within the recovered scopes.Examples include cutter containment, cutter separation, and sufficient through-cut depth.
  • CadQuery Generation: The code generator conditions on the description, recovered structure, and completed constraints, then checks execution through compilation and rendering.If execution fails, only the code is revised while the recovered structure and completed constraints remain fixed.
  • Experience Memory: Experience items encode an applicable scope, a reusable constraint pattern, and CadQuery implementation guidance for retrieval and actionability.Patterns include containment, non-overlap, symmetry, through-cut, parameter bounds, placement rules, and Boolean-operation settings.
  • Experience Construction: Experience memory is built from diagnosed failures such as overlapping holes, cutters outside the base, or cuts that do not pass through the object.These failures become scoped guidance for analogous future constructions.
  • Experience Use: During inference, queries match identified scopes with applicable situations in memory, retrieving targeted guidance for the current constraint-completion task.Retrieved experience conditions SCC alongside the current construction structure rather than matching whole CAD cases.

4 Experiments

Experiments on CADFUSION-HARD evaluate EXPCONCAD with validity-aware metrics, ablations, and analyses of construction understanding, experience memory, and transfer to unseen cases. Results consistently support construction-aware spatial constraint reasoning, with additional gains from reusable experience.

  • Main Results: EXPCONCAD achieves the highest VLM scores on both Qwen3.5-27B and GPT-5 under validity-aware evaluation.Non-renderable outputs are included using worst-decile scores estimated from valid outputs.
  • Main Results: Adding experience memory further improves VLM by 11.2% on Qwen3.5-27B and 7.5% on GPT-5.Geometry-based gains from experience are less consistent because underspecified descriptions may admit multiple valid geometries.
  • Ablation Study: Ablations show that verification reduces Invalid Ratio, while CSU and SCC progressively improve VLM by identifying scopes and completing missing spatial constraints.The full framework performs best when executable-code verification, construction-aware reasoning, and experience-enhanced SCC are combined.
  • Key Design Choices: Direct SCC produces limited and unstable gains, whereas CSU followed by SCC consistently improves performance across both backbones.Recovering construction elements, operations, and constraint scopes provides explicit targets for completion.
  • Key Design Choices: Experience improves SCC, but simply adding more items does not always help: Qwen3.5-27B peaks at 50 items before saturation.Case-level memories can mix several constraints, making them difficult to match to a specific scope in a new case.
  • Robustness and Transfer: EXPCONCAD improves over Vanilla on both CADFusion-Hard and Text2CAD-Hard, with further experience gains on unseen Text2CAD-Hard cases.CadFusion improves on CADFusion-Hard but drops below Vanilla on Text2CAD-Hard.

5 Conclusion

EXPCONCAD addresses underspecified Text-to-CAD descriptions by combining construction-aware spatial constraint reasoning with experience memory. The paper frames inferring unstated design intent as important for more human-centric CAD agents.

  • EXPCONCAD targets missing spatial constraints that hinder accurate CAD construction from underspecified descriptions.
  • The framework combines construction-aware spatial constraint reasoning with an experience memory.
  • The paper presents inferring unstated design intent as a broader direction for CAD agents or copilots collaborating with human designers.

Limitations

The main limitation is that EXPCONCAD does not study how its experience memory should be maintained over time.

  • EXPCONCAD does not study updating, deleting, merging, or validating experiences over time.Future work should retain high-quality patterns, merge redundant experiences, and keep the memory compact and reliable.
  • Continuously expanding the memory may increase storage and retrieval costs without adding genuinely new constraint-completion patterns.
  • Multiple experiences can instantiate the same pattern while differing in quality or executability.

Ethical Considerations

The paper describes a staged, execution-aware generation pipeline supported by a curated CadQuery guide and publicly released artifacts. Its experience construction uses scored and verified CAD generations.

  • The study states that it does not involve human subjects, sensitive personal data, or privacy-sensitive information.Source data come from open-source communities.
  • The authors state that code, data, and artifacts comply with applicable licenses, with EXPCONCAD artifacts released under CC-BY 4.0 for research purposes.
  • EXPCONCAD uses three stages: structure understanding and query generation, spatial constraint completion and code generation, then execution-error refinement.
  • A curated Function Guide supplies essential CadQuery API usage patterns across all three stages.
  • Experience extraction retains CAD generations with initial GPT-5 scores below 7 for later distillation.
  • Distilled experiences are stored only after regenerated results improve by more than two points, with at most three retry iterations.

B Case Studies of EXPCONCAD

Case studies show EXPCONCAD recovering coupled geometric structures and implicit spatial constraints that baselines often miss. The examples illustrate how construction structure and retrieved experience guide constraint completion.

  • Case Studies: The four hard cases involve multi-face through holes, hollow structures, internal partitions, rounded corners, and repeated local cutouts.
  • Case Studies: In the mounting-block case, EXPCONCAD better recovers multi-face hole structure and relative hole placement than baselines.
  • Case Studies: In the hollow-prism case, EXPCONCAD better preserves the hollow body and side-cutout interactions.
  • Case Studies: For internal bars, EXPCONCAD captures parallel partitions and the resulting rectangular void structure.
  • Case Studies: For the tall hollow prism, EXPCONCAD better preserves coupled shell, rounded-corner, and repeated-cutout constraints.
  • Case Studies: Overall, the qualitative cases and quantitative results jointly support CAD-context grounding, implicit constraint completion, and reusable experience for vague-shape-level generation.
  • Generation Process: In the third case, construction understanding identifies interactions between “voids_union” and “main_body” before retrieval and constraint completion.
  • Case Studies: Without experience, the model can omit through-cut constraints, leaving voids enclosed within the main body.

C.3 Evaluation Metrics

The evaluation combines text–shape consistency, geometric fidelity, and executability measures. It also introduces view- and rotation-aware metrics to reduce penalties from incomplete viewpoints or unknown orientation.

  • VLM2v scores a generated shape from two informative diagonal perspective views rather than conventional orthographic three-view renderings.The views are selected to provide complementary surface information while reducing occlusion redundancy.
  • Text–shape consistency evaluates global topology, component fidelity and count, and spatial distribution using VLM-based scoring.These dimensions assess overall structure, key component correctness, relative placement, scale, and structural inconsistencies.
  • VLM2v achieves higher correlation with human judgments than VLM3v, indicating better alignment with human preferences for text–shape consistency.The correlation study uses 50 text–shape pairs and five CAD-experienced annotators.
  • Rotation-invariant IoU maximizes voxel overlap across 24 discrete cube rotations of the ground-truth shape.This prevents orientation misalignment from unfairly lowering shape-correctness scores.
  • Chamfer Distance measures geometric similarity between point sets sampled from predicted and ground-truth mesh surfaces.The evaluation samples 2000 points from each mesh, while rotation-invariant CD takes the minimum distance across 24 prerotated ground-truth point clouds.
  • Invalid ratio measures failed program executions, while validity-aware evaluation includes non-renderable outputs instead of conditioning metrics on successful samples.A lower invalid ratio indicates more reliable generation, and mean and median geometric metrics summarize aggregate and typical performance respectively.

C.5 Additional Settings for Diagnostic Experiments

The diagnostic experiments isolate the effects of verification, construction-structure understanding, spatial constraint completion, and experience memory. Additional analyses test memory scaling, cross-dataset transfer, and evaluation protocols.

  • Ablation Study: The ablation progressively adds verification, Construction Structure Understanding, Spatial Constraint Completion, and experience-enhanced SCC to a prompting baseline.Verification targets non-renderable outputs; CSU recovers CAD elements, operations, and constraint scopes before SCC completes missing relations.
  • Construction Structure Understanding: The CSU analysis compares verification-only, direct SCC from shape descriptions, and SCC based on recovered construction structures.The variants have similar renderability, with invalid-ratio differences within 3 percentage points, so VLM is the primary metric.
  • Experience Memory: The experience-memory analysis varies memory size while keeping the overall EXPCONCAD pipeline fixed for Qwen3.5-27B and GPT-5.The tested memories range from 50 items built from 45 source cases to 500 items built from 158 source cases.
  • Cross-Dataset Generalization: The cross-dataset experiment tests Qwen3.5-27B on CADFUSION-HARD and unseen TEXT2CAD-HARD using experience built only from CADFUSION-side cases.It contrasts end-to-end learned patterns with explicit construction-aware constraint completion and retrieved experience.
  • Full CADFUSION Results: On the full CADFUSION benchmark, experience improves Qwen3.5-27B from 7.56 to 7.74 but slightly hurts GPT-5 on easier cases.The authors attribute the less stable gains to the structured pipeline already completing many missing constraints in simpler examples.
  • Ablation Study: EXPCONCAD achieves the best VLM score for both Qwen3.5-27B and GPT-5 in the ablation, while removing construction grounding or implicit constraint completion weakens performance.The gap between the experience-free and full variants indicates an additional benefit from reusable experience.

D.5 Additional Analysis of Experience in EXPCONCAD

The experience analysis shows that experience granularity affects retrieval usefulness. Fine-grained, pattern-specific experiences provide more targeted and reusable guidance than coarse case-level summaries.

  • A source case involving a cylindrical hole and an inserted cylinder contains two distinct constraint-completion patterns.The patterns concern constructing the hole and modeling the cylinder inside it.
  • Qwen3.5-27B combines both patterns into one experience, but its content only constrains the cylinder insertion.The resulting experience cannot guide hole construction when that task is required.
  • GPT-5 separates the same case into two pattern-specific experiences covering hole construction and cylinder insertion.The hole experience specifies that hole height must exceed body height, while the second details insertion constraints.
  • Fine-grained GPT-5 experiences improve retrieval accuracy, task performance, and reuse across different CAD contexts compared with coarse Qwen3.5-27B guidance.The analysis links greater versatility to experiences aligned with specific patterns rather than whole cases.

D.6 Token Consumption Analysis

ExpConCAD achieves strong performance with increased token usage, and this overhead varies across backbone models. Qwen3.5-27B exhibits context-saturating repetitive generation, whereas GPT-5 avoids this behavior and uses fewer tokens than CADCodeVerify.

  • ExpConCAD's performance improvement comes at the cost of increased token usage, with overhead varying across backbone models.
  • Under Qwen3.5-27B, repetitive generation occurs in 8.5% of Ours w/o Exp samples and 10% of Ours samples.The duplicated content can continue until the context window is exhausted, significantly inflating average token consumption.
  • With GPT-5, context-saturating repetition is absent, and ExpConCAD outperforms CADCodeVerify while consuming fewer tokens.
Loading 2608.24760v1…