Source-linked AI summary
SketchGen: Generating Constrained CAD Sketches
Wamiq Reyaz Para, Shariq Farooq Bhat, Paul Guerrero, Tom Kelly, Niloy Mitra, Leonidas Guibas, Peter Wonka
TL;DR
CAD sketch generation is challenging because primitive records and constraint references are heterogeneous. SketchGen uses a transformer and a syntax-guided token language to generate primitives and constraints, and the authors report improved CAD sketch generation over the state of the art. The framework also supports full sketch generation and adding generated constraints to existing sketches.
Problem
Heterogeneous primitive parameters and constraint references make CAD sketches difficult to model with standard generative approaches.
Source: §1 · p. 2Method
Results
Takeaways & Limitations
The framework supports generating complete CAD sketches or augmenting existing sketches with generated constraints.
Source: §7 · p. 9The model covers only the most common primitive and constraint types, and its autoregressive generation cannot correct errors in earlier sequence parts.
Source: §7 · p. 9
Abstract
from arXiv · showhide
Computer-aided design (CAD) is the most widely used modeling approach for technical design. The typical starting point in these designs is 2D sketches which can later be extruded and combined to obtain complex three-dimensional assemblies. Such sketches are typically composed of parametric primitives, such as points, lines, and circular arcs, augmented with geometric constraints linking the primitives, such as coincidence, parallelism, or orthogonality. Sketches can be represented as graphs, with the primitives as nodes and the constraints as edges. Training a model to automatically generate CAD sketches can enable several novel workflows, but is challenging due to the complexity of the graphs and the heterogeneity of the primitives and constraints. In particular, each type of primitive and constraint may require a record of different size and parameter types. We propose SketchGen as a generative model based on a transformer architecture to address the heterogeneity problem by carefully designing a sequential language for the primitives and constraints that allows distinguishing between different primitive or constraint types and their parameters, while encouraging our model to re-use information across related parameters, encoding shared structure. A particular highlight of our work is the ability to produce primitives linked via constraints that enables the final output to be further regularized via a constraint solver. We evaluate our model by demonstrating constraint prediction for given sets of primitives and full sketch generation from scratch, showing that our approach significantly out performs the state-of-the-art in CAD sketch generation.
1 Introduction
CAD sketches represent geometric primitives and their relationships, but heterogeneous parameters and references make them difficult to model with standard generative approaches. SketchGen uses a structured sketch language and transformers to generate primitives together with constraints, and the paper reports improved sketch quality on Sketch-Graph.
1 Introduction
- CAD sketches link planar primitives through relations, and preserving those relations supports editing and design exploration.Such sketches can form the 2D starting point for CAD models that are lifted to 3D by extrusion.Source: §1 · p. 1
- Heterogeneous primitive parameters and constraint references make CAD difficult to encode, while rasterization loses constraints and padding is inefficient for complex models.Different instructions have different parameter counts, data types, and semantics; constraints also point to primitives or their parts.Source: §1 · p. 2
- SketchGen represents sketches as token sequences in a simple-syntax language and uses transformers to generate both primitives and their constraints.Its syntax trees provide additional guidance, and generated constraints can support solver-based correction of errors introduced by quantization.Source: §1 · p. 2
- SketchGen is evaluated on Sketch-Graph [27], with the authors reporting noticeable improvement in generated-sketch quality over existing and contemporary works.The paper also presents ablations of its design choices.Source: §1 · p. 2
2 Related Work
Prior work represents geometric designs with vectors, graphs, and CAD-specific structures, but approaches vary in how they handle variable-length commands and constraints. SketchGen’s related work situates it among vector-generation, structured-model, and CAD-generation research.
2 Related Work
- Vector-generation work links images and vectors through differentiable rendering or learns vector sequences; DeepSVG [4] uses hierarchical command, coordinate, and index embeddings.DeepSVG is described as most closely related to SketchGen in this line of work.Source: §2 · p. 2
- Structured-model generation includes GAN- and VAE-based layout methods and transformer approaches such as [22] [36]; DeepSVG [4] and polygen [20] are identified as closer comparisons.Polygen [20] uses pointer networks to link generated mesh faces to previously generated vertices.Source: §2 · p. 2
- Among notable CAD datasets, only Fusion360 [37] and SketchGraphs [27] include constraints, and SketchGraphs [27] provides a generative-modeling framework.Other cited work focuses on boundary representations or recovering CAD-like representations from unstructured input.Source: §2 · p. 2
- Figure 1 illustrates SketchGen’s representation as primitive and constraint sequences, with constraint tokens referring to primitives and syntax trees shown for initial entries.The paper also flags several arXiv papers as concurrent work: [7], [38], and [40].Source: Figure 1 · p. 3, §2 · p. 3
3 Overview
SketchGen models CAD sketches as heterogeneous graphs and converts them into sequences for transformer generation. Its explicit syntax supplies structure to interpret sequences and, according to the authors, improves model performance.
3 Overview
- A CAD sketch is a graph of parametric primitives and constraints, with variable parameter tuples and references that can target whole primitives or specific parts.Some primitives serve as construction aids rather than parts of the final 3D model.Source: §3 · p. 3
- The central modeling challenge is converting heterogeneous sketch graphs into suitable token sequences for transformer generation.The sequence representation is described as a main factor influencing transformer performance.Source: §3 · p. 3
- SketchGen gives its sketch language an explicit syntax that constrains which tokens can occur at each sequence position and helps interpret token sequences.The grammar is more complex than repeating coordinate or index triples, but remains explicitly stated.Source: §3 · p. 3
- Providing the sequence syntax as additional transformer input helps interpretation and increases generative-model performance.The framework trains separate models for primitives and constraints, with primitives represented first in the language.Source: §3 · p. 4
4 A Language for CAD Sketches
The paper defines a grammar-based token language for CAD sketches, with productions for primitives and constraints. Syntax-tree information supports sequence interpretation and generation, while parsing follows a specified primitive-then-constraint order.
4 A Language for CAD Sketches
- Each valid sketch-language sentence is a token sequence Q, and grammar production rules specify primitives and constraints.Special constants mark the start of a primitive or constraint and the end of a sequence; other terminals encode types, construction indicators, references, and parameters.Source: §4 · p. 4
- A syntax tree records the grammar derivation, and ancestor sequences Q3 and Q4 provide token-level type and parameter-type information to the model.The tree can support interpretation, enforce well-formed generation, and help the model interpret prior tokens.Source: §4 · p. 4
- Parsing emits primitive tokens before constraint tokens, stores parent and grandparent symbols, and concatenates the per-item sequences into Q.In the experiments, primitives follow designer drawing order and constraints are ordered by decreasing frequency.Source: §4 · p. 4
5 Models
Section 5 presents a two-stage sketch generator: it generates primitives, then predicts constraints over them, using token sequences augmented with syntax information.
5 Models
- Primitive and constraint subsequences are handled by separate generators, trained with teacher forcing and cross-entropy loss.The model decomposes graph generation into primitive generation and constraint prediction p(C|P).Source: §5 · p. 4, Figure 3 · p. 5
5.1 Primitive Generator
- Continuous primitive parameters are quantized separately by type using k-means bins fitted to dataset values, with k = 256.Separate bins account for differing, non-uniform parameter distributions.Source: §5.1 · p. 5
- A transformer decoder autoregressively generates primitive tokens from embeddings of Q, syntax-tree levels Q3 and Q4, and global position QI.At inference, it samples tokens with nucleus sampling and constructs Q3 and Q4 from generated primitive types to guide generation and restrict token values to valid ranges.Source: §5.1 · pp. 5–6
5.2 Constraint Generator
- A Pointer Network predicts constraint tokens by scoring indices into the encoded primitive list from the previously generated constraint-token features.At inference, Q4 is constructed from the generated constraint type and provides guidance that limits generated values to valid ranges.Source: §5.2 · p. 6
- The constraint generator encodes each primitive from its terminal-token embeddings, using filler values for missing tokens and a jointly trained transformer encoder for context.Its input uses Q, Q4, and QI; adding Q3 did not increase performance.Source: §5.2 · p. 6
6 Results
SketchGen is evaluated on full sketch generation and constraint prediction for supplied primitives. Its learned sketch and constraint distributions are reported to align more closely with the dataset than the SketchGraphs baselines, and generated constraints can correct primitive errors.
Evaluation
- The evaluation covers full sketch generation from scratch and auto-constraining, which predicts plausible constraints for supplied primitives.Source: §6 · p. 6
Sketch Generation Results
- Generated sketch statistics closely align with the data, and test-set NLL is significantly smaller than SG-sketch's; both learned systems outperform the random baseline.The generated-sequence comparison also examines variants using different nucleus-sampling parameters.Source: Figure 4 · p. 7, Table 1 · p. 7, §6 · p. 8
- Optimizing generated sketches to satisfy generated constraints corrects quantization errors and can close gaps between adjacent line endpoints.The authors report the resulting sketches and constraint sets as visually plausible.Source: §6 · p. 8
Auto-constraining Sketches
- For constraint generation, SketchGen's learned distribution has significantly lower test-set NLL than SG-constraint, while both outperform the random baseline.On generated sequences, disabling nucleus sampling increases syntactic error but decreases statistical error.Source: §6 · p. 8, Table 2 · p. 9
- 98.4% average accuracy was achieved when generating constraints for perturbed test-set sketches with known ground-truth constraints.The generated constraints were used with the OnShape optimizer.Source: §6 · p. 9
Ablation
- Removing both syntax inputs Q3 and Q4 causes a larger primitive-generation performance drop than removing Q3 alone; removing Q4 or mixing parameter types also significantly hurts performance.The shared-embedding ablation concerns grouping tokens by parameter type.Source: §6 · p. 9
7 Conclusion
SketchGen combines transformers, a carefully designed sketch language, and explicit syntax to generate full sketches or add constraints to existing ones. The authors identify limited primitive and constraint coverage and autoregressive error correction as future-work limitations.
7 Conclusion
- SketchGen uses transformers with a carefully designed sketch language and explicit syntax to generate primitives and constraints or add generated constraints to existing sketches.The framework supports both full sketch generation and auto-constraining existing sketches.Source: §7 · p. 9
- The model covers only common primitive and constraint types, while its autoregressive sequence generation cannot correct earlier errors.The authors propose extending the grammar to less common types and exploring backtracking.Source: §7 · p. 9
- The authors identify generating complex parametric geometry and studying formal languages and generative language models as future research directions.Source: §7 · p. 9
A Primitives and Constraints
The paper describes the supported sketch primitives and constraints, presents their grammar, and compares learned constraint prediction with a hand-crafted snapping baseline. SketchGen avoids the false positives produced by that baseline at the reported thresholds.
A Primitives and Constraints
- The supported primitives include points, lines, circles, and arcs, while constraints can link whole primitives or their sub-parts.The tables specify primitive parameters and constraint types; line and arc parameterizations are described using the OnShape convention.Source: A Primitives and Constraints · p. 12, Table 5 · p. 12
- Splines are omitted because they occur in fewer than 3% of SketchGraphs sketches, making them difficult to learn accurately from the data.The authors defer splines to future work.Source: A Primitives and Constraints · p. 12
B Complete Grammar
- The complete grammar represents primitives and constraints as sequences of tokens, with constraint rules referring to primitives or their sub-parts.The grammar includes point, line, circle, and arc primitives and constraints such as coincident, parallel, equal, horizontal, vertical, midpoint, perpendicular, and tangential.Source: B Complete Grammar · p. 13, Figure 6
C Comparison to Hand-crafted Rules for Constraints
- At snapping thresholds ϵ = 0.03 and ϵ = 0.1, the baseline significantly overestimates coincident constraints, producing false positives.The comparison is against SketchGen and ground truth; the authors attribute the errors to missing context-dependent reasoning.Source: C Comparison to Hand-crafted Rules for… · p. 13
D Additional Quantitative Results
Additional results compare SketchGen's generated sketch statistics and constraint effects with the dataset and human-design constraints. The section also describes the transformer and pointer-network architecture used for the models.
D Additional Quantitative Results
- Generated primitive-count and constraint-count distributions are compared with data sketches, and the authors report that inferred constraints are similar to those in the dataset.Figures 7 and 8 show the distributions across primitive and constraint types, respectively.Source: D Additional Quantitative Results · p. 13, Figure 7 · p. 14, Figure 8 · p. 14
- The constraints generated by SketchGen are in very close agreement with those imposed by human designers.The comparison examines the degrees of freedom removed per sketch by generated and ground-truth constraints.Source: D Additional Quantitative Results · p. 13
E.1 Architecture
- The primitive model is an autoregressively masked transformer decoder, while the constraint model is an autoregressively masked pointer network with an unmasked encoder.The architecture builds on GPT-2-style transformer blocks and adds support for the embedding strategy and additional input sequences.Source: §E.1 · p. 14
E.2 Training
The authors describe training choices for the primitive and constraint models and explain how they adapted the SketchGraphs baseline. For full baseline generation, the complete constraint set is needed to determine primitive parameters by solving constraints.
E.2 Training
- Training used a constant learning schedule, no weight decay, dropout of 0.2, and early stopping based on validation performance.The constraint model also used gradient clipping to a norm of 1.0 for training stability.Source: §E.2 · p. 14
- The primitive model was trained for 40 epochs and about 30 hours, while the constraint model was trained for 80 epochs and about 16 hours.Source: §E.2 · p. 14
E.3 Setup for the SketchGraphs Baseline
- The SketchGraphs baseline was retrained on the authors' subset; SG-sketch used batch size 16384 and four GPUs, while full generation required all dataset constraints for well-defined solved primitive parameters.The SG-constraint baseline used the same constraint subset as SketchGen.Source: §E.3 · pp. 14–15
F Metrics: Esyntax and Estat
The section defines Esyntax as a measure of grammar interventions and Estat as a comparison between generated and ground-truth sketch statistics. Statistical errors aggregate Earth Mover’s Distances across groups of primitive and constraint properties.
F Metrics: Esyntax and Estat
- Esyntax measures how often inference must force a grammar-correct Λ token where the model would have sampled another token.The rate is measured as a percentage of the Λ tokens expected from the grammar.Source: F Metrics: Esyntax and Estat · p. 15
- Estat compares generated-sketch statistics with ground-truth statistics using Earth Mover’s Distance after normalizing distribution domains to [0, 1].Recorded properties include primitive counts, positions, sizes, and constraint counts and pairwise relationships.Source: F Metrics: Esyntax and Estat · p. 15
- Primitive statistical error EPstat averages EMDs within property groups and then averages across groups; constraint error ECstat uses the same aggregation pattern.The full Estat is the sum of primitive and constraint errors.Source: F Metrics: Esyntax and Estat · p. 15
- With nucleus sampling p = 1.0, samples cover the distribution tail well and both reported statistical errors are lower than with p = 0.9.The statistics are a sample of possible primitive-parameter and pairwise-relationship measures.Source: F Metrics: Esyntax and Estat · pp. 15–16
G Qualitative Results
The qualitative examples show primitive samples, full sketches optimized against generated constraints, and constraints inferred for perturbed sketches. They also illustrate failure cases and state that no societal impacts specific to the method are foreseeable.
G.1 Primitive Generation
- Generated primitive samples are visualized with different colors denoting different primitives.The examples tend to form plausible sketches with symmetries, right angles, and parallel lines, despite quantization inaccuracies.Source: Figure 9 · p. 16, §G.1 · pp. 16–17
G.2 Full Sketch Generation
- Optimizing generated sketches to satisfy their generated constraints can correct errors introduced by quantization or primitive generation.The examples compare sketches before and after optimization; the solver used is provided by OnShape.Source: §G.2 · p. 17, Figure 10 · p. 17
G.3 Auto-constraining Sketches
- For perturbed test sketches, generated constraints often improve primitive alignment while inferring plausible degrees of freedom.Figure 11 includes a case where tangency is maintained but the final tangent lies on the wrong side.Source: §G.3 · p. 17, Figure 11 · p. 18
G.4 Failure Cases
- Failures include out-of-distribution primitives, unsatisfiable constraints, and satisfiable constraints that mismatch primitives and yield out-of-distribution sketches.The authors suggest better generators, more evenly represented primitive and constraint types, or inference-time syntactic or semantic checks as possible improvements.Source: §G.4 · p. 17, Figure 12 · p. 18
H Broader Impact
- The paper reports no foreseeable societal impacts specific to its method and leaves broader impacts of generative modeling and machine learning out of scope.Source: H Broader Impact · p. 18