PaperlayerView exact revision on arXiv

Source-linked AI summary

SketchGraphs: A Large-Scale Dataset for Modeling Relational Geometry in Computer-Aided Design

Ari Seff, Yaniv Ovadia, Wenda Zhou, Ryan P. Adams

arXiv:2007.08506v1cs.LGstat.ML

TL;DR

Machine learning for parametric CAD is motivated by the prospect of supporting design workflows, while CAD also offers a well-scoped domain for studying program induction. SketchGraphs provides 15 million sketches with geometric constraint graphs and an open-source processing pipeline, and demonstrates benchmarks for sketch generation and constraint prediction.

  • Problem

    Learning to construct and reason about CAD designs is an open research direction, and inferring design intent and constraints from unconstrained geometry remains a sought-after capability.

    Source: §1 · p. 1, §4.1 · p. 6

  • Method

    The authors extract 15 million Onshape sketches, represent primitives and their constraints as geometric constraint graphs, and provide an open-source processing pipeline.

    Source: §1 · pp. 2–3, §3.2 · p. 5

  • Results

    SketchGraphs demonstrates unconditional sketch generation and conditional constraint inference as initial benchmark use cases.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The dataset and pipeline are intended to facilitate research in machine-learning-aided design, relational reasoning, and program induction.

    Source: §5 · p. 8

    External constraints involving default sketch geometry are not currently retrievable through Onshape’s API.

    Source: §3.2 · p. 5

Abstract

from arXiv · show

Parametric computer-aided design (CAD) is the dominant paradigm in mechanical engineering for physical design. Distinguished by relational geometry, parametric CAD models begin as two-dimensional sketches consisting of geometric primitives (e.g., line segments, arcs) and explicit constraints between them (e.g., coincidence, perpendicularity) that form the basis for three-dimensional construction operations. Training machine learning models to reason about and synthesize parametric CAD designs has the potential to reduce design time and enable new design workflows. Additionally, parametric CAD designs can be viewed as instances of constraint programming and they offer a well-scoped test bed for exploring ideas in program synthesis and induction. To facilitate this research, we introduce SketchGraphs, a collection of 15 million sketches extracted from real-world CAD models coupled with an open-source data processing pipeline. Each sketch is represented as a geometric constraint graph where edges denote designer-imposed geometric relationships between primitives, the nodes of the graph. We demonstrate and establish benchmarks for two use cases of the dataset: generative modeling of sketches and conditional generation of likely constraints given unconstrained geometry.

1 Introduction

Parametric CAD represents designs as sketches whose geometric primitives are linked by constraints, making the domain relevant to design assistance and program-induction research. SketchGraphs provides a large dataset and open-source tools, with initial benchmarks for sketch generation and constraint prediction.

Motivation

  • Parametric CAD offers a relatively well-scoped setting for studying program induction and synthesis because designs encode rich geometric structures through implicit programs.The introduction also motivates learning from human-designed structures as a route to reasoning about modularity, symmetry, and functional constraints.Source: §1 · p. 2

Dataset contribution

  • SketchGraphs contains 15 million CAD sketches represented by ground-truth geometric constraint graphs, targeting relational sketch geometry that existing datasets leave underexplored.The graph represents primitives as nodes and designer-imposed relationships as edges.Source: §1 · pp. 2–3

Open-source tools

  • The authors release an open-source processing and visualization tool suite alongside the dataset.The suite includes domain types and a local renderer.Source: §1 · pp. 2–3

Benchmark use cases

  • Initial benchmarks cover generative modeling of sketches and prediction of likely constraints from unconstrained geometry.The dataset is also described as supporting conditional completion, autoconstraining, and research on inference from rendered sketches.Source: §1 · p. 3

2 Related Work

Related work distinguishes SketchGraphs from datasets centered on 3D shapes or ordinary hand-drawn sketches, and situates its graph representation among generative modeling and geometric program-induction approaches.

CAD datasets

  • Unlike voxel- and mesh-based 3D datasets or ABC’s curve-and-surface focus, SketchGraphs targets 2D CAD sketches and their geometric constraints.The related-work discussion notes that common 3D representations do not retain the designer’s construction route.Source: §2 · p. 3

Sketch datasets

  • In CAD, a sketch is the constrained 2D basis of a 3D model, unlike datasets of hand-drawn images or vector drawings of general categories.QuickDraw and Sketchy focus on general-category drawings or image retrieval, whereas SketchGraphs focuses on relational geometry.Source: §2 · pp. 3–4

Graph-structured generative modeling

  • Graph-generative approaches include incremental node-and-edge prediction, linearized adjacency models, and soft-adjacency decoding; SketchGraphs uses a natural construction ordering to address graph-order ambiguity.The passage notes that an n-node graph may have up to n! node orderings.Source: §2 · p. 4

Geometric program induction

  • Geometric program-induction work includes inferring TikZ code from images and reconstructing 3D shapes, while SketchGraphs adds sketch rendering and constraint-solver querying to its pipeline.The passage frames these tools as support for research in geometric program induction.Source: §2 · p. 4

3 The SketchGraphs Dataset

SketchGraphs extracts sketches from public Onshape models and represents their primitives and imposed constraints as geometric constraint graphs. The dataset also preserves construction-order information, enabling sequence-based modeling and analysis of sketch structure.

3.1 Acquisition

  • Over 15 million sketches were extracted from public Onshape PartStudios in documents created between 2015 and 2020.The PartStudios also contain non-sketch features, which are not stored in the dataset.Source: §3.1 · pp. 4–5

3.2 Geometric constraint graphs

  • Each sketch is a multi-hypergraph of primitives and constraints, with loops for single-primitive constraints and separate sub-primitive nodes for point-specific relations.Primitive coordinates need not initially satisfy constraints; a geometric constraint solver adjusts them.Source: §3.2 · p. 5
  • 0.598 is the Pearson correlation between each sketch’s total DOF and the DOF removed by constraints; users can query sketches by constrainedness thresholds.The calculation excludes constraints involving sketch axes.Source: §3.2 · p. 5, Figure 4 · p. 6

3.3 Construction sequence extraction

  • 0.70 vs. 0.38 is the probability that adjacent construction-order nodes are graph-adjacent, compared with randomly selected nodes; earlier nodes also tend to have greater degree.The order in which users added primitives provides a ground-truth node ordering.Source: Figure 3 · p. 5, §3.3 · p. 6
  • The canonical sequence inserts each constraint immediately after its member nodes, while an imported-geometry setting can place all constraints after all primitives.The relative interleaving of primitives and constraints is not recorded in Onshape models.Source: §3.3 · p. 6

4 Case Studies of Supported Applications

The case studies establish initial baselines for predicting likely constraints from geometry and generating sketch graphs. They also describe potential uses in interactive design, image-based CAD inference, and semantic representations.

4.1.2 Evaluation

  • In the illustrated edits, the model infers many coincident, distance, and equality constraints but misses a 7 mm rectangle distance constraint while preserving circle midpoint alignment.The figure shows an upward drag and a combined enlargement and rightward drag of the top circle.Source: Figure 5 · p. 7

4.1.1 Model

  • Autoconstraining uses autoregressive message passing to predict edge partners and labels node by node, masking approximately satisfied constraints at inference.The model was trained on 2.2 million sketches with common primitive types and at most 16 primitives per sketch; 50K held-out sketches were used for testing.Source: §4.1.1 · p. 7, §4.1.2 · p. 7

4.1.2 Evaluation

  • Autoconstraining achieved average edge precision 0.74, average edge recall 0.74, average F1 0.71, and 0.495 bits per edge test negative log-likelihood versus 6.09 bits per edge uniform valid-constraint entropy.The model predicts edges on a held-out test dataset.Source: §4.1.2 · p. 7

4 Case Studies of Supported Applications

  • Generative modeling may support conditional completion and plausibility-based correction of dubious sketch operations.The case studies are intended as baselines for future research in mechanical design and related program synthesis and induction tasks.Source: §4 · p. 6, §4.2 · pp. 7–8

4.2 Generative modeling

  • The generative model predicts full sketch construction sequences, modeling constraint parameters and primitive types while leaving final primitive coordinates to a constraint solver.It adds nodes autoregressively, includes associated sub-primitive nodes and edges, and can stop graph construction with a stop token.Source: §4.2 · p. 8, §4.2.1 · p. 8

4.2.2 Evaluation

  • 28.2 bits per graph is the model’s average test negative log-likelihood, compared with 85.6 bits per sketch from a standard LZMA compressor.The paper also reports distributional statistics and generated-sketch renderings in supplementary material.Source: §4.2.2 · p. 8

4.3.1 CAD inference from images

  • Noisy rendering of sketches can produce training pairs of rendered images and their corresponding geometric constraint graphs for CAD inference from images.The rendering can simulate a hand-drawn appearance.Source: §4.3.1 · p. 8

4.3.2 Learning semantic representations

  • Unsupervised learning on SketchGraphs may support vectorial latent representations of CAD designs for downstream tasks such as search and content recommendation.The paper frames these as possibilities enabled by learned semantic representations.Source: §4.3.2 · p. 8

5 Conclusion and Future Work

The conclusion presents SketchGraphs as a dataset and processing pipeline for research in ML-aided design, relational reasoning, and program induction. The appendices specify primitive parameters and constraint parameter schemas.

5 Conclusion and Future Work

  • The paper introduces a large-scale sketch dataset and processing pipeline, with initial benchmarks for unconditional generation and conditional constraint inference; future work targets full construction operations and additional applications.The dataset is intended to support research in ML-aided design, relational reasoning, and program induction.Source: §5 · pp. 8–9

A Primitive Parameters

  • Primitive parameterizations include placement and construction status, with conversions from Onshape’s parameterization to standard forms.The appendix lists parameters for points, lines, circles, arcs, and ellipses, including their degrees of freedom.Source: A Primitive Parameters · p. 12

B Constraint Parameters

  • Constraint parameters specify acted-on primitives and any numerical or categorical values needed to define behavior; external constraints are excluded.The appendix notes that constraint parameter documentation is sparse and functionality was determined through developer discussions and solver usage.Source: B Constraint Parameters · p. 13

B.1 Parameters

  • The listed general parameters include primitive references, lengths, angles, orientation, distance measurement direction, and relative half-space.Source: §B.1 · p. 13

B.2 Schemata

  • Constraint schemas specify which primitive references and parameters commonly accompany constraint types, although some constraints use multiple schemas.For example, a distance constraint commonly uses two primitive references, direction, half-space values, and length.Source: §B.2 · p. 13

B.3 Numerical parameter distributions

Length and angle parameters have highly concentrated value distributions, with angles less diverse than lengths. The section summarizes the most common values and their cumulative frequency.

B.3 Numerical parameter distributions

  • The 300 most frequent values account for 95.8% of angle occurrences and 82.1% of length occurrences, with angles less diverse than lengths.Angle is scale-invariant; Table 2 lists common values for length and angle, with length values shown in units requiring the fewest digits.Source: §B.3 · pp. 13–14, Table 2 · p. 14
  • The cumulative-frequency plot sorts unique parameter values from most to least frequent.Source: Figure 6 · p. 14

C Example sketch constructions

The section presents example sketches through construction-step sequences, proceeding from top left to bottom right according to user-defined primitive orderings.

C Example sketch constructions

D.1 Task description

The model predicts constraints as a sequence of partner and label decisions, using graph representations and message passing to produce task-specific probabilities.

D.1 Task description

  • Given the current sketch and its constraints, partner prediction selects a node or a sentinel to advance to the next node, while label prediction predicts the constraint type.Source: §D.1 · p. 17

D.2 Model description

  • The model uses a global complexity parameter m set to 384 in the reported results.Source: §D.2 · p. 17
  • The model combines primitive and constraint embeddings with recurrent message passing over the graph, then computes a global graph representation.Primitive types and quantized continuous parameters are embedded; constraint types are embedded, and message passing uses three steps.Source: §D.2 · p. 17
  • The partner readout scores each node using node, current-node, and global representations, with a sentinel assigned an implicit zero log-probability.Source: §D.2 · pp. 17–18
  • The constraint-label readout uses the final representations of the current and partner nodes together with the global representation to score edge types.Source: §D.2 · p. 18

D.3 Training

Training uses about 2.2 million filtered sketches, samples edges uniformly, and runs for 150 epochs on specified hardware.

D.3 Training

  • Training uses about 2.2M sketches, filtered so the largest contains no more than 16 primitives, and uniformly samples possible edges or stop edges, weighting longer sketches more heavily.Source: §D.3 · p. 18
  • The model trains for 150 epochs with Adam, batch size 8192, and learning rate 10^-5, decayed by a factor of 10 at epochs 50 and 150.The learning rate applies to the loss summed over the batch.Source: §D.3 · p. 18
  • Training takes 3 hours and 30 minutes on four Nvidia Titan X (Pascal) GPUs and dual Intel Xeon E5-2667 v4 processors.Source: §D.3 · p. 18

D.4 Evaluation

On 50K held-out sketches, the model achieves average edge recall and precision of 0.74(±0.00). The generative-model description distinguishes its input, message passing, and readout design from the autoconstrain model.

D.4 Evaluation

  • Evaluation uses 50K held-out sketches and masks invalid edge predictions, then reconstructs and rescales predictions using the full joint distribution.The mask does not factorize because partner validity can depend on the constraint type.Source: §D.4 · p. 18
  • 0.74(±0.00) average edge recall and 0.74(±0.00) average edge precision are reported on the held-out evaluation.The brackets indicate the standard error of the estimate.Source: §D.4 · p. 18

E.1 Model description

  • The generative model ignores primitive parameters, represents constraint parameters, and uses node embeddings directly without a recurrent message-passing model.Source: §E.1 · p. 18
  • Its readouts predict the next primitive type from the graph embedding and sequentially predict constraint features with a recurrent neural network.Constraint-feature readout uses the constraint type and a representation of the participating primitives.Source: §E.1 · p. 18

E.2 Training

The generative model is trained on the autoconstrain model’s data, including numerical constraint parameters for frequent schemas, with a specified Adam training setup.

E.2 Training

  • Training uses numerical constraint parameters from the most frequent schemas as targets, on the same data as the autoconstrain model.Source: §E.2 · p. 19
  • Adam training uses batch size 6144, learning rate 10^-5, and 150 epochs, with the rate decayed by a factor of 10 at epochs 50 and 150.The learning rate applies to the loss summed over the batch.Source: §E.2 · p. 19
  • Training took 3 hours and 50 minutes on a server with four Titan X GPUs, although a CPU bottleneck limited use to three GPUs.Source: §E.2 · p. 19

E.3 Evaluation

The baseline generates some patterns seen in small sketches but does not produce many sophisticated sketches. Its evaluation compares generated and training-set statistics and uses a solver to configure sampled sketches.

E.3 Evaluation

  • The evaluation uses the same testing split as the autoconstrain model and estimates LZMA performance from compressed construction-sequence integers.The estimated average entropy rate is computed from the compressed-size difference between the first 100,000 and 50,000 elements, divided by 50,000.Source: §E.3 · p. 19
  • The evaluation compares sketch-size and degrees-of-freedom distributions for 10K generated samples and the training set.Figures 12 and 13 show the distributions; their error bars represent bootstrapped 5th and 95th percentiles.Source: §E.3 · p. 19, Figure 12 · p. 20, Figure 13 · p. 20
  • Primitive and constraint type distributions are also compared between generated sketches and the training set.Figure 14 reports average counts of primitives and constraints of each type per sketch.Source: §E.3 · p. 19, Figure 14 · p. 20
  • The baseline captures some small-sketch patterns but does not produce many sophisticated sketches.It does not model primitive coordinates; the solver determines configurations from uniform initialization.Source: Figure 11 · p. 19, §E.3 · p. 19
Loading 2007.08506v1…