Source-linked AI summary
SkexGen: Autoregressive Generation of CAD Construction Sequences with Disentangled Codebooks
Xiang Xu, Karl D. D. Willis, Joseph G. Lambourne, Chin-Yi Cheng, Pradeep Kumar Jayaraman, Yasutaka Furukawa
TL;DR
SkexGen addresses the challenge of generating CAD construction sequences that support substantial design variation and user control. It uses autoregressive Transformers with disentangled codebooks for topological, geometric, and extrusion variations. Experiments report diverse, high-quality CAD generation with effective control and exploration, while generation is slower than DeepCAD and interpolation is challenging when topology changes.
Problem
Existing parametric CAD models are brittle over large design changes, while user control over generated CAD designs remains elusive.
Method
SkexGen encodes topological, geometric, and extrusion variations into disentangled codebooks and autoregressively decodes sketch-and-extrude construction sequences.
Results
Extensive qualitative and quantitative evaluations on public benchmarks demonstrate state-of-the-art performance, including realistic and diverse CAD generation with effective control and exploration.
Takeaways & Limitations
Disentangled codebooks provide a representation for controlling and exploring topological and geometric CAD variations.
Takeaways & Limitations
SkexGen takes 90 seconds to generate 10,000 samples versus 15 seconds for DeepCAD, and interpolation remains challenging when many topological changes are required.
Abstract
from arXiv · showhide
We present SkexGen, a novel autoregressive generative model for computer-aided design (CAD) construction sequences containing sketch-and-extrude modeling operations. Our model utilizes distinct Transformer architectures to encode topological, geometric, and extrusion variations of construction sequences into disentangled codebooks. Autoregressive Transformer decoders generate CAD construction sequences sharing certain properties specified by the codebook vectors. Extensive experiments demonstrate that our disentangled codebook representation generates diverse and high-quality CAD models, enhances user control, and enables efficient exploration of the design space. The code is available at https://samxuxiang.github.io/skexgen.
1. Introduction
SkexGen addresses the difficulty of generating CAD models with large design changes and controllable variation by autoregressively generating sketch-and-extrude construction sequences from disentangled codebooks. Evaluations report high-quality, diverse generation with effective control and exploration.
- Motivation: Parametric CAD models support small corrections and parameter changes but are brittle over large topological modifications.Constructing such models also requires professional designer expertise.
- Method: SkexGen uses discrete codebooks and autoregressive Transformers to generate sketch-and-extrude CAD construction sequences.Sketch operations create 2D primitives, while extrusion lifts and combines them into 3D.
- Method: Three disentangled codebooks encode topological, geometric, and extrusion variations for controllable CAD generation.The codebooks are designed to support variation and exploration of the design space.
- Evaluation: SkexGen generates high-quality and diverse CAD construction sequences while enabling effective control and exploration of designs.These properties are stated as central contributions of the architecture and disentangled codebooks.
- Evaluation: Extensive qualitative and quantitative evaluations on public benchmarks demonstrate state-of-the-art performance.The evaluation compares SkexGen with multiple baselines and state-of-the-art methods.
2. Related Work
Prior work explores CAD reconstruction and construction-sequence generation with parametric, programmatic, and neural representations. SkexGen builds on this progress to target controllable exploration of CAD design variations.
- Representations: Constructive solid geometry represents 3D shapes as trees combining parametric primitives with Boolean operations.Parametric CAD remains the prevalent paradigm for mechanical design and extensively uses sketch-and-extrude operations.
- Construction-sequence generation: Large-scale parametric CAD datasets have enabled learning directly from modeling operations used by CAD users.Prior methods predict extrusion operations for partial construction-sequence recovery.
- User control: Recent construction-sequence models produce diverse shapes, but providing user control over an existing design remains elusive.Conditioning on images, point clouds, or hand-drawn sketches converts an existing design into a CAD construction sequence.
- Codebook architectures: Codebooks have improved diversity and user control in image and audio generation and suit structurally regular CAD sequences.The cited prior work motivates applying codebook architectures to CAD modeling sequences.
3. Sketch-and-Extrude Construction Sequence
SkexGen represents sketch-and-extrude CAD models as hierarchical constructions built from curves, loops, faces, sketches, extruded sketches, and Boolean combinations. Its token sequence records topology, geometry, primitive boundaries, extrusion operations, and sequence termination.
- Construction sequence: The representation is a hierarchy of primitives designed to be more expressive and learnable than earlier representations.It builds on Turtle-Gen and DeepCAD with several modifications.
- Primitive hierarchy: Curves form loops, loops bound faces, faces form sketches, and extruded sketches form 3D volumes.A face contains one outer loop and optional inner loops as holes.
- Primitive hierarchy: Multiple extruded sketches combine through intersection, union, and subtraction to form a sketch-and-extrude model.This representation can express sketches with multiple faces, unlike the cited DeepCAD representation.
- Construction sequence: Five token types encode curve topology, point geometry, primitive boundaries, extrusion and Boolean parameters, and sequence termination.The sequence includes topology, geometry, end-primitive, extrusion, and end-sequence tokens.
4. SkexGen Architecture
SkexGen learns disentangled topological, geometric, and extrusion variations through separate codebooks and generates sketch-and-extrude sequences autoregressively. Its architecture combines specialized encoders with masked Transformer decoders and vector-quantized code representations.
- Branch structure: SkexGen uses two network branches and three disentangled codebooks for topological, geometric, and extrusion variations.The sketch branch models topological and geometric variation, while the extrude branch models 3D extrusion variation.
- Sequence representation: The construction sequence uses topology, geometry, end-primitive, extrusion, and end-sequence tokens to represent sketch-and-extrude operations.The sample hierarchy shows curves nested within loops and faces, with end-primitive tokens marking their boundaries.
- Topology encoder: The topology encoder represents curve types and primitive boundaries with discrete tokens embedded by a Transformer encoder.Topology tokens identify lines, arcs, or circles; end-primitive and end-sequence tokens delimit the hierarchy.
- Geometry encoder: The geometry encoder discretizes 2D sketch coordinates into a 64 × 64 pixel grid and encodes them with coordinate-aware embeddings.Geometry tokens use a 4101-dimensional one-hot representation, with optional learnable x- and y-coordinate embeddings that improve results.
- Sketch decoder: The sketch decoder reconstructs geometry and end-primitive tokens from topology and geometry codebook vectors using masked autoregressive attention.Topology tokens are inferred from the number of geometry tokens required for each curve type rather than generated directly.
- Training: Vector quantization maps encoder outputs to codebook vectors, while reconstruction, codebook, and commitment losses jointly train the sketch components.The commitment-loss scale is β = 0.25, encouraging encoder outputs to commit to code vectors.
5. Experiments
Experiments evaluate SkexGen’s generation quality, codebook control, disentanglement, and design-exploration applications on sketch and sketch-and-extrude CAD data. SkexGen produces high-quality, diverse, and controllable outputs, while its autoregressive sampling increases runtime relative to DeepCAD.
- Experiment goals: Experiments assess generation quality and diversity, codebook control, disentanglement, and applications including design exploration and interpolation.The study uses the DeepCAD dataset, quantitative metrics, qualitative comparisons, and application demonstrations.
- Random generation: SkexGen achieves the best sketch-generation FID, while its unique and novel percentages are similar to or better than CurveGen, except for novelty relative to DeepCAD.DeepCAD’s higher novelty partly reflects invalid outputs, which FID identifies as far from the ground-truth distribution.
- Random generation: SkexGen sketches contain more complex shapes, fewer self-intersections, and stronger symmetry than the compared generators.CurveGen produces good-quality but simpler arrangements, whereas DeepCAD produces complex but noisier shapes.
- Random generation: SkexGen performs best across all CAD-generation metrics, producing complex, symmetric, arc-rich, and frequently multi-step models.The ablation shows that reducing multiple disentangled codebooks to one lowers generation quality, while removing codebooks performs worst.
- Running time: 90 secs are required to generate 10,000 samples, compared with 15 secs for DeepCAD; SkexGen is nevertheless 2× faster than CurveGen.The runtime disadvantage relative to DeepCAD is attributed to autoregressive sampling, while CurveGen uses two dependent autoregressive decoders.
- Controllable generation and applications: Fixed topology, geometry, or extrusion codes produce corresponding controlled variations, and code-mixing generates hybrid results combining properties from different references.These experiments demonstrate disentanglement and support controlled design exploration through interpolation and code mixing.
6. Conclusion
SkexGen enhances topological and geometric variation in CAD construction sequences to support design control and exploration.
- SkexGen enhances topological and geometric variations in CAD construction sequences.
- The model aims to enable better design control and exploration.
A. Construction Sequence: Full Specification
A valid sketch-and-extrude construction sequence is hierarchically organized from extruded-sketches through sketches, faces, loops, curves, and points, with explicit termination tokens.
- A sketch-and-extrude model consists of multiple extruded-sketches.
- A sketch consists of multiple faces, each face consists of multiple loops, and each loop consists of multiple curves.
- Curves are lines, arcs, or circles represented by 2, 3, or 4 points, respectively.
- Geometry tokens represent points, and end-primitive tokens terminate curves, lines, faces, loops, sketches, and extruded-sketches.
- The first loop defines a face’s external boundary, while remaining loops define internal holes.
B. Extrude Branch Architecture
The extrude branch represents extrusion operations with typed tokens encoding height, rotation, translation, scaling, Boolean operations, and sequence termination.
- The extrude branch uses extrude tokens, an end-primitive token, and an end-sequence token.Extrude tokens have five types: H, R, O, S, and B.
- H encodes two extrusion-plane displacements, R encodes a 3D rotation matrix, and O encodes a 3D translation vector.H and O numeric values use 64-bin quantization, while R entries use three values: -1, 0, or 1.
- S encodes a 2D scaling center and uniform scaling factor, while B encodes intersection, union, or subtraction.
- An extrude subsequence contains 19 individual tokens, each represented by a 72-dimensional one-hot vector.The flattened sequence is [HHRRRRRRRRROOOSSSBE], excluding End.
- The extrude encoder uses 72-dimensional inputs and distinguishes token types with an additional learnable embedding.
- The extrude decoder uses one codebook and generates extrude codes autoregressively.
C. Code Selector: Details
The code selector supplies topology, geometry, and extrude codebooks to the decoders through compressed embeddings and autoregressive code selection.
- The three codebooks provide 10 codes—4 topology, 2 geometry, and 4 extrude—for sketch and extrude decoders.
- Each codebook can contain up to 1,000 codes represented with 1,000-dimensional one-hot encodings.
- A learnable 256 × 1000 matrix compresses code embeddings to 256 dimensions.
- Topology and geometry codes are injected into the decoder through cross-attention, which autoregressively produces the missing extrude codes.
D. Additional Results
SkexGen produces randomly generated sketches, illustrating its sketch-generation capability.
- Figures 9 to 12 show randomly generated sketches produced by SkexGen.
D.2. CAD Generation
The additional results display randomly generated sketches and sketch-and-extrude CAD models produced by SkexGen.
- Together, the figures show SkexGen generating both sketches and sketch-and-extrude CAD models.
- Figures 9 to 12 present randomly generated sketches by SkexGen.
- Figures 13 to 18 present randomly generated CAD models by SkexGen.