Source-linked AI summary
Hierarchical Neural Coding for Controllable CAD Model Generation
Xiang Xu, Pradeep Kumar Jayaraman, Joseph G. Lambourne, Karl D. D. Willis, Yasutaka Furukawa
TL;DR
Existing machine-learning CAD work does not exploit CAD’s hierarchy for effective design control, despite the need to preserve design intent during edits. This paper uses hierarchical neural codes to control CAD generation and completion. It reports improved random-generation quality and geometry control on conditional-generation tasks.
Problem
Existing machine-learning CAD works do not use CAD’s hierarchical nature for effective design control, although design intent matters when models are edited.
Source: §1 · pp. 1–2Method
A generative network represents design intent as a three-level neural-code tree and uses it or an incomplete CAD model to control model generation or completion.
Source: §1 · p. 2Results
The system generates more realistic and complicated models in random generation and demonstrates superior geometry control in user-controlled conditional generation over SkexGen (Xu et al., 2022) and DeepCAD (Wu et al., 2021).
Source: §1 · p. 2Takeaways & Limitations
The authors conclude that hierarchical neural coding supports generation quality and applications including intent-aware editing and auto-completion.
Source: §8 · p. 9Generated CAD models can be invalid, and the sketch-and-extrude format excludes operations such as revolve, mirror, and sweep.
Source: §7 · p. 9
Abstract
from arXiv · showhide
This paper presents a novel generative model for Computer Aided Design (CAD) that 1) represents high-level design concepts of a CAD model as a three-level hierarchical tree of neural codes, from global part arrangement down to local curve geometry; and 2) controls the generation or completion of CAD models by specifying the target design using a code tree. Concretely, a novel variant of a vector quantized VAE with "masked skip connection" extracts design variations as neural codebooks at three levels. Two-stage cascaded auto-regressive transformers learn to generate code trees from incomplete CAD models and then complete CAD models following the intended design. Extensive experiments demonstrate superior performance on conventional tasks such as random generation while enabling novel interaction capabilities on conditional generation tasks. The code is available at https://github.com/samxuxiang/hnc-cad.
1. Introduction
Existing machine-learning CAD generators do not exploit CAD’s hierarchy for effective design control. The paper introduces a three-level neural code tree for controllable generation and completion, reporting stronger results on random generation and user-controlled conditional tasks.
1. Introduction
- Existing machine-learning CAD works do not use CAD’s hierarchical structure to provide effective design control, while parametric CAD can break under unanticipated edits.The paper identifies specialized skills as a requirement for manually specifying parametric constraints.Source: §1 · p. 2
- The model represents design intent as a three-level tree of neural codes, from local geometric features to global part arrangement, and controls generation or completion using that tree or an incomplete model.Generated modeling-operation sequences are converted to editable B-Rep format for mechanical CAD software.Source: §1 · p. 2
- A masked-skip-connection VQ-VAE learns three neural codebooks, and cascaded autoregressive transformers generate code trees from incomplete CAD models and then complete the models.Designers can also directly provide a code tree for generation.Source: §1 · p. 2
- The system generates more realistic and complicated models in random generation and provides superior geometry control in user-controlled conditional generation over SkexGen (Xu et al., 2022) and DeepCAD (Wu et al., 2021).The paper also reports state-of-the-art performance over previous methods.Source: §1 · p. 2
- The neural code tree is presented as enabling high-quality, complex models, design-intent-aware editing, and design auto-completion.The authors also identify their masked-skip-connection VQ-VAE as a contribution to enhanced codebook learning.Source: §1 · p. 2
2. Related Work
Prior CAD-generation work uses several representations and control inputs, but the paper distinguishes its approach by targeting hierarchical control at both global and local levels. It focuses on controllable parametric CAD built from sketch-and-extrude sequences.
2. Related Work
- CSG represents complex shapes as Boolean combinations of simple primitives, while parametric CAD builds and extrudes 2D sketches and is described as dominant for mechanical-part design and easy parametric editing.CSG trees can be converted to B-Rep using equivalent primitives and Boolean operations.Source: §2 · p. 2
- Direct CAD-generation methods construct curves, surfaces, sketches, or solid geometry, whereas this work focuses on controllable parametric CAD in sketch-and-extrude sequences.The cited methods include fixed or arbitrary topology for sketches and solid models.Source: §2 · p. 2
- Learning-based sketch-and-extrude methods use modeling-sequence history or sketch constraints, and generation can be conditioned on inputs such as B-reps, sketches, images, voxels, or point clouds.The paper characterizes this prior control as global, in contrast to its aim of hierarchical global and local control.Source: §2 · pp. 2–3
- Sketch2CAD and Free2CAD map sketch strokes or sketch groups to CAD operations, while SkexGen provides disentangled global topology and geometry control but is not designed for interactive editing or autocomplete.The paper positions its use of CAD’s natural hierarchy as providing both global and local control.Source: §2 · p. 3
3. Hierarchical CAD Properties
CAD models are represented as a Solid–Profile–Loop hierarchy, from the arrangement of extruded profiles down to curve geometry. This structure supports local and global control by encoding the relative positions of geometric entities at higher levels.
3. Hierarchical CAD Properties
- A Solid–Profile–Loop tree represents each model from the arrangement of extruded profiles down to profile and loop geometry.Higher levels encode the relative positions of lower-level geometric entities.Source: §3 · p. 3
- Each leaf loop contains lines, arcs, or a circle, represented by x-y coordinates and separator tokens.Curves are ordered from the smallest starting point coordinate, then counterclockwise through connected curves.Source: §3 · p. 3
- Profile nodes encode their loops using 2D bounding boxes, sorted by their bottom-left corners in ascending order.Each box records its bottom-left corner and width and height.Source: §3 · p. 3
- Solid nodes encode the arrangement of extruded profiles using 3D bounding boxes, sorted by their bottom-left corners.Each box records its bottom-left corner and three dimensions.Source: §3 · p. 3
- The illustrated pipeline encodes a construction sequence, generates hierarchical codes from partial-model embeddings, then generates a CAD model.The VQ-VAE encoder and decoder use a masked input sequence during codebook learning.Source: §3 · p. 4, Figure 4 · p. 4
4. Three-Level Codebook Learning
The method learns separate Loop, Profile, and Solid codebooks from sketch-and-extrude CAD models. A masked skip connection encourages codes to capture design patterns rather than instance-specific details or copied input.
4. Three-Level Codebook Learning
- A VQ-VAE learns three discrete codebooks that encode CAD models as hierarchical neural codes for downstream applications.Loop, Profile, and Solid codebooks are learned independently.Source: §4 · p. 4
- Masking the encoder-to-decoder skip connection prevents direct copying and makes the decoder infer missing details using design patterns encoded in the code.A standard VQ-VAE focuses on instance-specific details, while an unmasked skip connection can let the decoder copy the input.Source: §4 · p. 4
- The encoder pools token embeddings and uses nearest-neighbor vector quantization to select a 256D codebook vector.Loop coordinates use 65D one-hot tokens; profile and solid bounding-box parameters are processed similarly, without separator tokens.Source: §4 · p. 4
- The decoder predicts masked tokens, using squared Earth Mover’s Distance on ordinal coordinates and cross-entropy on separator tokens.The loss is applied only to masked tokens; codebook and commitment losses complete the training objective.Source: §4 · pp. 4–5
5. Controllable CAD Generation
For an incomplete CAD model, the system encodes the input, generates a hierarchical code tree, and then generates a complete model conditioned on both. The code tree assigns codes to Solid, Profile, and Loop entities to support design-intent control.
5. Controllable CAD Generation
- Given an incomplete CAD model, a model encoder produces latent embeddings, a first autoregressive Transformer generates a code tree, and a second generates the full model.The final model is conditioned on both the embedded input and the code tree.Source: §5 · p. 5
- The model encoder converts CAD command tokens into 256D latent embeddings, while the hierarchical codes and partial-model embeddings control generation through cross-attention.For 2D sketch experiments, only the geometry encoder is used.Source: §5 · p. 5
- The code tree assigns a code to each Solid, Profile, or Loop and represents the hierarchy in depth-first order, with separators marking new groupings.The example sequence is [S, <SEP>, P, L, L, <SEP>, P, L, L, L, L].Source: §5 · p. 5
- For unconditional generation, the code-tree generator omits the partial-model encoder and cross-attention layers, using query tokens alone.The model generator otherwise follows the SkexGen decoder, with partial-model embeddings and hierarchical codes providing control.Source: §5 · p. 5
6. Evaluation
The evaluation reports stronger unconditional-generation metrics, realism judgments, and qualitative complexity, alongside code-tree editing and completion controls. It also examines whether codes capture instance-agnostic design patterns.
6.1. Experiment Setup
- After filtering DeepCAD, training includes 124,451 CAD sequences and separate codebook-learning sets of 102,114 solids, 60,584 profiles, and 150,158 loops.Selected codebook sizes were approximately 3,500 for profiles and solids and 2,500 for loops.Source: §6.1 · p. 6
6.2. Metrics
- Evaluation uses point-cloud COV, MMD, and JSD, plus token-level Unique and Novel measures.COV measures ground-truth coverage; MMD reports minimum matching distance, while JSD compares point-cloud occupancy distributions.Source: §6.2 · pp. 6–7
6. Evaluation
- Our method outperforms both baselines on all three point-cloud metrics; Unique matches SkexGen, while Novel is slightly worse than SkexGen but better than DeepCAD.Table 1 reports averages across three runs.Source: §6.3 · p. 7, Table 1 · p. 7
6.3. Unconditional Generation
- For complex samples, 49.2% from our method were judged more realistic than training examples, versus 46.9% for SkexGen and 38.7% for DeepCAD.Qualitative comparisons show more complicated geometries and part arrangements in our generated solids.Source: Figure 5 · p. 6, §6.3 · pp. 7–8, Figure 6 · p. 7
6.4. Controllable Generation
- Hierarchical codes support loop-, profile-, and solid-level edits, preserve design relations during parameter edits, and guide completion from partial loops or extruded profiles.Against nearest-neighbor retrieval, generated completions better match user input with higher diversity.Source: §6.4 · pp. 8–9, Figure 11 · p. 9
6.5. Instance-Agnostic Design Pattern
- Codes group instance-agnostic design patterns while ignoring details such as exact curve count or type; Figure 12 shows loops and profiles sharing codes.Source: §6.5 · p. 9
7. Limitations
The system can generate invalid geometry, and its representation is limited to sketch-and-extrude CAD operations.
7. Limitations
- Generated CAD models may contain invalid self-intersecting edges or solids.Source: §7 · p. 9
- Current losses do not explicitly penalize invalid geometry, and recovery is challenging without an invalid-CAD dataset.Source: §7 · p. 9
- The sketch-and-extrude format excludes operations such as revolve, mirror, and sweep.Source: §7 · p. 9
8. Conclusion
The paper presents hierarchical neural coding for controllable CAD generation and reports evaluations supporting generation quality and user-in-the-loop applications. Human evaluation used crowd judgments of generated-model realism.
8. Conclusion
- Extensive evaluations demonstrate major boosts in generation quality and promising applications in intent-aware editing and auto-completion.The conclusion identifies three-level neural coding as capturing design patterns and intent at different modeling levels.Source: §8 · p. 9
- The paper frames hierarchical neural coding as a step toward intelligent generative design with users in the loop.The conclusion names design patterns and intent across different levels of the modeling hierarchy as a key part of the approach.Source: §8 · p. 9
A. Human Evaluation Details
- Human evaluation collected 19,950 crowd-worker annotations across 950 image pairs per generative method, with seven workers rating each pair.Workers judged whether a generated model was more realistic than a random model from training data.Source: A. Human Evaluation Details · p. 13
B. Additional CAD Results
The additional CAD results show multiple completions for partial inputs and further randomly generated models.
B. Additional CAD Results
- Each partial input is followed by two sketch-and-extrude completion sequences, reflecting different predicted code trees.The examples show construction steps from left to right, with user-provided extruded profiles colored orange.Source: B. Additional CAD Results · p. 13
- Figures 15 and 16 provide additional randomly generated CAD models from the method; Figure 14, by contrast, presents detailed sketch-and-extrude construction sequences shown from left to right.Source: B. Additional CAD Results · p. 13
C. Sketch Generation Results
Randomly generated sketches achieve a similar FID to SkexGen and a slightly better Novel score, while the section also presents sketch autocompletion from partial loops and curves.
C. Sketch Generation Results
- FID measures differences in mean and covariance between real and generated sketch data in a network feature space.The evaluation uses ResNet-18 pretrained on human sketch classification to extract features.Source: C. Sketch Generation Results · p. 13
- The random-sketch results report a similar FID to SkexGen and a slightly better Novel score; the table covers unconditional sketch generation.The accompanying samples are described as having complex shapes with few self-intersections or broken curves.Source: C. Sketch Generation Results · p. 13, Table 2 · p. 14
- The authors report additional sketch-autocompletion results in Figure 18 and say they trained their model to autocomplete a full sketch from partial curves, a more general user-interaction case.Source: C. Sketch Generation Results · p. 13
D. Additional Code and Data Mapping
The additional results visualize random sketches and CAD models, autocompleted sketches, and profile and loop data mapped to shared codes. Shared codes correspond to similar design patterns that omit instance-specific details.
D. Additional Code and Data Mapping
- Profile and loop data mapped to the same code share design patterns, such as circular layouts around a center or X-like geometry, while ignoring instance-specific details.Those details include the number of bounding boxes, curves, and curve types.Source: D. Additional Code and Data Mapping · p. 14, Figure 19 · p. 19, Figure 20 · p. 20
- The additional samples include randomly generated CAD models, models with at least three sketch-extrude steps, and randomly generated sketches.The passages identify these as Figures 15, 16, and 17, respectively.Source: Figure 15 · p. 15, Figure 16 · p. 16, Figure 17 · p. 17
- Figure 18 shows sketch autocompletions from partial loops and partial curves.Columns 2–4 show completions from the partial loop in column 1; columns 6–8 show completions from the partial curve in column 5.Source: Figure 18 · p. 18