PaperlayerView exact revision on arXiv

Source-linked AI summary

CAD-Tokenizer: Towards Text-based CAD Prototyping via Modality-Specific Tokenization

Ruiyu Wang, Shizhao Sun, Weijian Ma, Jiang Bian

arXiv:2509.21150v2cs.LG

TL;DR

Text-based CAD prototyping combines generating designs from instructions with editing existing CAD, but prior work explored these tasks separately. CAD-Tokenizer uses CAD-specific primitive tokens in a unified LLM training setup; the paper reports improved performance across the tasks and more computationally efficient fine-tuning. Its training on more complex shapes is constrained by a quality gap between open-source and private-sector CAD data.

  • Problem

    Prior work explored text-to-CAD generation and CAD editing separately, although engineers iteratively create and modify prototypes.

    Source: §1 · p. 1

  • Method

    CAD-Tokenizer compresses CAD sequences into primitive-level tokens with a VQ-VAE and fine-tunes an LLM on unified text-to-CAD and editing tasks.

    Source: §1 · p. 2

  • Results

    Across CAD editing and Text-to-CAD generation, CAD-Tokenizer improves F1 scores and surpasses task-specific baselines on nearly every generation metric, with a VLM-score exception.

    Source: §1 · p. 2, §4.2 · p. 7

  • Takeaways & Limitations

    Unifying generation and editing targets engineers’ iterative prototype-creation workflow rather than one-shot generation alone.

    Source: §1 · p. 1

    A quality gap between open-source and private-sector CAD data limits training on more complex shapes.

    Source: §5 · p. 10

Abstract

from arXiv · show

Computer-Aided Design (CAD) is a foundational component of industrial prototyping, where models are defined not by raw coordinates but by construction sequences such as sketches and extrusions. This sequential structure enables both efficient prototype initialization and subsequent editing. Text-guided CAD prototyping, which unifies Text-to-CAD generation and CAD editing, has the potential to streamline the entire design pipeline. However, prior work has not explored this setting, largely because standard large language model (LLM) tokenizers decompose CAD sequences into natural-language word pieces, failing to capture primitive-level CAD semantics and hindering attention modules from modeling geometric structure. We conjecture that a multimodal tokenization strategy, aligned with CAD's primitive and structural nature, can provide more effective representations. To this end, we propose CAD-Tokenizer, a framework that represents CAD data with modality-specific tokens using a sequence-based VQ-VAE with primitive-level pooling and constrained decoding. This design produces compact, primitive-aware representations that align with CAD's structural nature. Applied to unified text-guided CAD prototyping, CAD-Tokenizer significantly improves instruction following and generation quality, achieving better quantitative and qualitative performance over both general-purpose LLMs and task-specific baselines.

1 INTRODUCTION

The introduction frames text-guided CAD prototyping as a unified task for creating and modifying designs, and presents CAD-Tokenizer as a CAD-specific tokenization framework for that task. Its central idea is to represent CAD sequences as primitive-level tokens so LLMs can model relationships among CAD primitives.

1 INTRODUCTION

  • CAD’s operation sequences encode design history, making them suitable both for initializing prototypes and for subsequent modifications.CAD kernels compile operations such as sketches and extrusions into machine-interpretable 3D objects.Source: §1 · p. 1
  • A unified text-based CAD system must combine Text-to-CAD generation with precise editing of existing geometry, matching engineers’ iterative creation-and-modification workflow.Prior work explored the subtasks separately, while the introduction identifies their unification as previously unstudied.Source: §1 · p. 1
  • CAD-Tokenizer uses a VQ-VAE to compress CAD sequences into primitive-level tokens, enabling LLMs to model inter-primitive relations and predict operations rather than word pieces.The introduction attributes tokenizer fragmentation to attention focusing on punctuation or partial tokens instead of meaningful primitives.Source: §1 · p. 2
  • The framework combines a primitive-level VQ-VAE, adapters aligning its vocabulary with LLM embeddings, unified fine-tuning, and FSA-based sampling to enforce valid syntax.The introduction reports improvements in efficiency and performance across unified CAD prototyping tasks.Source: §1 · p. 2

2 RELATED WORKS

Prior CAD modeling work spans non-text inputs and separate text-based generation and editing tasks. The related-work section positions CAD-Tokenizer as a unified alternative, using a CAD-specific encoder–decoder tokenizer aligned with an LLM.

2.1 CAD MODELING

  • Classical CAD generation conditions on non-text inputs such as sketches, point clouds, sequence prefixes, or noise, often using separate encoders for different inputs.These works generally do not unify the encoding procedure into one model.Source: §2.1 · pp. 2–3
  • Prior text-based CAD work treats text-only generation and editing a supplied CAD sequence as separate tasks, while this work merges them into one prototyping task.Text-to-CAD work includes Wang et al. (2025a), Li et al. (2024; 2025b), Khan et al. (2024b), Govindarajan et al. (2025), Guan et al. (2025), and Wang et al. (2025b); CAD editing includes Yuan et al. (2025).Source: §2.1 · p. 3

2.2 LLM TOKENIZATION

  • Recent text-based CAD methods typically use off-the-shelf LLM tokenizers, whereas CAD-Tokenizer uses one VQ-VAE encoder–decoder pair for all CAD inputs and aligns its vocabulary with the LLM.The unified tokenization represents CAD primitives as discrete units rather than fragments from natural-language tokenization.Source: §2.2 · p. 3

3 METHODOLOGY

CAD-Tokenizer unifies Text-to-CAD generation and CAD editing by fine-tuning an LLM on CAD-specific tokens from a primitive-based VQ-VAE. Adapters align the tokens with the LLM, and FSA-guided decoding constrains generation to CAD grammar.

3.1 APPROACH OVERVIEW

  • The unified task maps an instruction and optional original CAD sequence to a generated sequence that reflects design intent and, when applicable, the original shape.Source: §3.1 · p. 3

3.2 PRIMITIVE-BASED VQ-VAE AS CAD TOKENIZER

  • Primitive-specific pooling encodes sketch–extrusion pairs into multiple discrete representations, capturing local and contextual information across the sequence.Adapters align VQ-VAE outputs with the LLM embedding space using frozen embedding and logit layers; the alignment yields native LLM-recognizable primitive IDs without modifying the backbone.Source: §3.2 · pp. 4–5

3.3 INSTRUCTION TUNING WITH CAD TOKENS

  • Instruction tuning encodes original and target CAD sequences into compact symbolic forms, then fine-tunes the pretrained LLM with cross-entropy on generated versus ground-truth tokens.The authors state that because the encoded sequences are shorter, fine-tuning is more effective and computationally efficient.Source: §3.3 · p. 5

3.4 MODEL INFERENCE AND TOKEN DECODING

  • At inference, an FSA updates its state and masks candidate logits to restrict generation to grammar-compliant tokens, reducing syntactic errors and improving sampling robustness.The method does not fully capture some context-dependent geometric constraints.Source: §3.4 · p. 5, Figure 3 · p. 6

4 EXPERIMENTS

The experiments evaluate CAD-Tokenizer on text-to-CAD and CAD editing, then ablate tokenizer variants and decoding strategies. It leads on nearly all reported task metrics, while the ablations highlight primitive pooling and FSA-guided sampling.

4.1 EXPERIMENTAL SETUPS

  • Experiments combine CADFusion and CAD-Editor data into approximately 100k training and 1,000 test pairs, balance editing and generation at 5:1, and compare task-specific models, native-tokenizer LLaMA, and five-shot GPT-4o.The evaluation includes sequence, distribution, validity, instruction-following, and human-evaluation measures.Source: §4.1 · pp. 6–7

4.2 MAIN RESULTS

  • CAD-Tokenizer leads on nearly all CAD-editing metrics and nearly every Text-to-CAD metric; editing F1 improves by around 10 points each, while CADFusion scores higher on generation VLM (5.41 vs. 3.82).The results are reported in Table 1 for both tasks.Source: §4.2 · p. 7, Table 1 · p. 7
  • Qualitatively, CAD-Tokenizer preserves multi-face structure in Text-to-CAD and modifies existing shapes to instructions in editing, while Vanilla-LLaMA often drops faces or misaligns structures.Vanilla-LLaMA almost collapses on the majority of quantitative metrics.Source: §4.2 · p. 8, Figure 5 · p. 9

4.3 ABLATION STUDIES

  • All CAD-Tokenizer variants outperform prior methods in reconstruction quality and compression; compressing more tokens per embedding trades reconstruction accuracy, and omitting primitive pooling sharply degrades reconstruction.Table 2 reports reconstruction results; Figure 4 compares compression ratios, with 100% denoting no compression.Source: §4.3 · p. 8, Table 2 · p. 8, Figure 4 · p. 8
  • During LLM fine-tuning, BPE and the unpooled single variant perform poorly, while loop pooling approaches curve pooling with slightly lower reconstruction quality and higher compression.The text attributes BPE's added fine-tuning burden to its tokens not aligning with the LLM vocabulary.Source: §4.3 · pp. 8–9, Table 3 · p. 10
  • FSA-guided sampling consistently outperforms top-p sampling and beam search across all metrics; beam search improves performance over top-p but has a significant invalidity-ratio trade-off.The comparison fixes the backbone to fine-tuned CAD-Tokenizer.Source: §4.3 · p. 10, Table 4 · p. 10

5 LIMITATIONS

The authors identify limitations in the available CAD data, CAD-editing metrics, and pretrained backbones.

5 LIMITATIONS

  • The gap between open-source and private-sector CAD data limits training on more complex shapes.The authors call for better datasets.Source: §5 · p. 10
  • Distributional metrics do not reflect actual CAD-editing performance because they do not evaluate the intent to keep the original shape.The authors anticipate a need for better metrics.Source: §5 · p. 10
  • A failure study suggests that improving spatial or commonsense reasoning requires better pretrained backbones.Source: §5 · p. 10

6 CONCLUSION

The conclusion presents CAD-Tokenizer as a unified framework for text-based CAD prototyping, combining CAD-specific tokenization with LLM fine-tuning and constrained decoding. Experiments validate its design choices and report advantages over task-specific and general-purpose baselines.

6 CONCLUSION

  • Experiments demonstrate CAD-Tokenizer’s advantage over task-specific and general-purpose baselines.The reported validations include primitive-specific pooling within the VQ-VAE and FSA-driven sampling.Source: §6 · p. 10
  • A pretrained transformer-based VQ-VAE converts sequential CAD inputs into primitive tokens rather than vanilla tokenizers’ word pieces.LLM backbones fine-tuned on these CAD-specific tokens showed better performance and training efficiency.Source: §6 · p. 10
  • Finite-state-automaton-driven sampling enforces sequence formatting to improve inference-time decoding quality.Source: §6 · p. 10

ETHICS STATEMENT

The authors state that the data is tailored for CAD generation and that its specialized nature minimizes misuse risk.

ETHICS STATEMENT

REPRODUCIBILITY STATEMENT

The authors report including information on experiments, datasets, model configurations, computational setups, and metrics, with details in Section 4.1 and Appendix C.1.

REPRODUCIBILITY STATEMENT

THE USE OF LARGE LANGUAGE MODELS

The passages define CAD’s sketch–extrusion sequence and the paper’s primitive units, while noting that LLM use was limited to language editing.

THE USE OF LARGE LANGUAGE MODELS

A.1 SEQUENCE REPRESENTATION

  • The Sketch-and-Extrude Modeling (SEM) format represents a CAD object as a series of sketches and extrusions.Sketch primitives include curves and any succeeding end tokens; extrusion primitives are divided into three parts, with R split into three tokens to avoid overly long sequences.Source: §A.1 · p. 14, §A.2 · pp. 14–15

A.2 PRIMITIVE DEFINITION

  • Loop-level primitives balance token count and information content, while a shared subspace and contextualization distinguish this design from separate models for smaller CAD particles.Per-curve tokenization produces too many tokens; per-face or per-sketch tokenization may produce too few correlations or require too much compression.Source: §A.2 · p. 15

B IMPLEMENTATION DETAILS

The implementation defines its encoder terminology, pools contextualized representations for sketch–extrusion pairs, and uses an automaton to constrain CAD token sampling.

PRELIMINARY: TERMINOLOGIES AND NOTATIONS

  • A VQ Encoder includes the transformer encoder, primitive pooling, and VQ layer before token extraction; CAD Encoder combines the VQ Encoder with adapter layers.The VQ Decoder is equivalent to the decoder, while CAD Decoder combines it with its corresponding adapter layers.Source: PRELIMINARY: TERMINOLOGIES AND NOTATIONS · p. 15

B.1 VQ AND PRIMITIVE POOLING

  • Primitive-specific pooling replaces global pooling to generate distinct representations for sketch–extrusion pairs, addressing alignment and contextualization limitations of separate primitive models.Source: §B.1 · p. 16
  • Masks isolate each primitive in the encoder output; max pooling yields its representation, and concatenated representations feed decoder cross-attention and subsequent LLM alignment.Source: §B.1 · p. 16
  • The method allows up to 12 primitives—9 flexible loop slots and 3 fixed extrusion slots—and filters only about 900 of 90,000 sequences for exceeding the limit.The dataset averages approximately four loops per sketch–extrusion pair.Source: §B.1 · p. 16

B.2 FSA SAMPLING

  • FSA sampling restricts each step to permissible next tokens, then updates the automaton state using the last generated token to determine subsequent options.Source: §B.2 · p. 16

C.1 EVALUATION SETUP

Evaluation materials pair human and VLM judgments with additional editing analyses that highlight limitations of distributional metrics and recurring failure categories.

C.1 EVALUATION SETUP

  • Five human judges evaluated 50 generation outputs—40 from editing tasks and 10 from Text-to-CAD tasks—and received task instructions and visual examples without model identifiers.Source: §C.1 · p. 17

C.2.1 ELABORATION ON CAD EDITING RESULTS ON THE DISTRIBUTIONAL METRICS

  • Original inputs can score highly on distributional measures despite being failed edits, while GPT-4o outputs stay closer to inputs than CAD-Tokenizer outputs; VLM scores better reflect edit attempts.The authors advise interpreting distributional metrics cautiously and considering all metrics, especially qualitative ones.Source: §C.2.1 · p. 18

C.2.2 ADDITIONAL QUALITATIVE RESULTS

  • Additional qualitative results are provided for CAD editing and Text-to-CAD generation.Source: §C.2.2 · p. 18

C.2.3 FAILURE CASES

  • The failure study groups errors into overcomplicated entries, misleading instructions, and weak spatial or commonsense reasoning.Source: §C.2.3 · p. 18
  • Complicated instructions can make correct shapes difficult to generate, such as describing a 4-like extrusion as “four polyhedra” without mentioning its resemblance to the number.Source: §C.2.3 · p. 18
  • Instructions may misrepresent the relationship between original and target shapes; CAD-Editor prompts generated by VLMs can pose instruction-following challenges.Source: §C.2.3 · p. 19
  • Concrete shapes such as keys or letters, and arranging multiple objects, remain challenging amid likely limitations in the backbone LLM’s reasoning.Source: §C.2.3 · p. 19
Loading 2509.21150v2…