PaperlayerView exact revision on arXiv

Source-linked AI summary

BRepCLIP: Contrastive Multimodal Pretraining on BRep Primitives for CAD Understanding

Muhammad Usama, Didier Stricker, Mohammad Sadil Khan, Muhammad Zeshan Afzal

arXiv:2606.05515v1cs.CV

TL;DR

CAD representation learning has focused heavily on point clouds and meshes, leaving native BRep geometry and its exact topology underused for multimodal understanding. BRepCLIP directly tokenizes BRep faces and edges, aligns their transformer embedding with frozen CLIP text and image encoders, and reports stronger retrieval and classification performance alongside a structure-aware generation metric. Its fixed geometric resolution and semantic taxonomy limit coverage of highly detailed or diverse engineering models.

  • Problem

    Native BRep geometry preserves exact parametric surfaces, curves, and topology, yet general-purpose multimodal representations for open-vocabulary CAD understanding remain limited.

  • Method

    BRepCLIP tokenizes face and edge primitives with separate surface and curve vocabularies, aggregates them with a transformer, and contrastively aligns the resulting embedding with frozen CLIP text and image encoders.

  • Results

    BRepCLIP consistently outperforms point-based and multimodal baselines on text-to-CAD retrieval and zero-shot CAD classification, while BRepCLIP-Score correlates more reliably with human judgments for CAD-generation evaluation.

  • Takeaways & Limitations

    Native BRep structure provides a foundation for multimodal CAD retrieval, generation evaluation, and broader engineering design workflows.

  • Takeaways & Limitations

    Fixed geometric resolution may miss finer local detail, and the fixed semantic taxonomy does not cover the full diversity of real-world engineering primitives and topology.

Abstract

from arXiv · show

Learning representations of CAD models is a largely open problem. While 3D representation learning has flourished around point clouds and meshes, the native format of CAD - boundary representations BReps, which encodes exact parametric surfaces, curves, and their topology, has received little attention as a representation learning substrate. We introduce BRepCLIP, the first framework to align BRep geometry with language and image embeddings through contrastive pretraining. We model each CAD object as a sequence of face and edge tokens with separate discrete vocabularies for surface and curve geometry, augmented with spatial and semantic descriptors that capture surface types (e.g., cylindrical, torus, NURBS) and curve primitives (e.g., line, arc, B-spline). A transformer encoder aggregates these tokens into a global BRep embedding, aligned with CLIP's text and image encoders via a joint contrastive objective. BRepCLIP generates more discriminative and semantically grounded embeddings than existing point-based alternatives, improving Top-1 retrieval over OpenShape by 40.4%, 22.0%, and 23.9% on ABC, CADParser, and Automate, respectively, and improving zero-shot classification on FabWave by 15% in Top-1 score. We further demonstrate its utility as a CAD-aware similarity metric for evaluating text and image-conditioned CAD generation, establishing the importance of structure-aware pretraining for multimodal CAD understanding. Project page is available at https://muhammadusama100.github.io/BrepClip2026/

1 Introduction

BRepCLIP addresses the open problem of learning general-purpose, open-vocabulary CAD representations by operating directly on structured BRep primitives and aligning them with language and images. Its structure-aware design targets fine-grained CAD understanding and supports retrieval, classification, and generation evaluation.

  • Motivation: BRep models encode exact parametric surfaces, curves, and topological adjacencies, but general-purpose representations for open-vocabulary CAD retrieval remain limited.CAD retrieval supports part reuse, design search, and manufacturing consistency.
  • Motivation: Point-cloud conversion discards analytic surface types, curve primitives, and topology, obscuring engineering-critical details such as holes, fillets, chamfers, and adjacency.This loss can make cylindrical bores and planar pockets, or filleted and sharp edges, difficult to distinguish.
  • Method: BRepCLIP operates directly on BRep face and edge primitives, learning separate discrete vocabularies for surface and curve geometry before transformer aggregation.The representation combines sampled local geometry, semantic type, and topological grouping.
  • Method: BRepCLIP aligns its BRep embedding with frozen CLIP text and image encoders through a symmetric contrastive objective.The framework bridges native CAD geometry with language and image modalities.
  • Results: BRepCLIP outperforms point-based baselines on text-to-CAD retrieval and zero-shot CAD classification across the reported benchmarks.The introduction reports state-of-the-art results for both tasks without supplying the individual retrieval values in these passages.
  • Applications: BRepCLIP-Score provides a CAD-aware similarity metric for evaluating text- and image-conditioned CAD generation against human expert judgments.It is presented as a structure-sensitive alternative to generic similarity measures.

2 Related Work

Prior 3D multimodal methods primarily align point clouds with language and images, while BRep methods largely focus on recognition or structural pretraining. BRepCLIP occupies the missing intersection by aligning native BRep primitives with language and image modalities and by supporting structure-aware generation evaluation.

  • 3D representation learning and CAD: Point-based multimodal methods achieve strong generic 3D understanding but lose CAD-specific topology and parametric detail when BReps are converted to point clouds.Engineering retrieval requires distinguishing globally similar parts through details such as holes, chamfers, and fillets.
  • 3D representation learning and CAD: Native BRep methods model typed faces, typed edges, and explicit connectivity for recognition, segmentation, reconstruction, or structural pretraining.Examples include UV-Net, BRepNet, BRep-BERT, BRT, MultiCAD, and BrepCoder.
  • 3D representation learning and CAD: Existing BRep methods do not learn language- or image-aligned representations over native BRep primitives for open-vocabulary retrieval.This distinguishes BRepCLIP from prior within-CAD recognition and structural-pretraining approaches.
  • CAD retrieval, generation, and evaluation: Learning-based CAD retrieval has largely used shape signatures, voxel descriptors, or rendered silhouettes that do not capture BRep topology and parametric richness.Earlier scan-to-CAD and contrastive approaches target scene-level alignment rather than language-driven engineering retrieval.
  • CAD retrieval, generation, and evaluation: BRepCLIP-Score addresses structure-blind generation evaluation by grounding similarity in BRep embeddings and validating it against human judgments on outputs from six recent text-to-CAD models.Chamfer Distance and CLIP score are described as unable to reflect errors in topology, holes, or edge types.

3 BRepCLIP Architecture

BRepCLIP represents CAD models as structured face-edge token sequences and aligns their global BRep embeddings with frozen CLIP text and image embeddings. Separate tokenizers, geometric descriptors, and dual contrastive objectives preserve modality-specific structure while enabling multimodal alignment.

  • 3.1 Hybrid Face-Edge Tokenization: BRepCLIP represents CAD models with separate face and edge point sets, tokenizers, and dedicated discrete tokens for surface and curve geometry.The resulting face-edge sequence is enriched with primitive-type semantics before transformer encoding.
  • 3.1 Hybrid Face-Edge Tokenization: The tokenization objective combines Chamfer Distance between reconstructed and ground-truth geometry with KL divergence regularization for either face or edge primitives.The same objective is applied independently to the face and edge branches.
  • 3.2 BRep Encoder: A unified sequence concatenates a learnable [CLS] token with face and edge tokens augmented by modality, spatial, and semantic descriptors.The transformer uses the final [CLS] representation as the global BRep embedding.
  • 3.3 Multimodal Contrastive Alignment: BRepCLIP trains only the BRep branch against frozen CLIP text and image encoders using symmetric BRep-text and BRep-image InfoNCE losses.The BRep encoder produces a projected global embedding, while text and image embeddings are mapped into the same latent space.
  • 3.3 Multimodal Contrastive Alignment: The multimodal design keeps alignment compatible with retrieval pipelines while grounding BRep representations in language semantics and visual appearance.The framework uses both caption and multi-view image embeddings as alignment targets.

4 Experiments

BRepCLIP is evaluated across retrieval, zero-shot classification, and CAD-generation evaluation, using native BRep representations alongside multimodal supervision. It consistently improves retrieval and classification while producing a structure-aware similarity score that better reflects prompt faithfulness.

  • Experimental Setup: BRepCLIP is trained on 400K ABC CAD models with captions and multiview renderings, using structured BReps, point clouds, and images for evaluation.The training setup holds out 10K samples for validation and evaluates retrieval on unseen datasets and gallery instances.
  • Text-to-CAD Retrieval: 40.4% higher Top-1 accuracy on ABC improves from 6.12 to 8.59 over OpenShape, while Chamfer Distance decreases from 0.071 to 0.058.BRepCLIP also achieves the strongest reported Top-k retrieval performance across ABC, CADParser, and Automate.
  • Text-to-CAD Retrieval: 22.0% and 23.9% Top-1 improvements over OpenShape are achieved on CADParser and Automate, respectively.On CADParser, BRepCLIP reaches 5.00 Top-1; on Automate, it reaches 9.42 Top-1 while lowering Chamfer Distance to 0.060.
  • Zero-Shot Classification: 38.62 Top-1, 70.28 Top-5, and 86.71 Top-10 accuracy make BRepCLIP the strongest overall zero-shot classifier on FabWave.The model transfers from ABC to FabWave without fine-tuning by matching CAD embeddings to class-level text descriptors.
  • BRepCLIP-Score: 17.71% and 104.17% score drops under mild and full prompt corruption exceed CLIP score and LongCLIP, indicating greater sensitivity to semantic mismatch.The corresponding drops are 2.78% and 25.00% for CLIP score, and 4.54% and 18.18% for LongCLIP.
  • Ablation Study: Jointly encoding faces and edges is essential: face-only and edge-only variants reduce Top-1 retrieval by 60.4% and 85.3%, respectively.The ablation supports complementary contributions from surface and boundary geometry.

5 Limitation

BRepCLIP has two main limitations: fixed geometric resolution and a fixed semantic taxonomy, both of which constrain coverage of complex real-world CAD data.

  • Fixed geometric resolution may miss fine local detail or dense primitive counts in complex CAD models.This can increase memory and compute requirements at scale.
  • A fixed taxonomy of face and edge types does not cover the full diversity of primitives and topology in real-world engineering data.The authors identify extending semantic vocabulary as future work.

6 Conclusion

The conclusion presents BRepCLIP as a multimodal pretraining framework that preserves native CAD structure through separate face and edge vocabularies. Its embeddings support stronger multimodal understanding and CAD-aware generation evaluation, motivating broader BRep-native models.

  • BRepCLIP models faces and edges as distinct entities, learns separate surface and curve vocabularies, and aligns BRep representations with text and image embeddings.This design is presented as preserving CAD semantics typically lost in point-based representations.
  • BRepCLIP consistently outperforms generic point-based encoders and strong multimodal baselines on zero-shot text-to-CAD retrieval and CAD classification.The conclusion states this pattern across both evaluation tasks.
  • BRepCLIP-Score supports CAD-aware evaluation of text- and image-conditioned generation as a structure-sensitive alternative to image-based similarity metrics.The metric is positioned for retrieval, evaluation, and broader engineering design workflows.

A Dataset Analysis

The dataset analysis describes the training and evaluation splits used to assess BRepCLIP across retrieval and zero-shot classification.

  • The training data uses 400K CAD models from the high-quality ABC subset of CADCap-1M, with 10K additional models for validation.
  • Retrieval uses a held-out ABC split with 91K samples and two unseen datasets, Automate and CADParser, for zero-shot evaluation.
  • FabWave is used for zero-shot classification, with an original manifest containing 45 categories.

A.1 Training Data Statistics

The 400K ABC training set spans mostly simple or moderately complex CAD models alongside a long tail of highly structured geometry. Its local face boundaries are generally compact, reflecting regular engineering surfaces.

  • Figure 7 summarizes distributions of edges per model, faces per model, and average edges per face in the 400K ABC training set.
  • Faces are right-skewed: the mean is 47.8 per model, the median is 27.0, and the 95th percentile is 165.0.Most models are simple or medium-complexity parts, while a smaller subset has rich surface decomposition.
  • Average edges per face is concentrated around 2.5, with mean and median 2.5 and a 95th percentile of 3.0.This reflects the predominance of regular planar, cylindrical, and smoothly connected analytic patches.
  • The training split spans broad CAD complexity, exposing BRepCLIP to regular mechanical primitives and harder long-tail geometries.

A.2 Evaluation Split Overview

The evaluation separates training, in-domain retrieval, and zero-shot transfer across distinct CAD datasets. FabWave is reserved as an unseen benchmark for category-level recognition.

  • 400K ABC models form the main training set, while a held-out 91K-model ABC split measures in-domain retrieval.
  • 65K Automate and 40K CADParser models provide zero-shot retrieval-transfer evaluations on unseen datasets.
  • Figure 8 summarizes the data usage across training and evaluation splits.
  • FabWave contains 4,378 valid samples across 39 categories and is never used during training, making it a strict zero-shot classification benchmark.

A.3 Primitive Type Statistics

The ABC training set is dominated by analytic face and edge primitives, especially planes, cylinders, lines, and circles. Edge relations likewise favor convex and smooth structures, supporting primitive-aware BRep modeling.

  • Semantic descriptors: The extraction pipeline assigns semantic labels to six face types, five edge types, and four edge relation attributes.
  • Face and edge primitive distributions: 61.3% of face primitives are planes and 28.9% are cylinders, while torus, cone, rational NURBS, and sphere faces are less frequent.
  • Face and edge primitive distributions: 58.4% of edge primitives are lines, followed by circles at 22.2% and non-rational B-splines at 17.4%.
  • Edge relation attributes: Convex edges comprise 53.1% of edge relations, followed by smooth edges at 23.1%, concave edges at 18.0%, and closed edges at 5.8%.
  • Implications for BRep modeling: These distributions support primitive-aware tokenization and semantic descriptors because face and edge types provide structural cues beyond raw point samples.
  • Qualitative results: Additional qualitative figures report text-to-CAD retrieval, BRepCLIP-Score, and zero-shot classification results.
Loading 2606.05515v1…