PaperlayerView exact revision on arXiv

Source-linked AI summary

CAD-Native Transformer Operators for AI-Aided Engineering

Daniel Leibovici, Nikola Borislavov Kovachki, Dawon Ahn, Ruben Ohana, Ira J. S. Shokar, Abouzar Ghasemi, Semih Akkurt, Rishikesh Ranade, Neil Ashton, Jan Kautz, Jean Kossaifi

arXiv:2609.36806v1cs.AI

TL;DR

Most neural surrogates use sampled geometry rather than the native CAD representation, while simulation pipelines mesh continuous CAD geometry. CANTO tokenizes NURBS patches directly to predict continuous physical fields, achieving state-of-the-art accuracy on most evaluated surface and volume tasks and supporting inverse design.

  • Problem

    Most neural surrogates use sampled geometric proxies, leaving a representation gap between exact CAD designs and the inputs used for analysis.

    Source: §1 · pp. 1–2

  • Method

    CANTO represents CAD geometry as NURBS patch tokens and uses them to condition a transformer neural operator that predicts physical fields.

    Source: §1 · p. 2

  • Results

    CANTO achieves state-of-the-art accuracy on most evaluated surface and volume prediction tasks across the four benchmarks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    CANTO's differentiability with respect to NURBS inputs enables gradient-based shape optimization and inverse design in CAD parameter space.

    Source: §6 · p. 10

    The method approximately decomposes trimmed surfaces into untrimmed tensor-product patches and assumes each patch has a suitable C1 parametrization.

    Source: §3 · p. 4, §4.2 · p. 5

Abstract

from arXiv · show

Modern engineering systems, from automobiles to aircraft, are designed by using precise, continuous parametric computer-aided design (CAD) models. Evaluating design changes through numerical simulation requires meshing the continuous geometry, a computationally expensive and often brittle process that can require manual intervention and replaces the continuous representation with a discrete approximation. Most neural surrogates accelerate the simulation, but inherit this representation gap by relying on meshes, point clouds, voxels, or other sampled approximations of geometry. We introduce CANTO, a transformer neural operator that maps directly from continuous CAD geometry to physical fields, without meshing the input geometry. We develop a theoretical framework for learning operators from geometric manifolds to function spaces of physical fields, representing geometry through sequences of parametric patches. CANTO instantiates this framework by directly tokenizing non-uniform rational B-spline (NURBS) patches from their control points, knot vectors, and weights, and predicts continuous surface and volume fields at arbitrary query locations. We evaluate CANTO on four automotive and aircraft aerodynamics industry benchmarks: AhmedML, WindsorML, DrivAerML, and HiLiftAeroML. CANTO achieves state-of-the-art accuracy on most evaluated surface and volume prediction tasks, including a 19.8% reduction in surface-pressure relative $L_2$ error compared with AB-UPT on HiLiftAeroML. Differentiability with respect to CAD parameters further enables gradient-based inverse design of designs. On AhmedML, CANTO identifies designs with 4.4 to 20.4% lower drag than the best dataset designs satisfying the same volume and lift constraints, with the improvements verified using the same CFD setup used to generate the original dataset.

1. Introduction

Engineering design repeatedly pairs changes to parametric CAD models with numerical simulation, but the mesh-based analysis pipeline separates exact CAD geometry from the representation used by solvers and most learned surrogates. CANTO instead tokenizes native NURBS geometry and predicts physical fields, with strong results across the evaluated aerodynamics benchmarks.

The representation gap

  • The representation gap arises because CFD and most learned surrogates use tessellated or volumetric meshes instead of exact, continuous CAD geometry.Industrial meshing can require hours of manual cleaning, repair, and refinement, bottlenecking iterative design and analysis.Source: §1 · pp. 1–2

Our approach

  • CANTO directly encodes native parametric NURBS surfaces and predicts physical fields as continuous functions queryable at arbitrary spatial locations.Its transformer combines patch tokens into a global geometry context for surface, volume, and engineering-quantity prediction.Source: §1 · p. 2

Framework

  • The paper formalizes CAD-to-physics surrogate modeling as a map from compact embedded C1 manifolds, represented by oriented boundary patches, to a Banach space of physical fields.This framework provides a foundation for tokenizing parametric geometries.Source: §1 · p. 2

Architecture

  • CANTO constructs geometry and surface tokens from NURBS control points, weights, and knot vectors, avoiding mesh-, voxel-, or point-cloud-based input discretization.The geometry encoding depends on the compact CAD description rather than a discretized mesh approximation.Source: §1 · p. 2

Empirical results

  • CANTO achieves state-of-the-art surface-field accuracy on AhmedML, WindsorML, and HiLiftAeroML, and competitive surface-field plus state-of-the-art volume-field accuracy on DrivaerML.It also enables inverse design, identifying AhmedML configurations with lower drag than any original dataset design under the considered constraints.Source: §1 · p. 2

2. Related works

Prior neural operators and engineering surrogates have improved geometric flexibility and scalability, but generally encode discretized proxies rather than native CAD. CAD-learning methods preserve parametric structure, yet primarily target geometry tasks rather than physical-field prediction.

Geometry-aware PDE and CFD surrogates

  • Neural operators and engineering surrogates model complex domains, but their geometry encoders still use discretized approximations such as meshes, point clouds, or implicit grids.Even methods that query output fields continuously may discretize input geometry before it reaches the model.Source: §2 · p. 3

CAD representations for geometry learning

  • CAD-learning methods preserve CAD structure but primarily target geometric representation learning, generation, or reconstruction rather than physical-field prediction.Examples include UV-Net, DeepCAD, NURBS-Diff, NeuroNURBS, and NURBSGen.Source: §2 · p. 3

Isogeometric analysis

  • Isogeometric analysis uses spline representations for geometry and numerical analysis, whereas CANTO learns from simulation data to predict fields for unseen configurations without a new numerical solve at inference.IGA remains a numerical PDE discretization method and solves the governing equations for each new configuration.Source: §2 · p. 3

3. Preliminaries

The preliminaries describe how NURBS curves and surfaces are built from spline bases, knots, control points, and weights. They also state the surface assumptions and explain how trimmed surfaces are converted into untrimmed patches.

NURBS parameterization

  • NURBS curves and surfaces use B-spline basis functions constructed through the Cox–de Boor recursion and specified using knot vectors.Surface definitions add a second knot vector and a tensor-product grid of control points with non-negative weights.Source: §3 · pp. 3–4

Surface assumptions and normals

  • The considered NURBS surfaces are non-intersecting, so their parameterizations are injective.The surface normal is computed from the cross product of the partial derivatives, with orientation determined by the parameterization and control-point ordering.Source: §3 · p. 4

Trimming

  • Trimming restricts a NURBS surface's parameter domain to represent complex boundaries and holes, and this work approximately decomposes trimmed surfaces into untrimmed tensor-product NURBS patches.Each point on the patch is assumed to correspond uniquely to a point in the trimmed parameter domain.Source: §3 · p. 4

4. Methodology: CAD-Native Transformer Operators

The methodology frames physical prediction as operator learning from geometric manifolds to function spaces, then represents CAD boundaries as sequences of parametrized patches. CANTO instantiates the patches with NURBS descriptions and uses their tokens to condition a transformer for continuous field prediction.

4.1. A framework for manifold operator learning

  • The framework treats physical prediction as approximating maps from a relevant class of manifolds to function spaces, with CFD domain-to-solution prediction as its motivating case.It can represent the solid through oriented boundary data, which determines the embedded obstacle and corresponding PDE domain.Source: §4.1 · pp. 4–5

4.2. Tokenized manifolds by patch embeddings

  • The framework represents each boundary as a finite sequence of patch parametrizations with associated outward normal fields.Because different patch decompositions, orderings, or parametrizations can describe the same solid, the method uses basis coefficients and training with multiple decompositions of the same surface.Source: §4.2 · pp. 5–6

4.3. Parameterizing CAD Geometries

  • Trimmed CAD surfaces are decomposed into untrimmed NURBS patches whose parameterizations and consistently oriented outward normals follow from their NURBS descriptions.The patch representation is specified by knot vectors, control points, and weights.Source: §4.3 · p. 6

4.4. NURBS tokenization and operator architecture

  • CANTO tokenizes NURBS geometry and uses the resulting geometry context to condition a transformer that predicts surface and volume fields at arbitrary query locations without meshing the input.Each variable-size NURBS surface is encoded into a fixed-dimensional representation, and patch tokens are jointly processed to incorporate global geometry context.Source: §4.4 · pp. 6–7
  • NURBS-derived geometry tokens condition separate surface and volume streams, with surface anchors replaced by NURBS-derived tokens while volume anchors remain randomly sampled.The anchored neural field decoder predicts at arbitrary query locations without quadratic attention over query points.Source: §4.4 · p. 7

5. Experiments

Experiments evaluate CANTO on four automotive and aircraft aerodynamics benchmarks, compare performance and design choices, and test constrained AhmedML inverse design.

5.1. Performance

  • HiLiftAeroML relative L2 errors fall 19.8% for surface pressure, 2.3% for wall-shear stress, and 16.9% for velocity, the lowest errors on all considered fields.Table 1 compares CANTO with prior methods across the four benchmark datasets.Source: §5.1 · p. 7, Table 1 · p. 8
  • AhmedML has the lowest error on three of four fields, including 11.6% lower surface-pressure error than the state-of-the-art method; WindsorML leads every field and lowers AB-UPT surface-pressure error by 5.5%.Source: §5.1 · pp. 7–8, Table 1 · p. 8
  • On DrivAerML, CANTO matches AB-UPT overall, with the lowest volume-velocity error while AB-UPT remains marginally better on surface pressure.Source: §5.1 · p. 7, Table 1 · p. 8

5.2. Ablation Study

  • Fixed-size and variable-size NURBS encoders achieve comparable errors across four fields, with each variant performing better on different fields.The variable-size encoder is practically preferred because real-world CAD surfaces are highly heterogeneous.Source: §5.2 · p. 8, Table 2 · p. 8
  • CANTO avoids sampling dependence and achieves lower errors across tested anchor budgets and quantities, while AB-UPT remains sensitive to anchor distribution.Figure 2 compares uniform-on-mesh anchors with anchors sampled uniformly in NURBS parameter space and allocated by surface area.Source: §5.2 · p. 8, Figure 2 · p. 9

5.3. CANTO-based inverse design

  • CANTO finds AhmedML designs with 4.4%–20.4% lower drag than the best dataset designs at corresponding volume thresholds.CFD confirms the gains and lift constraints; CANTO predictions remain within 2.2% of corresponding CFD values.Source: Figure 3 · p. 9, §5.3 · p. 10

6. Conclusion

CANTO operates directly on parametric CAD and achieves state-of-the-art accuracy on most evaluated field-prediction tasks across four benchmarks. Its differentiability with respect to NURBS inputs supports gradient-based shape optimization in CAD parameter space.

6. Conclusion

  • Across four benchmarks, CANTO achieves state-of-the-art accuracy on most evaluated tasks, with a 19.8% relative reduction in HiLiftAeroML surface-pressure error.The conclusion also states that differentiability with respect to NURBS inputs enables gradient-based shape optimization in CAD parameter space.Source: §6 · p. 10

A. Model designs

  • Three Perceiver Pooling modules separately encode weighted control points and the two knot vectors, then project their concatenated outputs into one fixed-dimensional token per patch.Self-attention across patch tokens adds global context from the complete CAD geometry.Source: Figure 4 · p. 13

A.1. Backbone architecture

  • NURBS geometry tokens condition the surface and volume streams through cross-attention; refined-patch surface tokens replace sampled surface anchors, while volume anchors remain sampled.The anchored decoder predicts fields at dense query locations without quadratic attention over all query points.Source: §A.1 · pp. 13–14

Training objective

  • The end-to-end objective uses normalized surface pressure and wall-shear-stress targets and volume pressure and velocity targets, with λ balancing the surface and volume objectives.The supplied training description says the complete model is trained end to end.Source: Training objective · p. 14

Hyper-parameter validation

  • CANTO uses the AB-UPT authors’ parameter settings and a small validation set to optimize hyperparameters, training with Lion and a cosine learning-rate schedule with linear warmup.Dataset-specific configurations are reported in Tables 3 and 5.Source: Hyper-parameter validation · p. 14

Parameter counts

  • CANTO has 12.61M–17.17M trainable parameters across datasets; nearly all variation comes from the geometry encoder, while the remaining architecture stays near 8M parameters.The geometry encoder size depends on dataset-specific geometry configuration.Source: Parameter counts · p. 15

A.3. NURBS Anchor Split Resolution.

  • Errors fall sharply as parametric splits increase from 1 to 4 across fields; after training at 4 splits, performance remains stable at 5.The 4-split resolution is the training level used in the reported experiments.Source: §A.3 · p. 15, Figure 5 · p. 15

A.4. Point vs. Geometry Anchors.

  • On WindsorML, CANTO-PA retains spatially sampled surface anchors and degrades as their count falls, whereas primary CANTO’s NURBS-derived surface tokens bypass that sampling limit.The comparison isolates the CAD geometry encoder from the geometry-anchor choice.Source: §A.4 · p. 16, Table 7 · p. 16

A.5. Model robustness for varying number of anchors

CANTO’s surface-field predictions avoid spatial surface-anchor sampling and remain more robust than AB-UPT as anchor counts fall. Its inference also scales to very large query sets, with adjustable runtime-memory trade-offs.

A.5. Model robustness for varying number of anchors

  • Reducing AB-UPT surface anchors from 214 to 28 more than doubles surface-pressure error; CANTO bypasses surface sampling and outperforms AB-UPT even at 214 anchors.The comparison reports surface-field error against surface-anchor counts.Source: §A.5 · p. 16
  • For volume fields, reducing surface-anchor counts leaves CANTO’s error flat, and CANTO’s performance degrades at lower volume-anchor counts than AB-UPT’s.CANTO also remains robust on surface fields as volume anchors are reduced, while AB-UPT’s error increases sharply.Source: §A.5 · pp. 16–17, Figure 8 · p. 17, Figure 7 · p. 17

B.2. Inference latency and peak memory requirement

  • Inference runs on a single node and handles up to 30 million surface points and 200 million volume points with peak memory below 55 GB.Reported inference times for those cases are approximately 75–160 seconds.Source: §B.2 · p. 18, §B.1 · p. 18, Table 8 · p. 18

B.3. Inference runtime-memory trade-off

  • On a full AhmedML evaluation, reducing peak memory from 22.7 GB to 3.6 GB increases runtime from 131 s to 209 s at a chunk size of 50,000.The chunk size can be adjusted to available GPU memory, with a moderate runtime impact; the reported evaluation includes 22 million volume and 1 million surface queries.Source: §B.3 · p. 18

B.4. End-to-end performance comparison with traditional CFD

CANTO inference is substantially faster than the corresponding CFD solve on the two datasets with available timing comparisons. Including raw-CAD untrimming still leaves large wall-clock ratios, though the reported comparisons have workflow and hardware qualifications.

B.4. End-to-end performance comparison with traditional CFD

  • CANTO inference is approximately 2,300× faster than the CFD solve on AhmedML and 900× faster on DrivAerML.These are inference-to-solve wall-clock comparisons.Source: §B.4 · p. 19, Table 11 · p. 19
  • Including untrimming in the raw-CAD workflow leaves an approximately 2,300× ratio for AhmedML and greater than 48× for DrivAerML, even without meshing time.The reported ratios are wall-clock comparisons across different hardware; the untrimming implementation processes each geometry serially.Source: §B.4 · p. 19, Table 11 · p. 19

C. Datasets

The evaluation spans three automotive CFD datasets and one high-lift aircraft dataset, with distinct CAD representations, simulation settings, and train-validation-test splits. AhmedML geometries are reconstructed analytically from design parameters rather than fitted to sampled geometry.

C. Datasets

  • AhmedML contains 500 Ahmed-body variants; each is reconstructed from design parameters as 13 untrimmed NURBS surfaces, with 400 training, 50 validation, and 50 test cases.The decomposition has nine planar and four curved surfaces; the dataset’s original distribution does not include CAD files for each configuration.Source: C. Datasets · p. 19, Figure 9 · p. 20
  • WindsorML contains 350 Windsor-body configurations simulated with WMLES on grids exceeding 280 million cells; its CAD models have 42 NURBS surfaces and seven design variables.The paper uses an ordered 60/20/20 train-validation-test partition.Source: C. Datasets · p. 20
  • DrivaerML comprises 500 vehicle variants, of which 16 geometries were unavailable; the split uses 400 training, 34 validation, and 50 test samples.Provided CAD files contain roughly 8.5k NURBS surfaces per geometry.Source: C. Datasets · p. 20
  • HiLiftAeroML comprises 1,800 simulations from 180 high-lift aircraft variants across angles of attack from 4° to 22° in 2° increments.Its eight-dimensional design space covers slat and flap deflections and gap multipliers; simulations use 300–500 million-cell grids, with 1,260 training, 180 validation, and 360 test cases.Source: C. Datasets · pp. 20–21

D.2. Untrimming

D.2 replaces exact untrimming’s patch growth with a boundary-first decomposition that preserves watertightness and keeps patch complexity comparable to the input, with sub-millimetre worst-case preservation error.

D.2. Untrimming

  • [40] decomposes trimmed surfaces into tensor-product NURBS patches diffeomorphic to [0, 1]^2, which become the model’s geometric inputs.The exact procedure can substantially increase patch count, degree, and control points.Source: §D.2 · p. 21

Construction

  • The boundary-first construction preserves watertightness exactly while keeping patch count, degree, and control-point count comparable to the input, with a small geometric residual.Adjacent patches share a lifted edge and corresponding control-point row.Source: §D.2 · p. 21, Construction · p. 21

Empirical evaluation

  • Worst-case per-patch surface-preservation error is sub-millimetre, with the median several orders of magnitude smaller.Table 12 reports maximum and median errors normalized by the model’s 3-D bounding-box diagonal.Source: Empirical evaluation · p. 22, Table 12 · p. 22

Untrimming memory requirements and runtime

  • Untrimming runs serially per geometry on CPU and concurrently across the dataset on a 144-core, 2 TB node; its output is reusable and preprocessing is separate from inference.The implementation prioritizes dataset-level throughput over minimum single-geometry latency.Source: Untrimming memory requirements and… · p. 22

E. Inverse design and Optimization

CANTO is used to optimize drag over the six-parameter AhmedML body family under volume and lift constraints, integrating predicted fields over CAD geometry and using differentiable gradients. CFD-validated designs achieve lower drag than the best dataset designs at corresponding volume thresholds.

E. Inverse design and Optimization

  • The optimization minimizes drag over the six-parameter AhmedML body family, subject to a lower bound on enclosed volume and an upper bound on lift.The lift threshold defines a feasible set containing 298 of the dataset’s 500 designs.Source: E. Inverse design and Optimization · pp. 22–23

Drag and Lift integration

  • CANTO integrates predicted pressure and wall-shear stress over CAD patches to obtain drag and lift, while automatic differentiation supplies gradients to SLSQP.The predicted coefficients are integrated directly over the CAD geometry without introducing a CFD mesh into the optimization loop.Source: Drag and Lift integration · p. 23
  • Across Figure 3’s validated front, every optimized design improves on the best dataset design at its corresponding volume by 4.4% to 20.5%.CFD satisfies the lift constraint at every threshold, and surrogate predictions are accurate to within 2.2% throughout.Source: Drag and Lift integration · p. 24

F. Qualitative Surface-Pressure Results

Qualitative AhmedML and WindsorML visualizations show CANTO’s surface-pressure predictions against CFD reference fields and pointwise errors.

F. Qualitative Surface-Pressure Results

Related papers

Loading 2609.36806v1…