PaperlayerView exact revision on arXiv

Source-linked AI summary

AB-UPT: Scaling Neural CFD Surrogates for High-Fidelity Automotive Aerodynamics Simulations via Anchored-Branched Universal Physics Transformers

Benedikt Alkin, Maurits Bleeker, Richard Kurle, Tobias Kronlachner, Reinhard Sonnleitner, Matthias Dorfer, Johannes Brandstetter

arXiv:2502.09692v4cs.LGcs.AI

TL;DR

Industrial automotive CFD surrogates must handle meshes exceeding 100 million cells, complex surface-volume interactions, and divergence-free nonlinear fields. AB-UPT combines multi-branch operators, low-dimensional anchored neural fields, and hard divergence-free formulations, achieving accurate scalable predictions with practical inference and training costs.

  • Problem

    Automotive CFD surrogates must scale to high-dimensional meshes with more than 100 million cells while modeling complex geometry-dependent physics and divergence-free vorticity from relatively few simulations.

  • Method

    AB-UPT uses separate geometry, surface, and volume branches with anchor tokens and neural-field decoding to produce scalable, arbitrary-resolution predictions and enforce divergence-free vorticity.

  • Results

    AB-UPT delivers state-of-the-art accuracy on automotive CFD simulations with up to 140 million mesh cells, while training in less than a day and predicting fields within seconds on one GPU.

  • Takeaways & Limitations

    The method supports accurate surface and volume simulation from CAD geometry alone, avoiding costly CFD mesh creation for inference while maintaining physical consistency through divergence-free predictions.

  • Takeaways & Limitations

    AB-UPT’s anchor self-attention has quadratic complexity, so very large anchor sets can cause computational demands to explode and may require linear-attention mechanisms.

Abstract

from arXiv · show

Recent advances in neural surrogate modeling offer the potential for transformative innovations in applications such as automotive aerodynamics. Yet, industrial-scale problems often involve volumetric meshes with cell counts reaching 100 million, presenting major scalability challenges. Complex geometries further complicate modeling through intricate surface-volume interactions, while quantities such as vorticity are highly nonlinear and must satisfy strict divergence-free constraints. To address these requirements, we introduce AB-UPT as a novel modeling scheme for building neural surrogates for CFD simulations. AB-UPT is designed to: (i) decouple geometry encoding and prediction tasks via multi-branch operators; (ii) enable scalability to high-resolution outputs via neural simulation in a low-dimensional latent space, coupled with anchored neural field decoders to predict high-fidelity outputs; (iii) enforce physics consistency by a divergence-free formulation. We show that AB-UPT yields state-of-the-art predictive accuracy of surface and volume fields on automotive CFD simulations ranging from 33 thousand up to 150 million mesh cells. Furthermore, our anchored neural field architecture enables the enforcement of hard physical constraints on the physics predictions without degradation in performance, exemplified by modeling divergence-free vorticity fields. Notably, the proposed models can be trained on a single GPU in less than a day and predict industry-standard surface and volume fields within seconds. Additionally, we show that the flexible design of our method enables neural simulation from a CAD geometry alone, thereby eliminating the need for costly CFD meshing procedures for inference.

1 Introduction

AB-UPT addresses the scale and physics challenges of automotive CFD surrogates by combining multi-branch geometry and field modeling with anchored neural fields. It targets industrial meshes with more than 100 million cells while preserving accurate, fast, and physically consistent predictions.

  • The architecture separates geometry, surface, and volume processing through multiple interacting branches, decoupling feature extraction from field prediction.Geometry is encoded into reduced latent tokens, while surface and volume interactions are modeled through cross-attention.
  • Anchor attention reduces computation by applying quadratic self-attention only to a small anchor set and linear cross-attention to the remaining query points.The number of anchor tokens controls computational capacity, while query predictions remain independently computable.
  • AB-UPT trains in less than a day and predicts surface and volume fields within seconds on a single GPU.
  • AB-UPT achieves state-of-the-art surface and volume accuracy on automotive CFD simulations with up to 140 million mesh cells.
  • AB-UPT can simulate from CAD geometry alone, avoiding the costly creation of a CFD simulation mesh for inference.
  • The anchored neural field supports arbitrary-resolution predictions and a divergence-free vorticity formulation for physically consistent outputs.Its conditional-field interpretation permits differential operators while retaining pointwise predictions at arbitrary query locations.

2 Preliminaries and related work

Automotive CFD combines high-dimensional meshes, limited simulation samples, geometry-dependent surface–volume interactions, and physically constrained fields. Neural operators address these challenges by mapping discretized inputs to continuous outputs, with AB-UPT supporting flexible geometry encoding and arbitrary-resolution prediction.

  • 2.1 CFD for automotive aerodynamics: Automotive CFD uses discretized meshes to represent complex flow domains, while turbulence modeling balances simulation accuracy against computational cost.Finite-volume methods transform CAD geometry into structured or unstructured volumetric meshes; RANS and LES provide alternative turbulence-modeling strategies.
  • 2.1 CFD for automotive aerodynamics: Automotive aerodynamic quantities span surface fields, volumetric fields, and integral drag and lift coefficients derived from surface forces.Surface predictions can estimate drag and lift without reconstructing the full 3D volume field.
  • 2.1 CFD for automotive aerodynamics: Vorticity is divergence-free because it is the curl of velocity, whereas incompressible flow imposes divergence-freeness directly on velocity.The two constraints arise from different mathematical and physical considerations.
  • 2.2 Neural surrogate modeling: Neural operators map functions between spaces and can produce continuous outputs across varying input sampling resolutions and patterns.Cross-attention can incorporate information from one latent representation into another and can compress or expand token representations.
  • 2.2 Neural surrogate modeling: AB-UPT uses flexible multi-branch encoding and anchor attention to support arbitrary-resolution outputs from different input representations.Its neural-field decoder conditions predictions on latent context while evaluating queried coordinates.

3 Anchored-Branched Universal Physics Transformers

AB-UPT separates geometry, surface, and volume processing while using a small anchor set to condition predictions at many query locations. This design reduces attention costs, supports arbitrary-resolution fields, and enables hard divergence-free vorticity constraints through an anchored neural field.

  • 3.1 Multi-branch architecture: AB-UPT separates geometry encoding, surface simulation, and volume simulation into interacting branches, with shared physics-block parameters and cross-branch attention.The geometry branch provides geometry context, while alternating cross-attention enables surface–volume interactions.
  • 3.2 Anchor attention: Anchor attention uses a small set of anchor tokens for self-attention and lets all remaining query tokens cross-attend to them.This makes anchor computation the quadratic component while query prediction remains linear in the number of queries.
  • 3.2 Anchor attention: Chunking query tokens reduces anchor-attention memory from O(N) to O(M + C), enabling constant-memory inference for meshes with more than 100 million points.C is the query-chunk size, while M is the number of anchor tokens.
  • 3.2 Anchor attention: Treating anchors as conditioning context turns AB-UPT into a conditional neural field whose predictions can be queried independently at arbitrary positions and resolutions.The same formulation permits local differential operators to act on predicted fields.
  • 3.3 Enforcing physics consistency: The anchored neural-field formulation enforces divergence-free vorticity by predicting it as the curl of a modeled velocity field.Because the divergence of a curl is zero, the constraint is architectural rather than a soft loss penalty.

4 Experiments

The experiments evaluate efficient training, AB-UPT’s architectural design, benchmark accuracy, physics consistency, and scaling behavior for automotive CFD surrogates. AB-UPT achieves strong accuracy across surface and volume fields while supporting efficient, scalable prediction and divergence-free vorticity.

  • 4 Experiments: The experiments use three automotive CFD datasets with varying surface and volume resolutions, variables, and limited numbers of expensive simulations.The datasets include ShapeNet-Car, AhmedML, and DrivAerML; high-quality fields include pressure, wall shear stress, velocity, and vorticity.
  • 4.2 Efficient training on million-scale CFD meshes: 16K input points provide accurate million-scale predictions at relatively low computational cost, with models using 16K surface/volume points training in less than 10 GPU-hours.Training on lower resolutions makes standard Transformer self-attention feasible for million-scale CFD outputs, although computational demand increases with input count.
  • 4.3 Model design: Anchor attention restricts self-attention to a small anchor subset while query points cross-attend to anchors, restoring neural-field and mesh-independent prediction properties.The number of anchors controls computational capacity, while query predictions scale linearly outside the anchor self-attention operation.
  • 4.4 Benchmarking AB-UPT against other neural surrogate models: AB-UPT consistently outperforms other neural surrogates across every evaluated metric and dataset, including complex vorticity, while maintaining strong surface and volume performance.The comparison covers ShapeNet-Car, AhmedML, and DrivAerML using relative L2 error for surface pressure, volume velocity, and volume vorticity.
  • 4.7 Divergence-free vorticity: The divergence-free vorticity formulation matches direct prediction performance while guaranteeing zero predicted-vorticity divergence by construction.The formulation relies on AB-UPT’s conditional neural-field structure, which permits applying differential operators to the predicted field.
  • 4.8 Scaling AB-UPT: Increasing anchor count improves performance only up to saturation: 65K training anchors suffice for DrivAerML, while 131K shows overfitting despite lower training loss.The 65K threshold is three orders of magnitude below DrivAerML’s 150M classical CFD mesh cells.

5 Limitations and future work

The paper identifies quadratic anchor self-attention as the main scalability bottleneck and outlines parallelization and broader modeling extensions as future work. It also bounds current evidence by fidelity–variability trade-offs and the difficulty of extending divergence-free vorticity constraints to mass conservation.

  • Computational scalability: More anchor tokens can improve accuracy, but their quadratic self-attention cost causes computational demands to rise rapidly.The paper notes that a small anchor set performs well on current public datasets, while larger sets increase accuracy and computation quadratically.
  • Computational scalability: For extremely large anchor sets, AB-UPT would require linear-attention mechanisms, while sequence or tensor parallelism could scale training further.Inference can also be parallelized over query tokens to reduce runtime linearly.
  • Future directions: Future work includes domain-informed or learned anchor sampling, heterogeneous-fidelity training, transfer learning, uncertainty quantification, and applications beyond automotive CFD.The paper presents these directions as outside the current study’s scope and expects AB-UPT to transfer to other simulation types.
  • Data limitations: CFD surrogate fidelity remains constrained by the underlying turbulence model and meshing strategy, especially for complex near-wall automotive flow.The paper highlights HRLES data acquisition and external-aerodynamics fidelity as continuing challenges.
  • Data limitations: Current public CFD datasets trade simulation fidelity against input variability, limiting simultaneous coverage of high fidelity and broad geometric variation.The paper focuses mainly on high-fidelity HRLES simulations, which are expensive to obtain.
  • Physics constraints: The demonstrated physics constraint targets divergence-free vorticity, while mass-conserving velocity fields require additional treatment because curl alone leaves harmonic components unresolved.The authors identify mass conservation as a non-trivial future extension.

6 Conclusion

AB-UPT addresses automotive CFD scale by learning dynamics in a reduced latent representation and decoding predictions at arbitrary resolution through anchor tokens. The method outperforms strong surrogate baselines, supports CAD-only inference, and imposes divergence-free vorticity constraints.

  • Conclusion: AB-UPT scales to automotive CFD simulations with hundreds of millions of mesh cells by learning complex dynamics at heavily reduced resolution.High-resolution outputs remain available through anchor attention rather than requiring full-resolution self-attention.
  • Conclusion: Anchor attention uses a small set of context tokens to generate arbitrary-resolution predictions with linear complexity in the query positions.The design separates reduced-resolution modeling from high-resolution field prediction.
  • Conclusion: AB-UPT outperforms strong neural surrogate benchmarks while enabling mesh-independent simulation directly from CAD geometry and physically consistent divergence-free vorticity prediction.These properties avoid costly CFD meshing during inference and use the neural-field formulation for the physical constraint.
  • Design study: Increasing geometry-input resolution does not significantly improve performance but increases runtime, and k-NN graph connectivity can match radius-graph results with greater input-count dependence.High-resolution geometries can nevertheless be represented with relatively few geometry inputs and tokens.
  • Design study: Integrating geometry information at the model start performs better than later or repeated geometry cross-attention, while supernode pooling followed by global exchange benefits the geometry branch.The study reports that additional Transformer blocks are unnecessary and that several alternative design differences are negligible.

A.3 Training without query tokens

The study examines whether query-token losses are needed when anchor attention shares weights between anchors and queries. Training with query supervision improves query generalization, while more query tokens improve predictions at added computational cost.

  • Training objectives: The main paper uses anchor-only loss because it is computationally more efficient despite the slight query-performance penalty.The reported discrepancy arises because anchors attend to themselves while queries only attend to anchors.
  • Training objectives: Training with anchor and query losses achieves the best performance and nearly equal anchor and query losses, according to the train–test discrepancy study.Training with only anchors causes a slight inference-loss increase on queries, whereas including queries removes that discrepancy.
  • Query-token scaling: Using more query tokens improves query predictions but increases computation, while combined anchor-and-query training permits fewer query tokens.Figure 9 presents this performance–cost trade-off on DrivAerML with 16K anchor tokens.
  • Evaluation: Random anchor sampling introduces evaluation variance, which the study measures by generating multiple predictions at identical query locations.The protocol concatenates anchor chunks as query positions to assess repeated predictions.
  • Query prediction: Anchor attention produces high-fidelity predictions for query points up to 140 million points after training on 16K anchors, outperforming simple k-NN interpolation.The comparison attributes the advantage to learned nonlinear interactions among anchor points.
  • Implementation: Mixed-precision training is stabilized by retaining positional embeddings and RoPE in fp32 while otherwise using fp16.The approach preserves predictive accuracy while accelerating training and inference.

B.1 DrivAerNet++ evaluation

On DrivAerNet++, AB-UPT is evaluated on a large, geometrically diverse dataset generated with a lower-fidelity RANS turbulence model. All surrogate models perform worse than on higher-fidelity datasets, suggesting dataset noise rather than model capacity may limit results.

  • Dataset and setup: DrivAerNet++ contains over 8,000 simulations with extensive geometric variation but relies on lower-fidelity RANS simulations.The evaluation uses the same setup as the main experiments and predicts the same fields across models.
  • Results: Transolver and Transformer are closer in performance on DrivAerNet++ than in the ShapeNet-Car, AhmedML, and DrivAerML comparisons.The passage reports this comparative pattern without providing the corresponding table values here.
  • Results: All surrogate models achieve worse relative L2 error on DrivAerNet++ than on DrivAerML or ShapeNet-Car.The paper reports this degradation across attempted increases in training duration, model size, and point counts.
  • Interpretation: Because training loss continues to decline while performance remains limited, the paper suggests model capacity is not the main bottleneck and that lower-fidelity simulations may contain irreducible noise.The primary cause of the discrepancy was not investigated further.

B.2 Benchmarking AB-UPT against TripNet on DrivAerNet++

AB-UPT is evaluated against TripNet on DrivAerNet++ using a shared split and five-run median metrics. The reported relative L2 errors show AB-UPT outperforming TripNet, often by a considerable margin.

  • B.2 Benchmarking AB-UPT against TripNet on DrivAerNet++: The evaluation uses Elrefaie et al.'s train/validation/test split and reports median metrics over five AB-UPT training runs.TripNet values are taken directly from Chen et al. (2025) because no public implementation was available and its preprocessing differs.
  • B.2 Benchmarking AB-UPT against TripNet on DrivAerNet++: The study follows TripNet's experimental setup for vector-valued fields and reports relative L2 error for wall shear stress and velocity quantities.This aligns the comparison across the listed surface and volume predictions.
  • B.2 Benchmarking AB-UPT against TripNet on DrivAerNet++: AB-UPT outperforms TripNet, often by quite a margin, across DrivAerNet++ surface and volume field relative L2 errors.The comparison covers surface pressure, wall-shear-stress magnitude, volume velocity magnitude, and directional velocity components.

B.3 Benchmarking AB-UPT against DoMINO

AB-UPT is compared with DoMINO on DrivAerML using the same data split and evaluation metrics. Under this matched protocol, AB-UPT outperforms DoMINO by a sufficient margin.

  • B.3 Benchmarking AB-UPT against DoMINO: The DrivAerML split includes 20% out-of-distribution test samples based on the range of drag-force values.AB-UPT was trained with the split provided for DoMINO to preserve protocol alignment.
  • B.3 Benchmarking AB-UPT against DoMINO: AB-UPT metrics are medians over five training runs, while DoMINO values are taken directly from Ranade et al. (2025).All other AB-UPT hyperparameters remain consistent with the settings reported in Appendix C.
  • B.3 Benchmarking AB-UPT against DoMINO: AB-UPT outperforms DoMINO by a sufficient margin when both models use the same DrivAerML train and test splits.The comparison reports relative L2 errors for surface and volume quantities, with lower values indicating better performance.

B.4 Benchmarking AB-UPT against Erwin

The Erwin comparisons examine whether local ball attention improves efficiency without matching the accuracy of quadratic Transformers. Across ShapeNet-Car and DrivAerML, AB-UPT retains the strongest reported accuracy, while ball attention becomes faster at larger input sizes.

  • B.4 Benchmarking AB-UPT against Erwin: Ball attention changes attention complexity from O(N^2) to O((N/B) * B^2), where B is the number of tokens per local ball.When B = N, the formulation reduces to full self-attention.
  • B.4 Benchmarking AB-UPT against Erwin: Full quadratic self-attention remains feasible for DrivAerML simulations with more than 100 million mesh cells, while local attention is presented as a runtime-oriented alternative.The Erwin adaptation uses 12 ball-attention blocks with dimension 192 and omits rotational embeddings.
  • B.4 Benchmarking AB-UPT against Erwin: Quadratic-complexity Transformers obtain better accuracies than Erwin, although Erwin is the strongest linear-complexity Transformer on ShapeNet-Car among the considered models.The ShapeNet-Car benchmark evaluates relative L2 errors for surface pressure and volume velocity.
  • B.4 Benchmarking AB-UPT against Erwin: On DrivAerML, ball attention initially incurs memory-restructuring overhead but becomes faster than branched self-attention as the number of inputs grows.The comparison uses the default 16K/16K surface/volume-point setting and reports a significant accuracy gap from AB-UPT.
  • B.4 Benchmarking AB-UPT against Erwin: Increasing Transolver's slice count slightly improves performance but substantially increases memory and compute demands, without approaching AB-UPT.Figure 12 compares performance, memory consumption, and runtime on ShapeNet-Car and DrivAerML.

B.6 Transolver and baseline reproducibility

The reproducibility study reruns ShapeNet-Car baselines under Transolver's reported setup and compares median and best performance across five seeds. The resulting baseline behavior largely matches the original publication, while AB-UPT remains ahead of the baselines.

  • B.6 Transolver and baseline reproducibility: The framework reruns ShapeNet-Car experiments with the data loading, training hyperparameters, model parameters, and input features reported by Wu et al. (2024).The input features include SDF, coordinates, and normal vectors.
  • B.6 Transolver and baseline reproducibility: AB-UPT outperforms all baselines by a sufficient margin in the reproduced ShapeNet-Car framework, even without additional SDF and normal-vector inputs.The comparison reports median and best pressure and velocity results over five training runs.
  • B.6 Transolver and baseline reproducibility: The reproduced Transolver median results are especially close to those reported in the original publication.The study evaluates pressure and velocity using the same experimental setup as the Transolver paper.
  • B.6 Transolver and baseline reproducibility: Differences from the original reported values are attributed to training randomness and minor implementation differences, while the overall results remain aligned.The authors use this agreement to conclude that the experimental framework is robust and fair for comparison with Wu et al. (2024).

B.7 Reproducibility of Transolver++

The appendix evaluates a stepwise reproduction of Transolver++ and finds that its defining modifications consistently reduce performance relative to vanilla Transolver and AB-UPT.

  • Each Transolver++ modification—reparameterization, Gumbel-Softmax, and adaptive temperature—worsened evaluation performance when applied sequentially.
  • Transolver++ performs consistently worse than vanilla Transolver and AB-UPT across all considered datasets.This conclusion is supported using both an independent reimplementation and a publicly available implementation.
  • Because both implementations underperformed vanilla Transolver, the authors excluded Transolver++ from the benchmark models.

B.8 Full results: benchmarking AB-UPT against other neural surrogate models

The full benchmark evaluates AB-UPT across automotive CFD datasets and surface and volume fields, with results indicating that it outperforms other neural surrogate models, often by a considerable margin.

  • AB-UPT outperforms other neural surrogate models, often by quite a margin, across ShapeNet-Car, AhmedML, and DrivAerML fields.Table 22 reports relative L2 errors for surface pressure, volume velocity, volume vorticity, wall shear stress, and volume pressure where available.
  • AB-UPT predicts velocity streamlines on a DrivAerML sample using values inferred by the model alongside ground-truth velocity streamlines.
  • The benchmark covers datasets ranging from small ShapeNet-Car meshes to approximately 20-million-cell AhmedML and approximately 140-million-volume-cell DrivAerML simulations.
  • Evaluation uses relative L2 error between predicted and ground-truth point-cloud outputs, with metrics computed on unnormalized targets and predictions.
  • The architecture combines a geometry branch with interleaved cross-attention and self-attention across surface and volume branches before final self-attention blocks.

C.5.3 Evaluation variance reduction

The evaluation protocol reduces variance by repeating training and evaluation, averaging model weights and test outputs, and matching inference chunks to training-scale inputs.

  • Each model is trained five times with different random seeds, and the median result is reported across runs.
  • Exponential moving averages of model weights and ten evaluation passes are used to stabilize reported results.The EMA update factor is 0.9999.
  • Test samples are chunked to match training inputs, with volume cells first subsampled to match surface-cell counts before concatenating chunks for metrics.
  • The protocol addresses high variance caused by few train/test samples, point sampling, initialization, dataset shuffling, and batch-size-one training.
  • These precautions reflect baseline constraints; AB-UPT can decode arbitrary mesh sizes and use batch sizes greater than one on a single GPU.
Loading 2502.09692v4…