PaperlayerView exact revision on arXiv

Source-linked AI summary

Cross-Domain Pretraining for Steady-State Neural CFD Surrogates

Anthony Zhou, Amir Barati Farimani, Shirley Ho, Rudy Morel

arXiv:2610.10398v1cs.LG

TL;DR

Neural CFD surrogates often generalize only to applications represented in their training data, while generating new solver data is costly. This paper studies cross-domain pretraining on existing steady-state CFD datasets and finds improved transfer to held-out datasets, with fewer samples needed to reach target error than training from scratch.

  • Problem

    CFD surrogates have limited generalization beyond their training applications, while deploying to a new application requires costly application-specific solver data.

    Source: §1 · pp. 1–2

  • Method

    The study aggregates diverse steady-state, mesh-based CFD datasets, pretrains models, and evaluates their downstream performance on held-out datasets.

    Source: §1 · p. 2

  • Results

    Cross-domain pretraining improves zero- and few-shot performance on unseen datasets, and finetuning reaches a target error with 8–16× fewer samples than training from scratch.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    The findings support leveraging existing steady-state CFD datasets through cross-domain pretraining as surrogates expand to more engineering applications.

    Source: §5 · p. 10

    The dataset count remains small relative to cross-domain PDE work, and main results assume ground-truth finetuning-set normalization statistics.

    Source: §5 · p. 9

Abstract

from arXiv · show

Neural surrogates for computational fluid dynamics (CFD) have the potential to greatly enhance engineering innovation through accelerating simulation. However, the primary limitation for neural surrogates is the lack of generalization to geometries and applications beyond the training set, which is significant given the diversity of engineering scenarios. Currently, this is addressed by generating a new dataset for a specific application; however, this requires running costly numerical solvers. In this work, we take a step toward addressing this by studying neural surrogates trained across different geometries, boundary conditions, and fidelities. We find that cross-domain pretraining improves zero- and few-shot performance on held-out datasets relative to both training from scratch and transferring from domain-specific experts. In particular, finetuning a pretrained, cross-domain model can achieve 2-3x lower errors at the same sample size and use 8x fewer samples to achieve the same error, compared to training from scratch. This benefit is architecture agnostic and improves with model size and pretraining dataset diversity. Furthermore, we study how and why cross-domain pretraining works in CFD surrogates, and find that simply pooling steady-state datasets is both sufficient and effective. Given the high cost of generating CFD data, leveraging existing datasets through cross-domain pretraining will likely be a valuable strategy as future surrogates expand to tackle new problems and use cases.

1 INTRODUCTION

Mesh-based CFD surrogates can predict fields and engineering quantities, but typically generalize only within their training application, making costly application-specific data necessary. This work studies cross-domain pretraining on existing CFD datasets and reports improved transfer to held-out datasets.

1 INTRODUCTION

  • Mesh-based surrogates can predict steady-state fields and aggregate quantities, but their capabilities are largely limited to the application covered by training data.Deploying to a new application therefore requires a new dataset and costly numerical-solver runs.Source: §1 · p. 1
  • The study aggregates diverse steady-state, mesh-based CFD datasets, pretrains models, and evaluates downstream performance on held-out datasets.Source: §1 · p. 2
  • A joint model pretrained across CFD domains outperforms single-domain experts in zero- and few-shot transfer on held-out datasets.The study evaluates zero- and few-shot performance on four held-out datasets after pretraining across six CFD datasets.Source: §1 · p. 2, Figure 1 · p. 2
  • 8–16× fewer training samples let the finetuned joint model reach a target held-out-dataset error than training from scratch.Source: §1 · p. 2
  • Finetuning performance on unseen datasets improves with both pretraining dataset diversity and model size.Source: §1 · p. 2
  • The authors release eight publicly available CFD datasets in a common format, with dataloaders and training code.They also release code and pretrained model checkpoints.Source: §1 · p. 2

2 RELATED WORKS

Prior cross-domain PDE work has often focused on uniform-grid, time-dependent problems, while engineering CFD commonly uses unstructured meshes. This work differs from earlier within-domain CFD transfer by pretraining across multiple domains and evaluating on unseen domains.

Cross-Domain Training for PDEs

  • Cross-domain PDE foundation models have shown improved finetuning on held-out datasets and generalization to challenging experimental applications.The cited work concerns uniform-grid, time-dependent PDEs.Source: §2 · p. 2
  • Prior PDE cross-domain approaches include meta-learning, in-context learning, test-time compute scaling, and self-supervised representation learning.Source: §2 · p. 2

Mesh-based CFD Surrogates

  • Engineering CFD typically uses unstructured meshes, and meshes can exceed 100 million cells, motivating efficient attention and sequential high-resolution prediction.Source: §2 · p. 3
  • Unlike transfer within one SUV geometry family, this work pretrains across domains such as automobiles and aircraft and evaluates transfer on unseen domains.Source: §2 · p. 3

3 METHODS

The methods formulate steady-state CFD as query-based prediction of surface and volume fields, then combine diverse datasets for pretraining and downstream finetuning. They standardize fields, geometries, and simulation parameters and evaluate predictions on unseen geometries.

Background

  • The surrogate maps query points, geometry, and simulation parameters to field values, conditioning predictions globally on geometry and potentially on other queries.The formulation supports sequential prediction in batches when full-resolution fields are needed.Source: §3 · p. 3
  • Targets comprise surface pressure and wall shear stress plus volume pressure and velocity, supporting force estimates and flow-field inspection.Surface fields can be integrated to derive drag and lift; volume fields show how geometry affects surrounding flow.Source: §3 · p. 3

Datasets

  • Six pretraining datasets cover varied automotive and aerospace regimes, while four finetuning datasets add distinct geometries and setups, including high-resolution aerodynamics and hydrodynamics.The authors note shared phenomena across CFD cases, including front-of-body stagnation and downstream low-velocity wakes.Source: §3 · pp. 3–4

Normalization

  • Because raw fields span orders of magnitude, the models predict nondimensional Cp, Cf, and u*, then normalize fields and center, scale, and align geometries and queries.Simulation parameters are globally min-max normalized; automotive datasets set them to 0.Source: §3 · p. 4

Models

  • SMART is the primary architecture, with AB-UPT and Transolver++ also evaluated; joint pretraining samples among six datasets without dataset-index information.Models use matched settings of 32M parameters and 100k gradient steps, so the joint model sees each dataset about six times less often than its corresponding expert.Source: §3 · pp. 4–5

Evaluation

  • Evaluation uses NRMSE on four surface and four volume fields at fixed sampled points, averages the four variable-level errors, and reports denormalized physical-space errors on unseen validation geometries.The table reports validation errors across four held-out target datasets.Source: §3 · p. 5, Table 2 · p. 5

4 RESULTS

Cross-domain pretraining improves zero- and few-shot performance on held-out CFD datasets, while its benefits vary with sample availability, model scale, and domain ambiguity. The results also show a trade-off between fitting individual pretraining datasets and consistent performance across them.

4.1 GENERALIZATION TO UNSEEN DATASETS

  • Cross-domain training achieves lower zero- and few-shot errors than other models, even under an equal pretraining budget.The authors suggest that training across domains helps limit geometry- or setup-specific bias.Source: §4.1 · pp. 5–6
  • At 4,000 steps, the joint model matched scratch-model error at S=64 using S≈4–8, or 8–16× fewer samples, across four finetuning sets.With 400 steps, a joint model using S=2 outperformed scratch models given even the full held-out dataset (S>64).Source: Figure 3 · p. 6, §4.1 · p. 6
  • Expert transfer depends on domain proximity and may underperform random initialization at zero-shot, while joint-model advantages also hold for AB-UPT and Transolver++.The authors report that the same trends hold for these architectures during zero- and few-shot evaluation.Source: §4.1 · pp. 6–7

4.2 SCALING MODEL AND DATASET SIZE

  • Increasing pretraining dataset diversity or model size generally improves downstream performance, though the largest models show some zero-shot exceptions; diversity is the stronger zero-shot lever.With downstream samples, larger models can use them more efficiently; the authors identify simulation cost as a major obstacle to building diverse pretraining data.Source: Figure 4 · p. 7, §4.2 · p. 7

4.3 PRETRAINING PERFORMANCE

  • As the pretraining pool grows, per-dataset error rises slightly while average error improves, trading expert-level fit on one dataset for more consistent joint performance without catastrophic forgetting.The authors hypothesize that accommodating multiple geometries and output fields helps yield better generalization; lower pretraining loss alone does not guarantee better held-out transfer.Source: Table 3 · p. 8, §4.3 · p. 8

4.4 EXAMINING AMBIGUITY IN CROSS-DOMAIN PRETRAINING

  • Geometry-only models struggle when hidden simulation parameters vary in Double-Delta-α and DrivAerNet-ν, but match the oracle in DrivAerNet/ML; context or labels do not consistently help.The experiments compare geometry-only, paired-simulation context, and direct-parameter inputs; the authors conclude that pooling datasets can work without source-domain context or hidden-variable knowledge.Source: §4.4 · pp. 8–9

5 CONCLUSION

Cross-domain pretraining improves zero- and few-shot CFD surrogate performance on unseen datasets, reducing deployment data needs. The work also identifies limitations in dataset diversity, normalization assumptions, and full-resolution evaluation.

5 CONCLUSION

  • Cross-domain pretraining improves zero- and few-shot performance on unseen datasets, lowering error and the number of samples needed for deployment.The authors see promise in pooled datasets supporting larger cross-domain models for broader engineering applications.Source: §5 · p. 9
  • The study uses relatively few datasets, and CFD datasets may be less diverse than time-dependent PDE benchmarks.Its main results also assume ground-truth normalization statistics for finetuning sets, which may not always be available.Source: §5 · p. 9
  • Discretization invariance and full-resolution errors on fields with millions of cells are not evaluated; prior work reports slight full-resolution error increases for AB-UPT and SMART.The authors note that non-dimensionalization can mostly address the ground-truth normalization-statistics assumption.Source: §5 · p. 10
  • The authors advocate generating steady-state CFD datasets across many geometries and applications for pooled training of larger, general-purpose surrogates.Source: §5 · p. 10

A DATASET DETAILS

This appendix describes ten CFD datasets spanning automotive, marine, and aerodynamic settings, along with shared preprocessing and sampling choices. Figures 6 and 7 visualize sample geometries and fields across the datasets.

A DATASET DETAILS

  • Shared preprocessing crops effectively constant far-field regions, converts raw samples to memory-mapped arrays, and permutes points for efficient random-subset loading.A voxel-occupancy-weighted sampling scheme aims to make point distributions more spatially uniform across datasets with differing discretizations.Source: A DATASET DETAILS · p. 16

A.1 / A.2 DRIVAERNET++ AND DRIVAERML

  • DrivAerNet++ contains 8,000 parametrically morphed cars, while DrivAerML contains 500 time-averaged hybrid RANS/LES morphs of a DrivAer notchback.DrivAerNet++ uses steady incompressible RANS; the appendix also describes mirroring and field-convention processing for its samples.Source: §A.1 · p. 16, §A.2 · p. 16

A.3 / A.4 AHMEDML AND WINDSORML

  • AhmedML covers 500 Ahmed-body variants; WindsorML covers 355 squareback variants and is the only dataset described as having no RANS in its closure.AhmedML uses SA-DDES, while WindsorML uses wall-modelled LES.Source: §A.3 · p. 19, §A.4 · p. 19

A.5 / A.6 SHIFT-SUBMARINE AND EMMI-WING

  • SHIFT-Submarine contributes 101 parameterized hulls and is the only water-flow dataset; Emmi-Wing contributes 29,609 steady compressible solutions over parameterized swept wings.SHIFT-Submarine solver details are undisclosed and its RANS setup is an assumption; Emmi-Wing uses steady compressible RANS.Source: §A.5 · p. 20, §A.6 · p. 20

A.7 / A.8 SUPERWING AND DOUBLE-DELTA

  • One dataset contains 28,856 transonic solutions over 4,239 kinked, double-swept wing planforms; another contains 2,448 solutions over 272 parametric cranked-delta wings.The latter models leading-edge vortices and their breakdown at large angles of attack, with native geometry-disjoint splits.Source: §A.7 · p. 20, §A.8 · p. 21

A.9 / A.10 HILIFTAEROML AND SHIFT-CCA

  • HiLiftAeroML comprises 1,787 wall-modelled LES solutions across 180 high-lift geometry variants; SHIFT-CCA contains 99 distinct transonic UAV geometries.HiLiftAeroML meshes are adapted by angle of attack; SHIFT-CCA solver details are undisclosed and its RANS setup is an assumption.Source: §A.9 · p. 21, §A.10 · p. 22

B MODEL DETAILS

The model details describe mesh-based transformer architectures and their mechanisms for efficient prediction on dense, unstructured meshes. They also document conditioning inputs and implementation variants.

B MODEL DETAILS

  • The appendix provides architecture schematics and hyperparameters, with models matched at roughly 30M parameters and sharing the listed optimizer settings.The SMART implementation also uses SwiGLU layers and separate output heads for surface and volume fields.Source: B MODEL DETAILS · pp. 22–23

SMART

  • SMART uses cross-attention so query points attend to geometry latents rather than one another, controlling cost and preserving prediction independence from query discretization.Its schematic shows shared encoder-decoder weights and simulation-parameter conditioning through FiLM layers.Source: B MODEL DETAILS · p. 22, Figure 8 · p. 23

SMART-IC

  • SMART-IC adds a context simulation whose sampled surface and volume fields update SMART’s latent state, while providing context and query simulation parameters.The context surface field is assumed to contain geometry information, so the context geometry is not separately supplied.Source: B MODEL DETAILS · p. 23

AB-UPT

  • AB-UPT uses subsampled anchor queries to relay information to other queries through cross-attention, bounding cost and making predictions independent of the full query set.The implementation replaces radius pooling with ball-tree decomposition and replaces MLPs with SwiGLU layers.Source: B MODEL DETAILS · p. 23

Transolver++

  • Transolver++ aggregates input points onto learned slices for attention, but its resulting representation remains tied to the input discretization.The implementation uses surface queries for geometry information and AdaLN for conditioning; the schematic depicts parameter conditioning through AdaLN.Source: B MODEL DETAILS · pp. 23–24, Figure 9 · p. 23

C.1 DIFFERENT ARCHITECTURES

Cross-domain pretraining improves held-out dataset performance for AB-UPT and Transolver++ as well as in the main experiments. Pretraining validation shows only marginal architecture differences, while query-independent models support denser sequential inference.

Generalization to Unseen Datasets

  • Cross-domain pretraining improves downstream performance over expert-model initialization and training from scratch for AB-UPT and Transolver++ on held-out datasets.The appendix reports results for both architectures across four held-out targets and multiple finetuning sample counts.Source: §C.1 · pp. 24–25

Pretraining Performance

  • On pretraining validation sets, SMART appears strongest, but every architecture performs well and differences are marginal.The authors say this corroborates prior comparisons of SMART/AB-UPT with Transolver.Source: Table 8 · p. 25, §C.1 · p. 25
  • Query-independent cross-attention architectures can be queried sequentially on denser meshes without a large increase in error or memory.Source: §C.1 · p. 25

C.2 GLOBAL NORMALIZATION

Global normalization uses known freestream reference values to nondimensionalize CFD fields, helping align values across datasets. With this scheme, the joint model remains competitive on held-out datasets, though normalization choice warrants further study.

C.2 GLOBAL NORMALIZATION

  • Freestream reference values support nondimensional variables such as Cp, Cf, and u∗, aligning ranges across datasets.Velocity magnitudes spanning about 1 m/s in AhmedML and 300 m/s in Emmi Wing collapse to an approximate |u∗| range of [0, 1.5]; Cp is approximately [−1, 1], though it can be more negative.Source: §C.2 · p. 25
  • Global scaling applies fixed output-field factors across datasets, including 0.33 for Cp, 0.002 for Cf, and 0.2 for u′.The factors are computed on the pretraining set, excluding finetuning data, so nondimensional fields have approximately unit scale.Source: §C.2 · p. 25
  • 15 of 16 settings match or outperform baselines, and the joint model has the lowest zero-shot error on all four held-out datasets.Few-shot errors change little from dataset-specific normalization, although absolute zero-shot error rises on three held-out datasets.Source: Table 10 · p. 26, §C.2 · p. 26
  • Global normalization performs well, but normalization choice remains important and warrants further study.Because simulations contain many mesh points, means and variances may still be estimated adequately from a few finetuning samples.Source: §C.2 · pp. 26–27

C.3 DISCRETIZATION

The study trains with spatially uniform query points for consistent discretization across domains and to reflect inference conditions, then evaluates on simulation meshes. Errors tend to rise, but the joint model’s zero- and few-shot advantage persists.

C.3 DISCRETIZATION

  • Training queries are sampled approximately uniformly in space to keep discretization more consistent across domains and reflect deployment, when simulation meshes are usually unavailable.Meshing is expensive, predominantly CPU-based, and still relies on human expertise; simulation meshes can nevertheless resolve steep gradients such as boundary layers or adaptively refined regions.Source: §C.3 · p. 27
  • The joint model retains better zero- and few-shot performance when queried on the simulation mesh, supporting cross-domain pretraining across discretizations.Errors tend to rise for all models because spatially uniform samples resolve more of the easier-to-model freestream.Source: §C.3 · pp. 27–28

C.4 SCALING MODEL SIZE OR COMPUTE

Increasing model size or training compute lowers pretraining loss, but gains do not consistently transfer to unseen datasets. In-context simulations do not improve steady-state predictions, where geometry and boundary conditions are the key cross-domain variations.

C.4 SCALING MODEL SIZE OR COMPUTE

  • Beyond a certain point, larger models or more gradient steps can raise zero-shot error on unseen datasets, while few-shot performance does not consistently improve.Larger models tend to be slightly more sample-efficient; the study identifies more CFD data, then model size, then training compute as the most promising improvement order for generalization.Source: §C.4 · p. 28

C.5 IN-CONTEXT PRETRAINING

  • Providing a same-dataset context simulation does not reduce pretraining error; SMART-IC ignores the context, and zeroing it at inference leaves predictions unchanged.The authors suggest context would help only if unobserved factors such as solver or closure-model differences made similar inputs ill-defined, which they do not find in current steady-state CFD datasets.Source: §C.5 · p. 28, Table 15 · p. 29
  • For steady-state CFD, domains share Navier–Stokes equations and vary instead in geometry and boundary conditions, suggesting value in better representations of those variations over in-context mechanisms.The authors contrast this with temporal PDE settings, where context can reveal differing dynamics from observed frames.Source: §C.5 · pp. 28–29

C.6 VARIABLE-SPECIFIC ERRORS

Across individual output variables, the main trends persist: the joint model generally outperforms domain experts and models trained from scratch, while surface fields are harder to predict than volume fields.

C.6 VARIABLE-SPECIFIC ERRORS

  • The joint model generally outperforms domain experts and scratch-trained models across surface Cp, surface Cf, volume Cp, and volume u∗, while surface fields are harder to predict.Much of the volume lies in the generally smooth freestream, making volume fields easier to model.Source: §C.6 · p. 29

D ADDITIONAL VISUALIZATIONS

Section D presents zero-shot and two-shot prediction visualizations from the main experiments, plus validation-error tables for three quantities on four held-out datasets.

D ADDITIONAL VISUALIZATIONS

Loading 2610.10398v1…