PaperlayerView exact revision on arXiv

Source-linked AI summary

Compute-optimal data scaling for neural surrogates via multi-fidelity training

Paul Setinek, Fabian Paischer, Nils Thuerey, Johannes Brandstetter, Felix Koehler

arXiv:2610.17221v1cs.LGcs.AI

TL;DR

PDE surrogates depend on computationally expensive simulated data, and the cost and accuracy of each sample vary with numerical fidelity. The paper formalizes training across a discrete fidelity spectrum and finds that optimal mixtures match high-fidelity-only accuracy with lower data-generation budgets across diverse settings.

  • Problem

    PDE-surrogate data generation can outweigh model-training cost, while fixed-fidelity datasets do not exploit the predictable cost–accuracy trade-off of numerical fidelity.

    Source: §1 · p. 2

  • Method

    The paper formalizes surrogate training under a fixed data-generation budget over a fine-grained discrete spectrum of simulation fidelities and trains with optimal fidelity mixtures.

    Source: §1 · p. 2

  • Results

    At matched test error, multi-fidelity training reduces data-generation budget by an average of ∼3.1× across 20 setups, with peaks up to 6.3×.

    Source: §6 · p. 10

  • Takeaways & Limitations

    The authors recommend solver-error fidelity as a practical starting point because it consistently outperforms discretization-error mixtures on resolution-invariance problems and changes only one solver parameter.

    Source: §6 · p. 10

    The experiments assume data generation dominates training cost and that models are sufficiently large and trained long enough to absorb additional data; the extrapolative recipe also degrades in some discretization-error settings.

    Source: §6 · p. 9

Abstract

from arXiv · show

Neural surrogates for Partial Differential Equations (PDEs) are usually trained on synthetic data from numerical simulators. As models grow and scaling laws demand larger datasets, the cost of data generation often outweighs the training cost and becomes the bottleneck in the scaling quest. Unlike data in other domains, however, each PDE sample has a tunable cost that depends on its numerical fidelity, with the resulting cost-accuracy trade-off governed by known convergence rates. We exploit this property by formalizing neural surrogate training as a function of both data quantity and fidelity under a fixed data generation compute budget, treating fidelity as a fine-grained discrete spectrum rather than a binary distinction. Building on this, we show that multi-fidelity training enables Pareto-optimal error scaling against data generation compute. By using simulations at lower fidelities as an additional training signal, training on the optimal fidelity mixture matches the accuracy of high-fidelity-only training, with up to $6.3\times$ less data generation compute across our settings. We validate these findings across six PDE problems spanning structured grids and unstructured meshes, two distinct error sources (discretization and iterative-solver error), and four surrogate architectures, totaling over 7,000 training runs.

1 Introduction

Neural-surrogate data generation can become a scaling bottleneck because each PDE sample requires computation, while simulation fidelity offers a controllable cost–accuracy trade-off. The paper formalizes training across data quantity and a discrete fidelity spectrum, and reports that optimal mixtures match high-fidelity-only accuracy with less generation compute.

1 Introduction

  • Up to 6.3× less data generation compute lets optimal multi-fidelity mixtures reach the error of high-fidelity-only training across diverse settings.Figure 1 compares optimal mixtures with high-fidelity-only training.Source: Figure 1 · p. 2
  • Data generation can dominate network-training cost for PDE surrogates, and expensive datasets delay when a surrogate becomes cost-effective for downstream tasks.The cited example is computational fluid dynamics.Source: §1 · p. 2
  • Unlike fixed-fidelity datasets, PDE samples have fidelity-dependent compute costs and a predictable cost–accuracy trade-off governed by solver convergence rates.The paper asks whether a fidelity mixture can improve surrogate accuracy at fixed data-generation cost.Source: §1 · p. 2
  • The paper formalizes training under a fixed generation budget across a fine-grained discrete fidelity spectrum, and reports Pareto-optimal scaling across PDEs, discretizations, and architectures.The study comprises over 7,000 training runs and more than 12,000 GPU-hours.Source: §1 · p. 2

2 Related work

Prior work studies data quality, multi-fidelity surrogate training, and scaling, but the supplied passages identify gaps in analyzing error against data-generation cost and in modeling realistic fidelity variation. This paper studies that trade-off across numerical error sources and surrogate settings.

2 Related work

  • Earlier data-quality scaling work balances quality and quantity under model-training compute, whereas PDE surrogate studies using ERA5 do not account for simulation cost or quality.The paper instead focuses on dataset-generation compute, identified as a bottleneck in many AI4Science applications.Source: §2 · p. 3
  • Prior multi-fidelity surrogate approaches use correction networks or transfer learning, but do not analyze error scaling with data-generation cost or the fidelity–dataset-size trade-off at fixed compute.Some studies vary two discretization fidelities, while other approaches have different fidelity or data-cost setups.Source: §2 · p. 3
  • The paper extends this landscape by studying error scaling over a fine-grained fidelity spectrum spanning discretization and solver error across PDEs, discretizations, and surrogate architectures.The reported settings include regular grids, unstructured meshes, and scalable transformers.Source: §2 · p. 3

3 Methodology

The methodology treats simulator outputs as imperfect approximations whose fidelity-dependent errors and costs shape the available training data. It then samples from a discrete fidelity spectrum under a generation budget and conditions the surrogate on fidelity.

3.1 Errors in numerical simulators

  • Numerical simulators map discrete input fields to target solutions, but their outputs are imperfect approximations of the true solution.The method assumes access to a simulator for generating surrogate training data.Source: §3.1 · pp. 3–4
  • Modeling error from governing equations, boundary conditions, and simplifying assumptions is outside the study’s scope.The authors cite limited evidence of meaningful transfer across modeling fidelities as the reason.Source: §3.1 · p. 4
  • Discretization fidelity corresponds to spatial resolution, with error decaying algebraically, while iterative-solver fidelity corresponds to iteration count, with error decaying exponentially.Increasing fidelity reduces error but incurs additional computational cost.Source: §3.1 · p. 4

3.2 Multi-fidelity formalism

  • The method defines an ordered discrete spectrum of admissible fidelities and normalizes its values so the mixture parameter is comparable across error sources.The spectrum is bounded by minimum and maximum fidelity.Source: §3.2 · p. 4
  • A softmax distribution over the fidelity spectrum uses λ to control the sample-quality and dataset-size trade-off; positive λ favors higher fidelities, negative λ lower ones, and zero gives a uniform distribution.Datasets are drawn under a fixed generation budget, with expected sample count depending on the expected fidelity cost.Source: §3.2 · p. 4
  • The surrogate receives normalized sample fidelity as an additional conditioning input, enabling fidelity steering and interpolation at inference.The model is evaluated on the highest fidelity in the support using a high-fidelity hold-out dataset.Source: §3.2 · p. 5

4 Validating the multi-fidelity framework

The fine-grained spectrum approach reaches 33% lower error than the binary baseline in the reported comparison, and both mixtures outperform high-fidelity-only training. Fidelity conditioning also enables inference-time steering across fidelity levels.

4.1 Formulating fidelity as a fine-grained spectrum

  • 33% lower error: the spectrum formulation beats the binary baseline, and both mixtures outperform high-fidelity-only training.The binary approach uses only the coarsest and finest resolutions, while the spectrum adds 19 intermediate resolutions at the same data-generation budget.Source: Figure 4 · p. 5, §4.1 · p. 5
  • Conditioning does not affect inference speed because the constant conditioning at ξmax can be baked into the network weights.Source: §4.1 · p. 5

4.2 Inference-time fidelity steering

  • Conditioning on sample fidelity lets the surrogate track the simulator’s error curve, interpolate at unseen interior fidelities, and rise in error beyond training support.Near ξ̂ → 1, the surrogate plateaus above the simulator but has lower error than a surrogate trained only on high-fidelity samples.Source: §4.2 · p. 6

5 Pareto-optimal data scaling

Across finite data-generation budgets, optimal fidelity mixtures enable Pareto-optimal scaling, while their scaling curves converge with high-fidelity-only training at large budgets. Experiments span grids and unstructured meshes, multiple PDEs, error sources, and surrogate architectures.

5 Pareto-optimal data scaling

  • Figure 5 tests fidelity conditioning using a Transolver hyperelasticity model trained at ˜B = 1,600 and λ⋆ = −3.59.The figure caption characterizes the result as respecting the conditioning signal.Source: Figure 5 · p. 6
  • Multi-fidelity training enables Pareto-optimal scaling for both Poisson 3D error sources, with interior optimal mixtures at finite budgets and convergence to high-fidelity-only training at large budgets.Figure 6 shows the iso-budget curves over λ and the compute-optimal Pareto fronts; experiments span the architectures, PDEs, representations, and error sources summarized in Table 1.Source: §5 · p. 6, Table 1 · p. 6, Figure 6 · p. 7

5.1 Poisson equation as a testbed for large parametric studies

  • At ˜B = 300, the 3D U-Net achieves 46.9% / 42.2% error reduction on the solver / discretization axes, using roughly 3× / 6× more training samples than the high-fidelity baseline.The discretization fidelity menu spans a 216× cost ratio versus 4× for solver error, accompanying fixed-threshold savings of 4.6× versus 2.7×, respectively.Source: §5.1 · p. 7
  • Across 1D and 2D experiments, the U-Net reaches 2.0–3.1× lower terminal error than the FNO, which recovers 57–90% of the U-Net’s relative multi-fidelity gain.The reported comparison is at the highest tested normalized budget; the authors suggest the multi-fidelity benefit depends on distance from the surrogate’s capacity floor.Source: §5.1 · p. 7

5.2 Structural mechanics on unstructured meshes

  • At ˜B ≈ 400, hyperelasticity multi-fidelity training reduces test error by 59% / 49.4% on the solver / discretization axes, using 8× / 10.4× more samples than the high-fidelity baseline.Figure 7 reports results for Transolver on unstructured meshes; the solver-axis savings are larger, and other mesh settings report up to 6.3× cost savings at the same 10% threshold.Source: §5.2 · pp. 7–8, Figure 7 · p. 8

5.3 Towards a real use-case in external aerodynamics

  • For airfoils, multi-fidelity training reduces data-generation cost by 2.4× / 2.5× for Transolver / AB-UPT at an average 2% volumetric relative L2 error threshold.For AB-UPT at ˜B ≈ 200, relative L2 error falls 54.1% on volume fields and 60.3% on surface fields; lift and drag coefficients also follow the multi-fidelity Pareto front.Source: Figure 8 · p. 8, §5.3 · pp. 8–9

5.4 Extrapolating optimal fidelity mixtures across budgets: a post-hoc analysis

  • A post-hoc calibration recipe fits λ⋆(˜B) = a + b log(˜B), with average extrapolation regret below 5% on solver-error setups and at most 10% on mesh-discretization settings.For almost all extrapolated mixtures, error remains below the corresponding high-fidelity-only surrogate, except some largest-budget cases where the scaling curves have converged.Source: §5.4 · p. 9, Table 2 · p. 9

6 Conclusion, limitations and future work

At finite data-generation budgets, optimal mixtures of simulation fidelities outperform high-fidelity-only training across the tested setups, while the approaches converge at very large budgets. The conclusion recommends solver-error fidelity as a practical starting point and identifies assumptions and scope limits for the findings.

Limitations and outlook

  • The experiments assume data generation dominates training cost and that models are large and trained long enough to absorb added data.Ablations of model size and training duration are reported as robust across setups; the study covers steady-state problems at moderate scale.Source: §6 · p. 9
  • A complete scaling-law analysis remains open because it would jointly vary model size, training compute, data-generation compute, and fidelity mixture across problems and error sources.The authors identify theoretical mixture prediction and active learning as directions that could avoid an expensive combinatorial scaling study.Source: §6 · pp. 9–10

Conclusion

  • At matched test error, multi-fidelity training cuts data-generation compute by an average of ∼3.1× across 20 setups, with peaks up to 6.3×.The figure reports savings in every setup; the approaches converge only at very large budgets.Source: Figure 9 · p. 10, §6 · p. 10
  • Solver-error fidelity is the recommended starting point when resolution invariance is needed, because it outperforms discretization fidelity and avoids variable-resolution meshing.The conclusion says this axis changes a single solver parameter rather than requiring a robust variable-resolution meshing pipeline.Source: §6 · p. 10
  • The largest gains arise at ∼100–500 equivalent high-fidelity samples; at ∼200 on airfoil AB-UPT, relative lift coefficient error falls 54.9% versus the high-fidelity baseline.The discussion places established datasets such as DrivAerML, with ∼500 high-fidelity samples, in this high-gain regime.Source: §6 · p. 10

A Reproducibility Statement

The appendix documents experimental and training details, fidelity-menu design, budgeted sampling, objectives, and test-error measurement. It also discloses AI involvement and describes the Poisson experiment setup.

A Reproducibility Statement

  • The reproducibility appendix covers numerical methods, fidelity menus, architectures, training details, scaling-study data, numeric tables, and design ablations.It describes experiments for Poisson, linear elasticity, hyperelasticity, and airfoil problems.Source: A Reproducibility Statement · p. 18

B AI Disclosure Statement

  • The authors report AI assistance with workflow orchestration, algorithmic coding, editing, literature review, and draft feedback, with human oversight and responsibility.The authors say they read and verified every cited work.Source: B AI Disclosure Statement · p. 18

C.1 Designing the fidelity support

  • The fidelity support Ξ sets the feasible range: its lower endpoint must retain learnable signal, while its upper endpoint should not overshoot the surrogate floor.The experiments target typical endpoint errors of about 20% and 0.5%, respectively.Source: §C.1 · p. 19, Figure 10 · p. 19

C.2 Fidelity normalization and clamping

  • Fidelity normalization uses the error-convergence relation, algebraic or exponential according to the error source, to make behavior consistent across menus.Source: §C.2 · p. 20

C.3 Budgeted sampler

  • The budgeted sampler draws fidelities independently until cumulative cost crosses the budget; λ trades sample fidelity against sample count.Source: §C.3 · p. 20

C.4 Training objective

  • Grid experiments train with mean squared error, whereas mesh experiments use a field-averaged relative L2 error.For a single predicted field, the mesh loss reduces to standard relative L2 error.Source: §C.4 · p. 20

C.5 Test error

  • Test error uses the highest in-support fidelity as reference, absorbing its residual error and isolating the training-mixture effect from fixed ceiling bias.Source: §C.5 · p. 21

D Poisson Experiments

  • The Poisson experiments cover 1D–3D homogeneous-Dirichlet problems, discretized with finite differences and solved by geometric multigrid.Source: D Poisson Experiments · p. 21

D.1 Overview of the Two Types of Errors

The Poisson experiments distinguish spatial discretization error from iterative multigrid error, whose convergence depends on resolution and V-cycle count, respectively. The appendix then describes the linear-elasticity model, geometry, and simulation pipeline.

D.1 Overview of the Two Types of Errors

  • Poisson discretization error decays quadratically with grid points per dimension, while multigrid truncation error decays exponentially with iteration count.The supplied figures show convergence for 1D, 2D, and 3D configurations.Source: §D.1 · p. 21, Figure 11 · p. 22, Figure 12 · p. 22

D.2 Distribution of Right-Hand Sides

  • The right-hand-side distribution activates the first M spectral modes, affecting pre-asymptotic convergence and error offset; higher-dimensional active modes satisfy ∥k∥1 < M.Source: §D.2 · p. 22

D.3 Geometric Multigrid V-cycle

  • Multigrid fidelity is the number K of V-cycles from a zero initial guess, and its per-sample cost is modeled as c(K) = K.The reported Poisson budgets are measured in V-cycles.Source: §D.3 · pp. 22–23

E Linear elasticity experiment

E.1 Governing equations

  • The model uses static linear elasticity with stress related to displacement through Hooke’s law and a plane-stress formulation.The weak form is solved computationally using the finite-element method.Source: §E.1 · p. 23

E.2 Domain and boundary conditions

  • The 2D plate has randomly placed circular holes, is clamped on the left, and receives randomized traction on the right.The experiment uses Young’s modulus 210 GPa and Poisson’s ratio 0.3.Source: §E.2 · p. 23, Figure 13 · p. 24

E.3 Simulation and postprocessing

  • Linear elasticity simulations use P1 triangular finite elements; discretization-error runs use direct LU solves, while solver-error runs vary symmetric Gauss–Seidel fidelity on a fixed mesh.Stress and strain fields are projected onto the continuous P1 space for nodal training values.Source: §E.3 · p. 24

E.4 Sources of error

Linear-elasticity discretization and solver errors follow distinct convergence behaviors, and the appendix proceeds to describe the hyperelasticity model, loading domain, and simulation pipeline. The hyperelasticity setup constrains traction direction to avoid element inversion.

E.4 Sources of error

  • The projected von Mises stress discretization error converges empirically as e(N) = O(N^-0.68), between the theoretical rates for displacement and unsmoothed stress.The stress and strain are derivatives of displacement; projection smoothing partially recovers the lost convergence order.Source: §E.4 · p. 24, Figure 14 · p. 25
  • With the mesh fixed, symmetric Gauss–Seidel solver error converges as e(k) = O(ρ^k), with ρ ≈ 0.99994 in this setup.Figure 15 reports mean and ±1 standard deviation across 90,000 simulations.Source: §E.4 · pp. 24–25, Figure 15 · p. 25

F Hyperelasticity experiment

F.1 Governing equations

  • The experiment uses a compressible Neo-Hookean material in static equilibrium, with stress derived from the deformation gradient and its strain-energy density.The constitutive problem is nonlinear and solved using finite elements and Newton-type methods.Source: §F.1 · pp. 25–26

F.2 Domain and boundary conditions

  • The perforated 2D plate is clamped on the left and loaded on the right; traction is restricted to a ±30° cone to avoid element inversion.The setup uses E = 2 MPa and ν = 0.3 and operates in a mildly nonlinear regime.Source: §F.2 · p. 26, Table 4 · p. 26

F.3 Simulation and postprocessing

  • Hyperelasticity simulations use P1 triangular elements and FEniCSx; discretization runs use full Newton–Raphson, while solver-error runs use a fixed mesh and modified Newton with a frozen Jacobian.Source: §F.3 · p. 27

F.4 Sources of error

The section characterizes discretization and solver error for hyperelasticity, then describes the airfoil simulation setup, governing equations, domain, and postprocessing. The airfoil experiments use steady-state incompressible RANS simulations over varied NACA geometries.

F.4 Sources of error

  • The von Mises stress discretization error converges empirically as e(N) = O(N^-0.70), between the theoretical O(N^-1) displacement and O(N^-1/2) stress bounds.The study uses P1 Lagrange elements on triangular meshes, with average edge length controlling fidelity.Source: §F.4 · p. 27
  • With the mesh fixed at approximately 1,600 nodes, the modified Newton chord solver has empirical convergence rate ρ ≈ 0.95.Its Jacobian is computed at the initial guess and frozen, yielding linear rather than standard Newton–Raphson quadratic convergence.Source: §F.4 · p. 27

G Airfoil experiment

  • The airfoil case models airflow around 2D wings using geometries from NASA NACA 4- and 5-digit series.Geometry parameters vary camber, camber position, and thickness; the far-field boundary is 200 chord lengths away.Source: G Airfoil experiment · p. 28, §G.2 · p. 29

G.1 Governing equations

  • The simulations solve steady-state incompressible RANS equations, with incompressibility justified by the subsonic Mach number below 0.3.The k–ω SST turbulence model supplies closure for turbulent viscosity.Source: §G.1 · p. 28

G.2 Domain and boundary conditions

  • Airfoil geometries and flow conditions are sampled across the ranges summarized for the dataset, excluding problematic near-leading-edge camber values for 4-digit airfoils.The far-field boundary is positioned 200 chord lengths from the airfoil to minimize boundary-condition effects.Source: Table 5 · p. 29, §G.2 · p. 29

G.3 Simulation and postprocessing

  • OpenFOAM cell-centered simulation fields are projected to mesh points by averaging values from incident cells, introducing slight smoothing.The simulations use simpleFoam with SIMPLEC pressure-velocity coupling and fixed outer-iteration counts of 2 × 10^4 or 4 × 10^4.Source: §G.3 · p. 29

G.4 Sources of error

The airfoil experiments vary spatial discretization but do not study solver error. Their solver iterations lack a single clean fidelity parameter with monotonic convergence.

G.4 Sources of error

  • Figure 20 reports airfoil discretization-error convergence using the median and 10th–90th percentiles across 1,627 simulations.The experiments therefore characterize spatial discretization error, not solver error.Source: Figure 20 · p. 30, §G.4 · p. 30
  • Airfoil solver error is excluded because SIMPLEC iterations combine interacting solves and corrections, producing non-monotonic convergence without a well-defined solver-fidelity parameter.All simulations instead use a fixed iteration count, and fidelity is varied only along spatial discretization.Source: §G.4 · p. 30

H Architecture and training details (grid experiments)

  • The grid-based Poisson experiments use FiLM conditioning with regular U-Net and FNO architectures across dimensions 1–3 and both truncation and discretization fidelity axes.The code path is shared across these settings; dimension and error axis enter through array shapes and the cost oracle.Source: H Architecture and training details… · p. 30

H.1 Training objective, optimizer, and schedule

The training setup defines objectives, data pipelines, fidelity menus, cost models, and architecture choices for grid and mesh experiments. Grid and mesh samples use different fidelity constructions and generation workflows.

H.1 Training objective, optimizer, and schedule

  • Grid experiments use per-pixel mean-squared error on whitened outputs, Adam, and a warm-up-cosine schedule with 20,000 total steps.All runs share the loss, whitening, optimizer, and schedule; only the data source differs.Source: §H.1 · pp. 30–31

H.2 Data pipeline

  • Grid samples are either procedurally streamed or cached, with truncation fidelity given by V-cycle count and discretization fidelity by native grid resolution.For discretization samples, both right-hand sides and solutions are interpolated to the common N⋆=64 target grid after solving at native resolution.Source: §H.2 · p. 31

H.3 Fidelity menus, cost model, and swept budget range

  • Grid sample costs are measured in V-cycles for truncation and native grid points for discretization, with menus and normalized budgets specified for the Poisson sweeps.Table 6 gives the fidelity menus and cost units; Table 7 summarizes shared grid-run hyperparameters.Source: §H.3 · p. 31, Table 6 · p. 32, Table 7 · p. 33

I Architecture and training details (mesh experiments)

  • Mesh experiments use Transolver and AB-UPT with sinusoidal fidelity encodings injected into transformer blocks through adaLN modulation.The normalized fidelity tag is encoded with geometrically spaced sine and cosine frequencies.Source: I Architecture and training details… · p. 34

I.1 Data normalization

  • Mesh fields use per-channel z-score normalization, while training minimizes relative L2 error averaged across output fields and, for airfoils, equally across volume and surface.Mesh experiments use AdamW with a one-cycle learning-rate schedule; structural-mechanics and airfoil settings are summarized separately.Source: §I.2 · p. 34, §I.1 · p. 34, Table 11 · p. 37

I.3 Data pipeline

  • Mesh datasets are pre-generated because FEM and CFD simulations are costly, and node subsampling limits training inputs while evaluation remains at full resolution.Transolver training subsamples structural mechanics to 16,000 nodes and airfoils to 32,000 nodes.Source: §I.3 · p. 34

I.4 Fidelity menus, cost model, and swept budget range (linear elasticity)

  • Linear and hyperelasticity sample costs are solver iterations or mesh nodes, whereas airfoil costs are measured CPU core-hours across 11 mesh fidelities.Airfoil fidelity ranges from about 7,600 to 180,000 nodes, with mean simulation times from about 0.9 to 8.3 CPU core-hours.Source: §I.4 · p. 35, §I.5 · p. 35, §I.6 · p. 36

J Additional results

  • The additional-results section reports scaling curves, numerical tables, and computational resources for the experiment families.The airfoil fidelity menu uses empirically measured simulation-time costs, while Tables 11 and 12 summarize hyperparameters for solid mechanics and airfoils.Source: Table 10 · p. 36, Table 11 · p. 37, Table 12 · p. 38

J.1 Poisson experiments

The results section presents multi-fidelity and high-fidelity comparisons across Poisson, hyperelasticity, linear elasticity, and airfoil experiments. It also describes calibration procedures for extrapolating mixture settings and estimating their overhead.

J.1 Poisson experiments

  • Poisson experiments span 1D–3D, truncation and discretization axes, and 5,450 training runs; the supplied figures and tables summarize test-error scaling and compute use.The experiments consumed approximately 1,500 A100-equivalent GPU-hours, dominated by 3D runs; exploratory experiments used another untracked equivalent amount.Source: Table 13 · p. 39, §J.1 · p. 39, Figure 21 · p. 39

J.2 Hyperelasticity experiments

  • Hyperelasticity results report solver- and discretization-error comparisons, with figures showing iso-budget test error and compute-optimal scaling.Tables 17 and 18 summarize training-sample counts and relative changes against high-fidelity-only training.Source: Figure 23 · p. 43, Table 17 · p. 43, Table 18 · p. 44

J.3 Linear elasticity experiments

  • Linear elasticity results report solver- and discretization-error comparisons, with tables summarizing sample counts and error changes against the high-fidelity baseline.Figure 7 presents iso-budget and compute-optimal scaling curves for the experiments.Source: Figure 24 · p. 45, Table 20 · p. 45, Table 21 · p. 46

J.4 Airfoil experiments

  • Airfoil results cover volume-field, surface-field, lift-coefficient, and drag-coefficient errors across the fidelity mixtures.Figures 25 and 26 show iso-budget and compute-optimal scaling; the corresponding tables compare multi-fidelity and high-fidelity results.Source: Table 23 · p. 47, Table 24 · p. 47, Table 25 · p. 47

J.5 Bootstrapping approach

  • The savings ratio compares budgets at which optimal-mixture and high-fidelity-only training reach a specified error, using log-log interpolation between budgets.Confidence intervals use 2,000 seed-bootstrap draws and reflect training randomness, since each cell's sampled dataset is fixed.Source: §J.5 · p. 49

K.1 Fitting the functional form

  • The extrapolation procedure fits a parabola around each budget's local error minimum, then fits the optimal mixture parameter across budgets.It recommends at least three calibration budgets separated by factors of two, shifts calibration upward when minima indicate pre-asymptotic behavior, and caps saturation at λsat = 15.Source: §K.1 · p. 49

K.2 Overhead cost

  • Calibration samples can be reused across plans and folded into deployment data, so overhead depends on samples deployment cannot absorb rather than the sum of all calibration plans.For linear-elasticity solver error, overhead is about 51% of the deployment budget at 2B3 and about 9% at 16B3; sweeping λ directly at deployment costs an additional 170%.Source: §K.2 · p. 50

K.3 Detailed results

Solver-error extrapolation remains reliable, while abrupt Poisson discretization transitions challenge the fitted recipe. The appendix also compares binary and spectrum fidelity mixtures and examines how training budget, mixture balance, and conditioning resolution affect interpolation smoothness.

K.3 Detailed results

  • Regrets stay below 14% in every solver-error cell and do not increase with extrapolation depth, even up to 16× the calibration ceiling.Predictions are evaluated at the nearest swept λ grid cell, so reported regrets include a quantization component.Source: §K.3 · p. 50, Table 27 · p. 50
  • Poisson-1D FNO reaches 48.9% regret and Poisson-2D U-Net reaches 70.3%/99.2% at transition budgets, where the log-linear fit misses abrupt optimal-λ jumps.Once the extrapolated prediction crosses λsat, saturation switches to HF-only deployment, which is optimal after the transition.Source: §K.3 · p. 50, Table 28 · p. 51

L.1 Binary vs. spectrum formulation of fidelity

  • In the solver-error setting, neither fidelity formulation dominates across mixing ratios; spectrum mixing is more robust to the ratio choice, while binary mixing has marginally lower error at its optima.The two binary endpoints are closer in this setting because samples share the same discretization.Source: §L.1 · p. 51, Figure 27 · p. 52
  • The softmax mixture is an expressive but simple choice; alternative distribution families, including interior-peaked mixtures, remain for future study.The framework permits arbitrary parametrizable distributions, but adding parameters enlarges the mixture-hyperparameter search space.Source: §L.1 · p. 51

L.2 On interpolation smoothness

  • At ˜B = 400 interpolation errors deviate markedly, especially above ˆξ > 0.5; at ˜B = 1,600 they are smoother, and a balanced mix at ˜B = 6,400 is smooth across the support.The authors attribute smoothness mainly to sample counts in adjacent fidelity regions and conditioning resolution.Source: §L.2 · pp. 51–52, Figure 28 · p. 52
  • The finest encoding, ℓmin = 2π/1000, gives the lowest test error at ˆξ = 1, while ℓmin = 2π/50 preserves smoother interpolation at a small test-error increase at moderate budgets.Decreasing encoding resolution (larger ℓmin) makes interpolation errors smoother; the main experiments prioritize highest-fidelity test performance.Source: §L.2 · p. 52

L.3 On interpolation errors for the grid experiments

The appendix tests how interpolation, model size, training duration, fidelity-menu endpoints, and test references affect the reported multi-fidelity results. It finds that interpolation error makes grid gains conservative, while endpoint and reference changes leave key comparisons robust within specified regimes.

L.3 On interpolation errors for the grid experiments

  • Removing interpolation error increases rather than decreases the multi-fidelity gain, so multilinear interpolation biases against multi-fidelity training and makes the grid gains conservative.The comparison uses the 1D U-Net Poisson discretization experiment with discrete sine transform interpolation, applicable because its states are bandlimited.Source: §L.3 · pp. 53–54, Table 29 · p. 53

L.4 Model size ablation

  • Multi-fidelity savings remain about 3× as model capacity grows, while the error floor drops from 4% to 1.4%.Update steps are held constant, so wall-clock time and training FLOPs grow with model size.Source: §L.4 · p. 54, Table 30 · p. 54

L.5 Training duration ablation

  • At ˜B = 300, the multi-fidelity error falls from 18.1% at 1,000 steps to 5.9% at 80,000 steps, a 3.07× reduction, versus a 1.09× reduction for HF-only.The authors interpret the smaller HF-only improvement as indicating that its dataset is too small.Source: §L.5 · p. 54, Table 31 · p. 55

L.6 Ablating fidelity menu endpoints

  • The compute-optimal error saturates at ≈1.8% for Kmax ≥ 6, and changing the endpoint from K = 7 to K = 9 changes it by <1% relative.The chosen K = 7 endpoint lies on the compute-optimal envelope, and drawing the endpoint finer buys essentially nothing.Source: §L.6 · pp. 54–55, Table 32 · p. 55
  • Endpoint choice is budget-dependent: a coarser top can be preferable at the smallest budget, while K = 6, 7, and 9 are within ≈3% from ˜Bc = 3,000 onward.An over-specified endpoint costs accuracy at low budget, whereas an under-specified endpoint caps achievable accuracy.Source: §L.6 · p. 55

L.7 On the test reference fidelity

  • With the converged reference, errors are higher but the relative-change shift is at most 0.2 percentage points.The residual reference error also applies to HF-only models.Source: §L.7 · p. 56, Table 33 · p. 56

Related papers

Loading 2610.17221v1…