PaperlayerView exact revision on arXiv

Source-linked AI summary

From Numerical Simulators of PDEs to Neural Emulators and Back

Felix Koehler

arXiv:2608.24547v1cs.LG

TL;DR

Repeated or fast PDE simulation remains costly, while neural emulators are often treated as opaque alternatives to the solvers that generate their training data. This thesis uses Fourier-spectral analysis to connect the two and concludes they are more alike than different.

  • Problem

    The cost of numerical PDE solvers remains a bottleneck when fast or repeated evaluations are needed, while neural emulators are often treated as opaque alternatives to their training-data generators.

    Source: Zusammenfassung · p. 5, §1.1 · p. 17

  • Method

    The thesis uses mode-wise Fourier-spectral analysis to relate solver errors, emulator architectures, and training, alongside systematic benchmark and differentiable-physics studies.

    Source: §6.1.1 · p. 59, §7.2.1 · p. 66, §1 · p. 83

  • Results

    The thesis concludes that numerical solvers and neural emulators are more alike than different: emulator inductive bias can regularize solver errors, and PDE-suited architectures tend to perform best.

    Source: §6.4 · p. 63

  • Takeaways & Limitations

    For emulator design, match architecture to problem characteristics and use unrolled training particularly for challenging problems or limited receptive fields.

    Source: §7 · p. 91

    The theoretical analyses and experiments focus on semi-linear PDEs with periodic boundaries on uniform Cartesian grids, excluding Dirichlet or Neumann boundaries and complex geometries.

    Source: §6.2 · p. 60

Abstract

from arXiv · show

Simulation is central to modern engineering and science, but the cost of numerical solvers for partial differential equations (PDEs) remains a bottleneck whenever fast or many-query evaluations are required. Neural emulators trained on solver-generated data promise significant speedups, yet they are usually framed as opaque alternatives to the very methods that produce their training signal. This thesis argues the two paradigms are more alike than different: neural architectures mirror classical discretizations, their errors are amenable to the same spectral analysis, and insight flows profitably in both directions. We approach the relationship by disentangling the multiple roles a solver plays in the emulator learning pipeline. Mode-wise Fourier analysis then provides a common language in which solver errors, architectural inductive biases, and training objectives can all be read off simultaneously. Taken together, this allows synthesizing three contributions. (1) APEBench, a comprehensive benchmarking suite for autoregressive neural emulators of PDEs that uses fast differentiable pseudo-spectral solvers in JAX. (2) Progressively Refined Differentiable Physics, an investigation of the effect of unconverged solvers on surrogate training. (3) Neural Emulator Superiority, an analysis of the influence of numerical errors and architectural inductive biases.

1. Prof. Dr.-Ing. Nils Thürey · 2. Assistant Prof. Dr. William de Almeida Gilpin (schriftliche Beurteilung) · 2. Prof. Dr. Riccardo Marin (mündliche Prüfung)

Die Dissertation verbindet numerische PDE-Löser und neuronale Emulatoren durch die These, dass ihre Architekturen und Fehler mit gemeinsamen spektralen Werkzeugen untersucht werden können. Sie stellt drei Arbeiten zu Benchmarking, nicht vollständig konvergierten Lösern und numerischen Fehlern sowie architektonischen Biases vor.

2. Prof. Dr. Riccardo Marin (mündliche Prüfung)

  • Die Dissertation wurde am 21.04.2026 an der Technischen Universität München eingereicht und am 02.07.2026 durch die TUM School of Computation, Information and Technology angenommen.Source: §2 · p. 1

Zusammenfassung

  • Numerische PDE-Löser bleiben bei schnellen oder vielfach wiederholten Auswertungen ein Kostenengpass, während neuronale Emulatoren auf Simulatordaten erhebliche Geschwindigkeitsgewinne versprechen.Source: Zusammenfassung · p. 5
  • Die Arbeit argumentiert, dass neuronale Architekturen klassische Diskretisierungsverfahren widerspiegeln und Löser- wie Emulatorfehler mit derselben spektralen Analyse untersucht werden können.Source: Zusammenfassung · p. 5
  • Sie trennt die verschiedenen Rollen numerischer Löser im Emulator-Trainingsprozess und nutzt modenweise Fourier-Analyse, um Löserfehler, architektonische Biases und Lernziele gemeinsam zu erfassen.Source: Zusammenfassung · p. 5
  • APEBench ist eine Benchmark-Suite für autoregressive neuronale PDE-Emulatoren, die auf schnellen differenzierbaren pseudo-spektralen Lösern in JAX basiert.Source: Zusammenfassung · p. 5
  • Progressively Refined Differentiable Physics untersucht, wie nicht vollständig konvergierte Löser das Training neuronaler Surrogate beeinflussen.Source: Zusammenfassung · p. 5
  • Neural Emulator Superiority analysiert den Einfluss numerischer Fehler und architektonischer induktiver Biases.Source: Zusammenfassung · p. 5

List of Publications

Three publications form the basis of the dissertation, covering APEBench, progressively refined differentiable physics, and neural emulator superiority. Asterisks mark authorship, with core factors of 1 for sole first authorship and 0.5 for shared first authorship.

APEBench: A Benchmark for Autoregressive Neural Emulators of PDEs

  • APEBench: A Benchmark for Autoregressive Neural Emulators of PDEs lists Felix Koehler*, Simon Niedermayr, Rüdiger Westermann, and Nils Thuerey.The thesis describes APEBench as a framework for systematic benchmarking of neural PDE emulators.Source: APEBench: A Benchmark for… · p. 15, §1.3 · p. 19

PRDP: Progressively Refined Differentiable Physics

  • PRDP: Progressively Refined Differentiable Physics lists Kanishk Bhatia*, Felix Koehler*, and Nils Thuerey.The thesis describes progressively refined differentiable solvers as a way to train emulators efficiently without fully converging iterative solvers.Source: PRDP: Progressively Refined… · p. 15, §1.3 · p. 19
  • Neural Emulator Superiority: When Machine Learning for PDEs Surpasses its Training Data lists Felix Koehler* and Nils Thuerey.The thesis studies how numerical errors affect surrogate quality and reports that emulators can sometimes exceed the fidelity of their training data.Source: PRDP: Progressively Refined… · p. 15, §1.3 · p. 18

2. Foundations of Numerical Simulation

This section lays out the foundations of PDE simulation, covering spatial discretization, time integration, and implicit solution methods. It also connects numerical solvers and neural emulators through spectral error analysis, differentiable implementations, and architectures reflecting classical discretizations.

2. Foundations of Numerical Simulation

  • The method-of-lines pipeline discretizes a PDE in space, then applies an evolution operator that incorporates spatial derivatives and time integration to advance the state.The operator is also expected to support batching, efficient accelerator execution, and differentiation for optimization or hybrid emulators.Source: Figure 2.1 · p. 22, §2 · p. 22

2.1.1 Spatial Discretization

  • Spatial discretization specifies domain representation, derivative approximation, and boundary conditions; periodic grids wrap the endpoint, while Dirichlet and Neumann grids use interior nodes.Higher-dimensional grids are built by tensor products.Source: §2.1.1 · p. 22

Finite Difference Methods

  • Finite differences approximate derivatives with local stencils, yielding polynomial-order error; padding must match the boundary condition, using circular, zero, or reflection padding.Axis-wise derivatives can be composed to form vector-calculus operations and discretized nonlinear operators.Source: Finite Difference Methods · p. 23

Spectral Methods

  • Fourier-spectral methods transform periodic states to coefficient space for global differentiation, achieving exponential convergence for smooth, bandlimited functions but less efficiency on sharp or discontinuous states.Derivatives are exact for bandlimited states; the approach relies on Fourier transforms and pseudo-spectral evaluation uses dealiasing for nonlinearities.Source: Figure 2.2 · p. 24, Spectral Methods · p. 24

2.1.2 Temporal Discretization

  • With fixed time steps, explicit schemes advance from the current state and may require stability-limited step sizes, while implicit schemes impose a constraint on the next state.Implicit methods can be unconditionally stable in theory, but may require difficult linear or nonlinear solves and introduce more numerical diffusion than explicit schemes for hyperbolic problems.Source: §2.1.2 · p. 25, Explicit Schemes · p. 25, Implicit Schemes · p. 25

2.2 Implicit Integrators, Stationary Problems and Iterative Solves

  • Implicit stepping and stationary PDE solving both seek a state with zero residual; linear solver options include relaxation, Krylov, and multigrid, which can be combined.Multigrid can achieve optimal O(N^D) complexity for elliptic problems such as Poisson; as the implicit time step tends to infinity, the integrator reduces to a stationary solver.Source: Implicit Schemes · p. 25, §2.2 · p. 26, §2.2.1 · p. 27

Exponential Time Differencing (ETDRK)

  • ETDRK treats the linear PDE component exactly with an integrating factor and approximates the nonlinear component by quadrature, enabling efficient mode-wise exponentials for Fourier-spectral solvers.Higher-order quadrature methods tend to improve accuracy; mode-wise evaluation underpins the Exponax solvers used by APEBench.Source: Exponential Time Differencing (ETDRK) · pp. 25–26

2.2.2 Nonlinear Solvers

  • For nonlinear root finding, the initial guess affects both convergence speed and which root is found, so time-stepping solvers naturally warm-start from the previous state.Minimizing the residual norm is a simple approach but typically converges slowly.Source: §2.2.2 · p. 27

2.2.2 Nonlinear Solvers; Newton-Raphson; Picard Iteration

  • Newton-Raphson linearizes the residual and converges quadratically near the root, whereas Picard iteration alternates matrix assembly and linear solves with linear convergence.Picard drops certain Jacobian terms but offers explicit control over matrix assembly and sparsity; quasi-Newton methods such as Broyden trade quadratic convergence for lower per-iteration cost.Source: Newton-Raphson · pp. 27–28, Picard Iteration · p. 28

2.2.3 Differentiating Through Iterative Solvers

  • For iterative solvers, differentiating through K iterations is possible under contractivity and smoothness, but differentiating the converged solve is often preferable.Implicit differentiation yields a linear solve whose cost does not depend on the number of forward iterations, enabling custom forward- or reverse-mode rules.Source: §2.2.3 · p. 28

2.3 Spectral Error Analysis

  • BTCS always under-attenuates diffusion, while FTCS’s error sign depends on γ2 and mode ϕ; both are exact at ϕ = 0 and deviate most near ϕ = 1/2.BTCS is unconditionally stable. Fourier analysis on periodic grids compares each scheme’s mode multiplier with the analytical multiplier, exposing discretization error mode by mode.Source: §2.3 · p. 29

2.4 Modern Implementations: Solvers in Machine Learning Frameworks; 2.4.1 Why JAX?

  • Reimplementing solvers in machine-learning frameworks enables accelerator portability and native autodiff, while JAX adds vectorization, time-step scans, higher derivatives, reproducibility, and compiled kernels.This avoids the two-language barrier of offline solver-generated datasets, but reimplementing or differentiably wrapping established solvers is difficult, and some algorithms do not fit tensor-style frameworks easily.Source: §2.4 · p. 30, §2.4.1 · pp. 30–31

2.4.2 Exponax: Differentiable Spectral Solvers

  • Exponax implements Fourier-spectral ETDRK solvers in JAX for periodic semi-linear PDEs, using optimized FFTs and vectorization to generate large datasets quickly on accelerators.With no implicit solves, autodiff has no iterative convergence tolerances affecting gradient quality; substeps also support moderately nonlinear systems such as low-to-medium-Reynolds-number Navier–Stokes.Source: §2.4.2 · p. 31

2.4.3 Picardax: Differentiable Finite Difference Solvers

  • Picardax implements differentiable implicit time integrators in JAX and enables controlled studies of how under-converged solves and selective gradient cuts affect training signals.It supports θ-method stepping and configurable Picard convergence criteria, including deactivating backpropagation through the assembly routine Λ.Source: §2.4.3 · p. 31

2.4.4 Chaotax: Inferring Dynamical System Properties with Autodiff in JAX

  • Chaotax uses autodiff-enabled Exponax and Picardax solvers to compute Lyapunov properties, which characterize dynamics and inform evaluation horizons for autoregressive emulators.The largest exponent can be approximated by integrating a tangent-linear perturbation alongside the primal system; a full spectrum helps classify dynamics and characterize attractors.Source: §2.4.4 · pp. 31–32

Neural Emulators for PDEs; 3.1 Neural Architectures as Learned Discretizations; 3.1.1 Convolutional Networks and Finite Difference Stencils

  • The neural emulators considered are full-field surrogates with solver choices fixed, and a single linear, bias-free convolution with kernel size 3 is a finite-difference stencil with learned coefficients.CNN depth, channel mixing, and nonlinearities extend this stencil-like operation to couple fields and represent nonlinear operators.Source: Neural Emulators for PDEs · p. 33, §3.1.1 · p. 33

3. Neural Emulators for PDEs

Neural PDE emulators echo classical discretizations, while distinct solver roles across data generation, training, and evaluation shape what they learn and how they are judged.

3.1.2 Fourier Neural Operators and Spectral Methods

  • FNO layers learn mode-wise Fourier multipliers like spectral differentiation, but truncating higher modes creates an inherent low-pass filter and their Fourier basis assumes periodic boundaries.FNOs typically retain only the lowest M ≈12–20 modes; truncation can reduce Gibbs effects on non-periodic problems, while padding or GINO offers alternatives.Source: §3.1.2 · p. 34

3.1.3 UNets as Multigrid Methods

  • UNets grow receptive fields through resolution hierarchies that parallel multigrid: convolution blocks smooth, downsampling restricts, upsampling prolongs, and skip connections carry fine-scale information.At four coarsening levels, one size-3 convolution spans 16 grid points, whereas a flat ConvNet would need 16 layers.Source: §3.1.3 · p. 35

3.1.4 Transformer-based Architecture

  • Vanilla attention mixes globally but costs O(N^2D) and encodes little physical structure; ViT patches and SWIN windows reduce attention cost, while whether physical biases matter at scale remains open.SWIN shifts local windows between layers to obtain global connectivity over depth.Source: §3.1.4 · pp. 35–36

3.1.5 Other Types of Architectures

  • Other architectures expand connectivity through dilated convolutions, internal iterative refinement, or graph message passing, with graph hierarchies extending multilevel processing to unstructured meshes.Exponentially growing dilation rates can cover the full domain in logarithmically many layers.Source: §3.1.5 · p. 36

3.2 Neural-Hybrid Approaches

  • Neural-hybrid methods combine a coarse simulator with a learned correction or interaction, leveraging solver structure; differentiable training may use unconverged solvers to save computation.The neural component can correct solver outputs, interact with internal states or numerics, or provide a learned forcing term.Source: §3.2 · pp. 36–37

3.3.1 Synthetic Data Generation and Training Configurations

  • Training typically matches consecutive solver states, while unrolled objectives compare multi-step trajectories; warmup selection and input noise can target physical regimes and reduce rollout distribution shift.Unrolled training requires backpropagation through the autoregressive chain; the Markovian H = C = L = 1 setup is a common baseline.Source: §3.3.1 · pp. 37–38

3.3.2 Different Aspects of Generalization

  • Evaluation distinguishes temporal, state-space, metric, and solver generalization; long rollouts can confound temporal and state-space shifts, and emulator performance can depend on spectral composition.Beyond these four axes, tests may vary resolution, constitutive parameters, or the PDE system.Source: §3.3.2 · pp. 38–39

The Solver-Emulator Relationship

  • The chapter links three puzzling observations to distinct solver roles: emulators can outperform their training source, training solvers need not converge, and benchmark verdicts can depend on unstated assumptions.It formalizes five solver roles and their fidelity relationships, including training-data sources and evaluation references.Source: The Solver-Emulator Relationship · p. 41, §4.1 · p. 41

4. The Solver-Emulator Relationship

This section examines the roles numerical solvers play in emulator training, evaluation, and benchmarking, including how solver errors and convergence relate to emulator architectures and training. It also considers benchmark accuracy in relation to computational cost and the solver being replaced, and introduces APEBench with procedurally generated data.

4. The Solver-Emulator Relationship

  • Solver roles distinguish data generation, reference solutions, in-loop physics, structural analogues, and baseline comparators; roles can collapse across setups, while fixed datasets may hide the reference and data-generating solver.The structural-analogue role is conceptual rather than computational and can be absent when an architecture lacks a clear classical counterpart.Source: §4 · pp. 42–43

4.2.1 Spectral Error Analysis of Learned Emulators

  • An FTCS-form emulator trained on BTCS data can lie closer to the analytical solution than its training source beyond the fitted mode, as its inductive bias acts as implicit regularization.The spectral example is restricted to linear PDEs and linear ansätze; fitting the same form to FTCS data instead recovers the FTCS parameter exactly.Source: §4.2.1 · pp. 43–44, Figure 4.2 · p. 44

4.2.2 From State-Space to Autoregressive Superiority

  • Autoregressive superiority can emerge after the first step even on identical train and test distributions, and appeared across tested architectures; state-space superiority was mainly observed for local convolutional architectures.Superiority depends on mismatches between emulator bias and solver errors, between training and test conditions for state-space gains, and on the training solver being sufficiently inaccurate.Source: §4.2.2 · p. 45, Figure 4.3 · p. 45

4.3.1 The Bilevel Optimization Perspective

  • In differentiable physics training, the iterative solver is an inner optimization problem, and early training can use coarse gradients before progressively increasing solver iterations as learning matures.Full convergence is unnecessary even late in training: minibatch noise, optimizer behavior, and the network’s approximate accuracy can make further refinement beyond Kmax yield no measurable accuracy gain.Source: §4.3.1 · p. 46, §4.3.2 · p. 46, §4.3 · p. 46

4.3.3 The PRDP Algorithm and Computational Savings

  • PRDP cuts cumulative solver iterations by up to 86% versus fully converged physics, with corresponding wall-clock reductions and lower-cost inference when deployed at Kmax.It starts with a coarse solver, increases iterations when validation loss plateaus, and stops refining when further steps do not meaningfully improve validation loss.Source: §4.3.3 · p. 47

4.4 Implications for Benchmarking

  • Benchmarks should use high-fidelity references, configurable data generators, and rollout metrics, because training-source errors can confound evaluation and autoregressive errors can change model rankings.For chaotic systems, rollout comparisons should account for the Lyapunov time scale rather than compare only a fixed number of steps.Source: §4.4 · p. 48
  • Benchmarking should specify the emulated system and metric reductions, compare multiple seeds with distributional summaries, separate architecture from training effects, and report computational efficiency.Even nRMSE is ambiguous without specifying axis reduction order and whether normalization is per channel or global.Source: §4.4 · pp. 48–49
  • Benchmarks should report emulator accuracy relative to computational cost and compare it with the solver it is meant to replace.A meta-analysis by McGreivy and Hakim (2024) finds that reported speedups often shrink or vanish against properly optimized classical solvers; the five-role decomposition exposes when solver roles are collapsed onto an under-specified baseline.Source: §4.4 · p. 49

Systematic Emulator Benchmarking

  • APEBench supports linear, nonlinear, and reaction-diffusion PDE dynamics, including advection, Burgers, and Gray-Scott.Figure 5.1 shows a qualitative subset of the supported dynamics in two-dimensional spatiotemporal plots.Source: Figure 5.1 · p. 51

5.1.1 Design Principles

  • APEBench treats the solver as a first-class component rather than a black-box data source and is built around four design principles.APEBench (Koehler et al., 2024) is presented as realizing the benchmarking desiderata described in Section 4.4.Source: §5.1.1 · p. 51
  • APEBench procedurally generates training and evaluation trajectories from Exponax, enabling reproducible datasets of arbitrary size without storage overhead.A single 1D trajectory takes milliseconds on a modern accelerator, and experiments are exactly reproducible from a random seed.Source: §5.1.1 · p. 51

5. Systematic Emulator Benchmarking

APEBench makes autoregressive PDE emulator research systematic through configurable solver-generated experiments, a shared training taxonomy, and standardized evaluation. Its experiments show how emulator performance depends on PDE difficulty, architecture, and training methodology, while the thesis connects those findings to numerical analysis.

5.1.2 46 PDE Configurations with Normalized Identifiers

  • APEBench provides 46 PDE configurations across linear, nonlinear, and reaction-diffusion dynamics, with normalized identifiers and controllable physical parameters.Linear periodic problems can be solved with ETDRK without spatial discretization error, isolating emulator error; the benchmark excludes high-Reynolds-number flows and Euler equations, but covers core phenomena including transport, incompressibility, dispersion, chaos, and polynomial nonlinearities.Source: §5.1.2 · pp. 52–53

5.1.3 The Training Taxonomy

  • APEBench’s (T, B) training taxonomy distinguishes precomputable targets when B = T from differentiable solver branches required when B < T.The taxonomy also supports neural-hybrid emulators; for T ≥ 2, their coarse solver must be differentiable because it lies in the autoregressive chain.Source: §5.1.3 · pp. 53–54

5.1.4 Metrics and Reduction Orders

  • APEBench defines metrics by their reduction order and reports per-channel normalized RMSE, full rollout trajectories, and geometric-mean temporal summaries.Per-channel normalization prevents weak channels from being masked, while the geometric mean is less sensitive to late-time divergence; multi-seed scalar summaries use medians and interquartile ranges.Source: §5.1.4 · pp. 54–55, Figure 5.3 · p. 54

5.1.5 Software Ecosystem in JAX

  • The JAX ecosystem combines Exponax solvers, PDEquinox architectures, and Trainax training configurations into composable experiments, with SuperAPE enabling comparisons across solver families.APEBench exposes solver parameters and differentiable physics; vectorizing over seeds makes a 50-seed 1D sweep roughly as costly as a single run. APEBench complements PDEBench and PDEArena with emphasis on autoregressive generalization, fine-grained solver control, and systematic training methods.Source: §5 · p. 52, §5.1.5 · pp. 55–56, §5.2 · p. 56

5.3 Systematic Investigation of the Emulation Pipeline

  • Across 46 PDEs, no architecture was universally best: ResNets were most consistent, FNOs excelled on global structure, and local convolutions on high-frequency reaction-diffusion patterns.In 1D advection, accuracy degraded as difficulty increased when local architectures’ receptive fields failed to cover propagation per time step; FNOs were largely insensitive to difficulty. UNets gained an advantage over dilated ResNets in higher dimensions. Local convolutional models converged with as few as five training trajectories, while FNOs required more data; training horizon mattered most for multi-stage dynamics. The full experimental suite took approximately 900 GPU-hours on 8× RTX 2080 Ti.Source: §5.3 · pp. 57–58
  • In 2D advection, diverted-chain training (B = 1, T > 1) tended to outperform supervised unrolling (B = T) for local architectures.The approach combines autoregressive exposure with one-step reference corrections.Source: §5.3 · p. 57

6.1.1 The Spectral Lens as a Unifying Framework

  • Across the thesis, mode-wise Fourier analysis links numerical-solver errors, emulator behavior, and learned spectral time steppers, enabling analysis of when emulators exceed their training source.The analysis uses periodic, semi-linear PDEs as a tractable setting; the thesis argues the mechanisms can transfer to more complex settings even when spectral analysis cannot.Source: §6.1 · p. 59, §6.1.1 · p. 59

6.1.2 The Receptive Field–Domain of Dependence Connection

  • For PDEs with a clear domain of dependence, local convolutional emulators perform well when their receptive fields cover the physics’ per-step propagation, mirroring the CFL condition for numerical stencils.This receptive-field connection emerged as one of the most important links between architecture choice and PDE difficulty in APEBench.Source: §5.3 · p. 57, §6.1.2 · p. 59

6. Discussion and Outlook

The discussion argues that numerical solvers and neural emulators can be understood in a shared analytical framework, while identifying limits and directions for extending that work. The APEBench summary illustrates this approach through a broad benchmark and solver-enabled training methods.

6.1.3 Representational Capacity Limits

  • Neural emulators typically reach ≈10^-2 nRMSE, with best cases near ≈10^-3, so solver convergence to machine precision does not ensure accurate PDE representation or provide precision the network can exploit.Although theory permits arbitrary accuracy with sufficient capacity, practice is limited by spectral biases, autoregressive distribution shifts, and optimization; errors may arise from discretization, modeling, or learned approximation.Source: §6.1.3 · p. 60

6.1.4 Autoregressive Generalization and Distribution Shifts

  • Autoregressive errors eventually compound on sufficiently challenging problems, but stability varies: stationary or steady-state dynamics can permit long rollouts, and learned dynamics may accumulate errors more favorably than their solver.Non-periodic, non-stationary dynamics are especially challenging; deployment rollout length should match the finite prediction horizon, and unrolled training can extend the temporal receptive field.Source: §6.1.4 · p. 60, §6 · p. 60

6.2 Limitations and Open Questions

  • The thesis is limited to semi-linear PDEs on periodic uniform grids with fast reference solvers, full-field nRMSE, single-solver emulators, and five architecture families, leaving derived quantities and several training strategies unexamined.Open questions include training-versus-testing rollout horizons, instability under non-stationary dynamics, inaccurate gradients, transfer learning, data augmentation, architecture search, and scaling laws.Source: §6.2 · pp. 60–61

6.3.1 Foundation Models for PDEs

  • Pretraining could amortize data generation and training into universal emulators, while online generation and active learning could make solver-produced trajectories responsive to training.Pretraining across heterogeneous solvers complicates the meaning of a single evaluation reference and baseline comparator.Source: §6.3.1 · p. 61

6.3.2 Compute-Optimal Data Generation

  • Because current emulators cannot exploit solver accuracy beyond ≈10^-2, training data could mix solver fidelities and condition the emulator on fidelity instead of using only fully converged data.The compute-optimal balance among solver fidelity, trajectory count and length, and model capacity remains unanswered; high-fidelity datasets can cost thousands of CPU-hours.Source: §6.3.2 · pp. 61–62

6.3.3 Beyond Structured Grids

  • Extending the work beyond structured grids requires non-Cartesian differentiable solvers, while JAX-based solvers show how autodiff and accelerators can benefit both emulator workflows and classical inverse problems.Spectral interpolation can project structured data onto unstructured points, but dense-tensor frameworks suit unstructured solvers poorly; automated GPU translation of solver code is promising, though full-pipeline scalability remains uncertain.Source: §6.3.3 · p. 62, §6.3.4 · pp. 62–63

6.4 Conclusion

  • Numerical solvers and neural emulators are more alike than different: emulator inductive bias can regularize solver errors, while architectures resembling suitable numerical methods tend to perform best.The thesis’s structured, periodic-grid setting enabled precise mechanism isolation, though extending findings to complex boundaries, unstructured meshes, and foundation models remains an expectation rather than a demonstrated result.Source: §6.4 · p. 63

7.1.2 Key Experiments and Findings

  • Across 46 PDEs, the best architecture tends to match the corresponding numerical method: local convolutions excel on reaction-diffusion, FNOs on global structure, and ResNets are most consistent.The benchmark also finds that emulators fail when their receptive field cannot cover the PDE’s domain of dependence; diverted-chain training combines rollout stability with one-step accuracy.Source: §7.1.2 · p. 65

7. Summary of Publications … H.1 General Details

This section presents APEBench and related studies of how numerical solvers and neural emulators interact in PDE emulation. It covers differentiable Fourier-based solver methods, benchmark interfaces and experimental setups, and investigations of solver refinement, training strategies, and architectural inductive biases.

7.1.3 Impact and Adoption

  • APEBench’s full experimental suite costs about 900 GPU-hours on eight RTX 2080 Ti GPUs, and its solver and framework have supported later data generation and research.Exponax has been translated to PyTorch and used for PDE-Transformer and P3D pretraining datasets; APEBench has also supported theses on foundation models and architecture comparison.Source: §7.1.3 · p. 66

7.2.1 Algorithm Description

  • PRDP progressively increases solver iterations as training plateaus and stops refining when validation no longer improves, but requires an iterative solver in the computational graph.Progressive-refinement savings apply universally; incomplete-convergence savings depend on problem structure, occurring in network-limited settings but not simple double-convex Poisson inverse problems, and nonlinear-solver extensions remain unvalidated.Source: §7.2.1 · p. 66

7.2.2 Results and Savings Achieved

  • PRDP matches fully converged-physics accuracy while cutting training time by 62% in 2D Navier–Stokes and solver iterations by up to 86%.The validation spans Poisson inverse problems, 1D and 2D heat emulators, Burgers, and a Navier–Stokes neural-hybrid setting; results were validated over multiple seeds (50 in 1D, 20 in 2D and 3D).Source: §7.2.2 · pp. 66–67, §7 · p. 66

7.3.1 Theoretical Results

  • Linear emulator ansätze act as implicit regularizers: advection and diffusion show forward superiority at higher frequencies, while unconverged Richardson Poisson shows backward superiority at low frequencies.The analysis fits ansätze corresponding to first-order upwind advection, FTCS diffusion, and a second-order Laplacian for Poisson, deriving their learned amplification factors in closed form.Source: §7.3.1 · p. 67

7.3.2 Empirical Validation

  • Autoregressive superiority appears across all tested architectures, but state-space superiority occurs only in local ConvNets and ResNets, and strengthens with single-mode training.Tests cover ConvNets, ResNets, FNOs, UNets, and Transformers on diffusion, advection, and Burgers; global architectures do not show state-space superiority under the same distribution shift.Source: §7.3.2 · p. 68

7.3.3 Implications for the Field

  • Emulator superiority means training-data fidelity is not an accuracy ceiling; benchmarking against the training solver can hide it, so evaluation needs high-fidelity references.APEBench uses spectral reference solutions to separate emulator error from solver error.Source: §7.3.3 · p. 68

Abstract

  • APEBench provides a JAX pseudo-spectral framework for 46 PDE dynamics across 1D–3D, with differentiable simulation, training taxonomy, dynamics identifiers, and rollout-focused metrics.Its integrated solver supports procedural data generation, differentiable-physics training, and neural-hybrid emulators; procedural datasets can be regenerated deterministically in seconds.Source: Abstract · p. 82, §1 · p. 83

2 From Classical Numerics to Learned Emulation

  • In advection, learned linear-convolution emulators beat the first-order upwind scheme for the first 13 steps, while longer unrolled training improves later rollout performance.Twenty-step unrolled training performs better after 30 steps but has 11% higher first-step error; increasing training rollout length moves the learned stencil closer to the upwind scheme.Source: §2 · p. 84, Figure 2 · p. 84

4 Components of the APEBench benchmark

  • APEBench’s ETDRK solvers integrate linear PDE components exactly under periodic boundaries, while normalized dynamics and difficulty coefficients help identify and scale linear dynamics.The solver suite is differentiable and GPU-suited; the reduced identifiers use N, D, and coefficient values to distinguish dynamics and their difficulty.Source: §4 · p. 86, §B.1 · p. 96
  • APEBench unifies emulator training by unrolling length and reference-branch length, and supports both diverted-chain training with a differentiable reference solver and neural-hybrid correction using a coarse solver.One-step training and supervised unrolling are special cases of the taxonomy; the framework also supports altered backpropagation.Source: §4 · p. 87, §5.2 · p. 88

5.1 Bridging Numerical Simulators and Neural Emulators

  • Local convolutional emulators fail when their receptive fields cannot meet advection difficulty, whereas FNO performance is largely difficulty-agnostic and unrolling can substantially improve ResNet temporal generalization.ResNets are on par with FNO at low difficulty; additional training-time unrolling improves ResNet performance even at the highest difficulty.Source: Figure 3 · p. 88, §5.1 · p. 88

5.2 Diverted Chain: A Learning Methodology with A Differentiable Fine Solver

  • For nonlinear 1D PDEs, rollout training improves long-term accuracy, especially for KdV; diverted-chain training further improves KdV long-term accuracy while retaining near one-step short-term performance.Diverted-chain training does not surpass supervised unrolling on Burgers or KS long-term accuracy.Source: Figure 4 · p. 89, §5.2 · p. 89

5.3 Neural-Hybrid Emulators and Sequential Correction

  • On 2D advection, supervised unrolling improves ResNet and ResNet-hybrid performance over FNO, while ResNet performs best when correcting a coarse solver handling 50% of the difficulty.FNO changes little with unrolling or task difficulty; ResNet benefits significantly from unrolling.Source: Figure 5 · p. 89, §5.3 · p. 89

5.4 Relating Architectures and PDE Dynamics

  • ResNets perform consistently well across dynamics and dimensions, while architecture fit varies: local networks suit reaction-diffusion, long-range models help hyper-diffusive 1D PDEs, and FNO excels on Navier-Stokes Kolmogorov flow.For linear cases, local architectures excel on 1D hyperbolic dispersion, while UNets are consistent on 3D unbalanced advection and perform well on 2D anisotropic diffusion.Source: Figure 6 · p. 90, §5.4 · p. 90
  • Burgers emulation worsens exponentially with dimension, but ResNets remain top-performing; increasing resolution can improve emulation until architectures are fully utilized, while the 3D UNet benefits from its global receptive field.Dilated ResNets lose performance in 3D, and the FNO struggles on the resolution-adapted 2D KS example because the spectrum is fully populated at both resolutions.Source: §5.4 · pp. 90–91

7 Conclusions and Outlook

  • The authors conclude that architecture fit, unrolling, and hybrid approaches improve PDE emulation, while identifying broader solver and discretization support and deeper study of unrolling as future work.They also release Exponax, PDEquinox, and Trainax as separate software packages.Source: §7 · p. 91, §6 · p. 91, A How to use APEBench · p. 96

B.1 Motivation and Background

  • For stable constant-coefficient linear PDEs with periodic boundaries and bandlimited initial data, Fourier spectral integration is exact in space and time and permits arbitrarily large time steps.Fourier space diagonalizes derivative operators, reducing them to elementwise multiplication. Nonlinear terms are instead evaluated pseudo-spectrally, with ETDRK stages approximating their integral.Source: §B.1 · pp. 98–100
  • For nonlinear PDEs, pseudo-spectral evaluation computes nonlinear terms in state space via inverse and forward Fourier transforms, while ETDRK stages approximate their time integral.The framework also includes stable contour-integral coefficient computation and dealiasing to limit numerical instabilities and aliasing effects.Source: §B.1 · pp. 99–101

B.2 ETDRK Methods in other Software Libraries

  • The JAX solver framework succeeds the ChebFun spinX.m ETDRK module with vectorization, backend-agnostic execution, and automatic differentiation for deep-learning workflows.The MATLAB-based ChebFun implementation was used to generate data in earlier physics-based deep-learning studies, but is difficult to call from Python and does not support differentiation through the solver.Source: §B.2 · p. 101

B.3 Limitations of Fourier Pseudo-Spectral ETDRK Methods

  • Fourier pseudo-spectral ETDRK methods remove linear stiffness for semi-linear PDEs, but require periodic uniform grids, smooth bandlimited solutions, and first-order-in-time dynamics.They also assume no channel mixing in the linear part and are less advantageous when nonlinear stiffness dominates; they do not apply directly on non-Cartesian domains such as spheres. When problem constraints fit, the approach is highly efficient on GPUs and integrates with automatic differentiation.Source: §B.3 · p. 102

B.4 Limitations of the Implementation

  • The implementation supports equal domain extents and grid counts across dimensions, real-valued PDEs, constant time steps, and only ETDRK0 through ETDRK4.The fixed step size aligns with autoregressive emulator architectures, while restricting to real-valued problems enables use of the real-valued FFT.Source: §B.4 · pp. 102–103

B.5 Linear and Nonlinear Differential Operators

  • The package defines solvers through Fourier-space linear and nonlinear differential operators, allowing custom operators and dealiasing while excluding channel mixing in the linear operator.Linear operators can encode arbitrary derivative orders, coefficients, and spatial mixing; available nonlinearities include convection, gradient norm, polynomial, and vorticity convection forms.Source: §B.5 · p. 103

B.6 Generic Time Steppers for General Dynamics

  • Generic steppers represent isotropic linear operators as derivative-order coefficients and combine them with linear, convection, gradient-norm, polynomial, vorticity-convection, or broader nonlinear dynamics.The isotropic representation excludes spatial mixing such as cross-derivatives.Source: §B.6 · p. 104

B.7 Normalized Dynamics: A Unifying Framework

  • Normalized dynamics reduce PDE parameters to coefficients governing time-discrete behavior, including derivative-order ratios and polynomial nonlinearities, enabling targeted emulator evaluation.For example, diffusion is governed by ν∆t/L^2; polynomial nonlinearities without additional derivatives normalize by L^0 = 1. Burgers is expressible as a special case of the general convection stepper.Source: §B.6 · p. 104, §B.7 · p. 105

B.8 Difficulty of Emulating Dynamics: Bridging the Continuous and Discrete

  • Emulation difficulty depends on normalized dynamics together with spatial resolution and dimension, with difficulty factors measuring proximity to explicit finite-difference stability limits.Identifying a dynamic requires its nonzero linear and nonlinear difficulty values plus resolution and dimension; the nonlinear measure also depends on the expected maximum state magnitude.Source: §B.8 · pp. 105–106

C Emulator Architectures: Leveraging Equinox for Flexible and Powerful Neural PDE Emulators

  • The Equinox/JAX architecture suite supports 1D–3D models, Dirichlet, Neumann, and periodic boundaries, dimension-agnostic spectral convolutions, and curated Conv, ResNet, UNet, Dilated ResNet, and FNO architectures.It also provides parameter-counting and effective-receptive-field diagnostics.Source: C Emulator Architectures: Leveraging… · p. 106

Notable Advantages of using Equinox & JAX

  • Equinox/JAX uses single-batch emulator calls with vectorization via jax.vmap, which can also parallelize initialization seeds and lets simulators and emulators share a module structure.Seed parallelism can provide seed statistics when a training run does not fully use the GPU, as in the described 1D scenarios.Source: Notable Advantages of using Equinox &… · pp. 106–107

D Training methodologies

  • APEBench supports unrolled losses from one-step supervised training to diverted-chain training and general multi-step branches, with the latter configuration not evaluated in the reported results.Diverted-chain training with T > B requires the reference solver dynamically for later targets and requires differentiability when gradients pass through that solver.Source: D Training methodologies · p. 107

D.1 Continued Taxonomy

  • Gradient cuts can address vanishing or exploding gradients, distribution shift, undesirable minima, or solver differentiability, and APEBench supports cuts through backpropagation or differentiable physics.The Brandstetter et al. (2022) trick is represented with T = B = 2, weights w1 = 0 and w2 = 0, and backpropagation-through-time disabled.Source: §D.1 · pp. 107, 109

D.2 Differentiable Physics in Learning Setups

  • Whether a setup needs a differentiable solver depends on the training configuration, prediction versus correction, and where gradient flow is cut.For example, one-step pure prediction needs no solver, while diverted-chain pure prediction using an unmodified fine solver requires differentiability; cutting that chain removes this requirement.Source: §D.2 · pp. 109–110

E APEBench Details

  • APEBench scenarios procedurally generate reproducible training data for supported PDE dynamics, and expose difficulty, normalized, or physical parameter interfaces alongside prediction and solver-correction tasks.Default discretizations are N = 160 in 1D, N = 160^2 in 2D, and N = 32^3 in 3D; some dynamics require physical parameterization, and correction combines a network with a coarse solver that performs only part of the reference integration step.Source: E APEBench Details · p. 110, §E.1 · p. 110, §E.2 · p. 115

E.3 Correction Tasks and Neural-Hybrid Emulators

  • APEBench supports sequential and parallel neural-hybrid correction, with experiments using sequential correction to share the coarse solver’s receptive field with the network.Modified layouts with gradient cuts are also selectable.Source: §E.3 · p. 117

F Metrics

  • APEBench metrics vary by spatial aggregation, error exponent, state comparison, normalization, and spectral modification, with channel contributions summed and samples averaged after per-channel normalization.The specified reduction order matters because spatial aggregation and division are nonlinear.Source: F Metrics · pp. 117–118
  • Unmodified Fourier MSE and RMSE agree with their state-space counterparts by Parseval’s theorem, whereas MAE and Fourier-aggregated H1 MAE do not align with the corresponding function norms.H1 losses with derivatives sum gradient directions and emphasize higher-frequency, smaller-scale errors.Source: F Metrics · p. 118

G An Interactive Transient 3D Volume Renderer for Simulation Trajectories in Python

  • APEBench includes a real-time interactive 4D volume renderer for visualizing time-varying simulation volumes within Python environments.Written in Rust using WebGPU, it can be embedded in Jupyter Notebooks or Visual Studio Code and uses a user-defined transfer function for volume rendering.Source: G An Interactive Transient 3D Volume… · p. 119

H.1 General Details

  • Unless otherwise specified, experiments use 160 degrees of freedom per spatial dimension and ETDRK2; training uses 50 solver-generated 50-step trajectories, while testing uses 30 trajectories of 200 steps.The reference stepper typically uses no substepping unless stated otherwise.Source: §H.1 · pp. 120–121
  • Architectures include convolutional networks, post-activation ResNets, UNets, dilated ResNets, and FNOs, implemented with periodic padding and without learning boundary conditions.Architectures are dimension-agnostic; default sizes are approximately 30k, 60k, and 200k parameters in 1D, 2D, and 3D, respectively.Source: §H.1 · p. 121
  • Experiments report median and 50% IQR across 50 seeds in 1D and 20 seeds in 2D and 3D; default training uses Adam with warmup cosine decay for 10,000 updates and evaluates geometric-mean aggregated nRMSE.The evaluation nRMSE is normalized to the reference-state magnitude, and the geometric mean aggregates errors over time to the default horizon of 100 steps.Source: §H.1 · pp. 121–122

H.2 Motivational Experiment … I.2 Optimization Configuration Ablation

The supplementary experiments specify configurations for the thesis’s emulator studies and ablate training choices. They report that longer unrolling remains advantageous for long-term accuracy, while extended training generally improves rollout error without changing architecture rankings.

H.2 Motivational Experiment

  • The motivational advection experiment finds that beyond a certain time point, the FOU stencil again outperforms learned stencils because it is consistent with the equation.The stencil optimization uses Newton iterations to double floating-point precision; longer unrolled training posed convergence problems, motivating initialization from the minimizer at the preceding unroll length.Source: §H.2 · p. 123, Table 7 · p. 124

H.3 Bridging Experiment

  • The bridging experiment compares architectures across three 1D advection difficulties and reports ResNet errors when trained with additional unrolled steps.The convolutional network’s depth is varied, with depth 10 matching its default configuration.Source: §H.3 · p. 125

H.4 Diverted Chain Experiment

  • The diverted-chain experiment reports median errors at time steps 1 and 100 across three nonlinear scenarios and three training configurations.Results summarize 50 seeds with 50% interquartile ranges.Source: §H.4 · p. 125

H.5 Task and Rollout Training Experiment

  • The task-and-rollout experiment varies coarse-proportion across three values and compares 2D ResNet and FNO test-rollout error over 100 steps.It uses diff_adv at advection_gamma 10.5 and reports the geometric mean of rollout error.Source: §H.5 · p. 125

H.6 Broad Comparison Experiment

  • The broad comparison evaluates one-step supervised emulators across dimensions and scenarios, with reduced 3D resolution and test-set size to fit 12GB GPU memory.It uses 50 seeds for 1D and 20 for 2D and 3D; Gray-Scott uses Gaussian-blob initial conditions, while Kuramoto-Sivashinsky is tested at default and reduced resolutions.Source: §H.6 · pp. 125–126

I Ablation Studies

  • The ablation studies retain compute-efficient default settings, aggregate results over 50 initialization seeds using medians and 50% IQRs, and classify KS 3D and reaction-diffusion problems as under-resolved.APEBench permits fine-grained control over settings; under-resolution is defined by Nyquist and hypothetical higher modes not falling below 10^-5.Source: I Ablation Studies · pp. 126, 128, 129

I.1 Unrolled Training Ablation

  • Longer unrolling still yields better long-term accuracy, although compensating short-unroll models with more updates substantially improves their rollout performance.At roughly equal GPU wall-clock time, one-step supervised training extends its accuracy duration, but configurations with more unrolled steps and fewer updates remain superior long term.Source: §I.1 · p. 130, Figure 15 · p. 130

I.2 Optimization Configuration Ablation

  • Peak learning rate 0.01 fails to show clear convergence for ConvNet, ResNet, and Dilated ResNet, while extended training generally improves error without changing architecture rankings.The default peak learning rate is 0.001, though 0.003 may also be viable; UNet and FNO generally improve stably except in advection, and performance plateaus without apparent overfitting except at the highest rate.Source: §I.2 · p. 131

I.3 Ablation for Size of the Training Dataset · I.4 Parameter Scaling Ablation

Training-data size and temporal coverage affect emulator performance in dynamics- and architecture-dependent ways, while increasing parameter counts generally helps until scaling becomes ineffective. The default dataset and parameter settings support fair architectural comparisons, but do not eliminate specific failure modes.

I.3 Ablation for Size of the Training Dataset

  • Local ConvNet and ResNet architectures converge with as few as five training samples, while global convolutions and FNOs require more data; gains eventually become minor across architectures.Sample-count convergence is qualitatively similar across architectures. Longer training horizons help multi-stage dynamics most, while advection changes little and Kuramoto–Sivashinsky effectively gains additional samples because it starts within the chaotic attractor.Source: §I.3 · p. 131, Figure 17 · p. 133
  • In the Korteweg-de Vries scenario, ResNet underperforms notably, but unrolled training substantially improves its performance; the failure’s root cause remains unclear.The authors describe this as an intriguing failure mode consistent with their optimization-configuration ablation findings.Source: §I.3 · p. 131
  • Longer horizons most improve Burgers and Korteweg-de Vries, have a smaller effect on diffusion, and make little difference for advection; all curves converge to approximately the same level.Kuramoto–Sivashinsky starts within its chaotic attractor, so longer horizons act like additional samples and yield faster convergence; diffusion’s trend reflects later low-magnitude states being mapped to even lower magnitudes.Source: §I.3 · p. 131, Figure 17 · p. 133
  • Fifty initial-condition samples and a training horizon of 50 provide a reasonable balance without changing the relative architecture ranking, enabling fair comparisons.A horizon that is too short can omit physical regimes, such as shock propagation in Burgers, which emulators must handle during the 100-step test rollout.Source: §I.3 · p. 131

I.4 Parameter Scaling Ablation

  • Increasing parameter counts generally improves test performance across architectures, but convergence rates depend on the dynamics under study.The comparison measures geometric-mean test error over 100 rollout steps; the gray region in Figure 18 marks the parameter range used for the main experiments.Source: §I.4 · p. 131, Figure 18 · p. 134
  • Some architectures plateau or worsen beyond a parameter threshold when scaling does not increase receptive field or adapt the optimization configuration effectively.The ablation increases hidden channels while retaining each network’s default receptive field.Source: §I.4 · pp. 131, 134
  • The default parameter region for 1D architectures is presented as a setting for fair comparisons.Figure 18 marks this region with gray shading.Source: §I.4 · p. 134, Figure 18 · p. 134

J Datasheet … F.1 POISSON EQUATION - INVERSE PROBLEM

This section documents APEBench’s data-generation and release practices and details the methods and experiments for differentiable PDE solvers and PRDP. It covers iterative solvers and reverse-mode differentiation, PDE discretizations and experimental settings, and Poisson inverse-problem results.

J Datasheet

  • APEBench tightly integrates a differentiable JAX solver and regenerates training and test trajectories per experiment, with a representative subset planned to mitigate ecosystem dependence.The benchmark specification is encoded by each scenario; features such as scenario modification, diverted-chain training, and correction learning remain exclusive to the full suite.Source: J Datasheet · p. 135

Any other comments?

  • The benchmark covers more than 46 PDE scenarios across three spatial dimensions, with reproducible structured-array data and defaults of 50 training trajectories × 51 steps and 30 test trajectories × 201 steps.Scenario metadata describes training, and users can create new scenarios easily.Source: Any other comments? · p. 135
  • Linear-PDE reference data is exact for bandlimited initial conditions, while nonlinear accuracy depends on configuration and can be improved with higher-order methods or temporal substeps.The simulations are deterministic and synthetic, with separately drawn initial conditions; scenario generation was checked to produce stable, non-NaN trajectories without preprocessing.Source: Any other comments? · pp. 136, 138

Who was involved in the data collection process (e.g., students, crowdworkers,

  • Only the authors were involved in data collection; the ETDRK solver framework was developed from September 2023 to May 2024, and APEBench from January to May 2024.Source: Who was involved in the data… · p. 137

When will the dataset be distributed?

Who will be supporting/hosting/maintaining the dataset?

How can the owner/curator/manager of the dataset be contacted (e.g., email address)?

ABSTRACT

  • PRDP reduces training time by 62% when emulating the Navier–Stokes equations, while preserving network accuracy.The approach applies to unrolled and implicit differentiation and is validated across inverse problems, autoregressive emulators, and correction-based neural-hybrid solvers.Source: PRDP: Full Paper · p. 141, ABSTRACT · p. 142

1 INTRODUCTION

  • PRDP is tested with differentiable linear solvers across Poisson, heat diffusion, Burgers, and Navier–Stokes problems, including inverse problems, physics losses, and correction-based approaches.Experiments include 1D problems and more complex 2D and 3D time-stepping settings.Source: §1 · p. 143

2.1 LINEAR SOLVER VJPS

  • Differentiating an incompletely converged linear solve introduces both primal and adjoint gradient inaccuracy; implicit differentiation solves a transposed system, while unrolled differentiation couples adjoint accuracy to primal iterations.The paper frames differentiable physics as bilevel optimization: a neural-network learning problem outside an iterative linear solve, where noisy training gradients need not be fully converged.Source: §1 · p. 143, §2 · p. 144, §2.1 · pp. 144–145

2.2 SCHEDULING INNER ITERATIONS

  • 28% fewer total inner iterations achieve the same 10^-5 relative parameter suboptimality in the Poisson inverse problem under a progressively increasing Jacobi iteration schedule.The schedule starts at K = 25 and adds ΔK = 6 per outer iteration until Kε = 600; it requires slightly more outer iterations but fewer inner iterations overall.Source: Figure 3 · p. 145, §2.2 · p. 145

2.3 NETWORK TRAINING UNDER INCOMPLETELY CONVERGED DIFFERENTIABLE PHYSICS

  • For the heat emulator, fixing physics refinement at Kmax = 10 yielded 60% fewer solver iterations for training, with no noticeable validation-performance improvement beyond that threshold.Source: §2.3 · p. 146

3 PROGRESSIVELY REFINED DIFFERENTIABLE PHYSICS

  • PRDP begins with coarse physics, typically K0 = 1, and increases solver iterations by ΔK when validation metrics plateau, automatically adapting refinement to training progress.The adequate refinement level and schedule depend on the PDE, discretization, iterative solver, and learning dynamics.Source: §3 · p. 146
  • PRDP retains refinement when validation stagnation exceeds τstop and also refines when rc > 1.0 to avoid initially divergent training.For implicit differentiation, primal and VJP solves use the same solver and inner-iteration count; with unrolled differentiation, Jacobian refinement follows the primal iteration count.Source: §3 · p. 147

4.1 LINEAR INVERSE PROBLEM

  • 33% savings on the doubly-convex inverse Poisson problem exceeded the manually tuned schedule’s 28%, while the three-dimensional parameter-space tests saved 48% with Jacobi and 43% with steepest descent.PRDP applied across the tested combinations of Jacobi or steepest-descent solvers and implicit or unrolled differentiation.Source: §4.1 · p. 148, §4.2 · p. 148

4.2 LINEAR NEURAL EMULATOR LEARNING

  • 79% total savings in Heat 1D, 86% in Heat 2D, and 81% in Heat 3D demonstrate PRDP’s substantial savings across dimensions.Heat 1D savings combine 60% from incomplete convergence with 19% from progressive refinement; Heat 2D combines 72% and 14%. Training time fell by 24% in 2D and 78% in 3D.Source: §4.2 · p. 149, Figure 7 · p. 149

4.3 NONLINEAR NEURAL EMULATOR LEARNING

  • 59% total savings on the nonlinear Burgers emulation combined 30% from incomplete convergence with 29% from progressive refinement.The strongly divergent initial regime required K0 = 4; incomplete convergence reduced Kϵ = 23 to Kmax = 16.Source: §4.3 · p. 149, Figure 8 · p. 149

4.4 NEURAL-HYBRID EMULATOR FOR THE NAVIER-STOKES EQUATION

  • 81% fewer iterations reduced neural-hybrid Navier–Stokes training time by 62%, and training remained 57% faster than fully converged training without validation-loss computation.The total iteration savings comprised 65% from incomplete convergence and 16% from progressive refinement; validation incurs extra cost because the fully converged solver runs during inference and validation.Source: §4.4 · p. 150

6 LIMITATIONS AND OUTLOOK

  • PRDP is limited to settings with iterative linear solvers, leaving unstructured higher-dimensional and multiphysics discretizations as untested cases.Future directions include smoother refinement schedules, separate primal and adjoint iteration counts, and refinement through spatial or temporal resolution.Source: §6 · p. 151

7 CONCLUSION

  • Up to 86% fewer cumulative physics-solver iterations and 78% less training time were achieved versus fully converged physics.PRDP begins with coarse physics and adaptively refines it as training progresses, ending below primal convergence while retaining training accuracy.Source: §7 · p. 151

B.1 JACOBI RELAXATION; B.2 STEEPEST DESCENT

  • Jacobi slows as high system-matrix condition numbers push iterator spectral radii toward 1; steepest descent selects an optimal step per iteration and has asymptotically exponential-linear residual convergence governed by κ(A).Jacobi convergence depends on the iterator matrix’s spectral radius; steepest descent follows the negative residual and determines the step size for maximum decrease at each iteration.Source: §B.1 · p. 158, §B.2 · p. 158

B.3 GMRES

  • GMRES builds a Krylov subspace and solves a least-squares problem, restarting every m iterations; the batched JAX implementation can terminate only at restart boundaries.An iteration comprises construction of the full m-dimensional subspace and its subsequent least-squares solve.Source: §B.3 · p. 158

C MORE DETAILS ON DIFFERENTIATING OVER ITERATIVE LINEAR SOLVERS

  • Reverse-mode differentiation of an iterative linear solve requires a vector-Jacobian product through the solver and assembly routines; implicit differentiation uses an adjoint solve, while unrolling propagates through each iteration.The two approaches produce intermediate gradients for the system matrix and right-hand side, which are then propagated through their assembly routines.Source: C MORE DETAILS ON DIFFERENTIATING OVER… · p. 160, Figure 11 · p. 162

C.1 IMPLICIT DIFFERENTIATION

  • Implicit differentiation solves an adjoint system with the transposed matrix; its tolerance-dependent iteration count may differ from the primal solve, and gradients require only the final primal solution.The adjoint gives the right-hand-side gradient, while the system-matrix gradient is the negative outer product of the adjoint and primal solution; JAX can transpose matrix-free operators.Source: §C.1 · p. 160

C.2 UNROLLED DIFFERENTIAION

  • Unrolled differentiation applies AD through each solver iteration and retains or recomputes iterates; observed burn-in was practically irrelevant, and PRDP savings were nearly identical to implicit differentiation.The implementation retains the entire iterate sequence, although other approaches can trade computation for memory.Source: §C.2 · p. 161

D DIFFERENTIABLE SOLVERS

  • The differentiable solvers use finite differences on uniform Cartesian grids; linear PDEs depend on prior computation through the right-hand side, while Picard-based nonlinear PDEs make both matrix and right-hand side dependent.The discretizations are implemented matrix-free in JAX, materializing matrices only for smoothing or direct decomposition solvers.Source: D DIFFERENTIABLE SOLVERS · p. 161

D.1 POISSON EQUATION IN 1D

  • The 1D Poisson example uses homogeneous Dirichlet conditions and a parameterized right-hand side, with a tridiagonal, non-parameter-dependent system matrix.The finite-difference discretization uses N interior degrees of freedom, and the right-hand-side assembly negates the input elementwise.Source: §D.1 · pp. 162–163

D.5 BURGERS IN 1D

  • For 1D Burgers, previous-time-state linearization yields an asymmetric system matrix, and omitting reassembly accepts an O(∆t) nonlinear-residual error while the linear residual converges.The O(∆t) error is considered acceptable because the temporal discretization is first order.Source: §D.5 · pp. 165–166

D.6 BURGERS IN 2D

  • The 2D Burgers discretization builds directional upwinding from each velocity component and assembles both component equations jointly into a block-diagonal system matrix.Both components at the previous time level are needed to assemble each component’s upwinding matrix.Source: §D.6 · pp. 166–167

D.7 COUPLED NAVIER-STOKES IN 2D

  • For staggered-grid 2D Navier–Stokes, bilinear interpolation maps velocities for convection, while forward differences define divergence and backward differences define pressure gradients.The coupled system has a saddle-point structure; the authors solve it without further modifications, citing low resolution despite its high condition number.Source: §D.7 · pp. 167–168

E MORE DETAILS ON PRDP PARAMETERS

  • PRDP’s stepping threshold controls how flat validation error must be before refinement, while its stopping threshold controls how strictly stagnation is checked across refinement levels.A lower stepping threshold speeds refinement but trades off savings; a lower stopping threshold enables more aggressive iteration savings.Source: E MORE DETAILS ON PRDP PARAMETERS · pp. 168–169

F EXPERIMENTAL SETUPS

  • For heat, Burgers, and Navier–Stokes, PRDP validation uses spatially normed, reference-normalized solution error at a specific time, averaged as mean-squared error over the validation set.Neural-network training trials use 10 initialization seeds; plots show their mean and shaded variability within one standard deviation.Source: F EXPERIMENTAL SETUPS · p. 169

F.1 POISSON EQUATION - INVERSE PROBLEM

  • The Poisson inverse problems keep the system matrix fixed and parameterize the right-hand side with amplitudes of the first P sine modes, testing P = 1 and P = 3.The objective is MSE between the iterative physics solution and a reference generated by direct solution at a reference parameter value; optimization uses gradient descent.Source: §F.1 · p. 171
  • PRDP enabled 33% solver iteration savings in the one-dimensional Poisson inverse problem.The problem converged at Kϵ = 600; PRDP started with K0 = 25 iterations and added ∆K = 10 at each refinement.Source: §F.1 · p. 171
  • PRDP enabled 40-50% savings for Jacobi and steepest descent, with both unrolled and implicit differentiation, in the three-dimensional parameter space.The physics converged with 450 inner iterations for Jacobi or 240 for steepest descent.Source: §F.1 · p. 171

F.2 HEAT DIFFUSION - AUTOREGRESSIVE NEURAL EMULATOR TRAINING … G.3 PRDP SAVINGS VS. NEURAL NETWORK EXPRESSIVENESS

The supplementary experiments apply PRDP to heat and Burgers emulators and a Navier–Stokes neural-hybrid corrector. They show that training-time savings grow with the share of computation spent on iterative solves, while increasing emulator expressiveness barely changes solver-iteration savings.

F.2 HEAT DIFFUSION - AUTOREGRESSIVE NEURAL EMULATOR TRAINING

  • For 1D–3D heat diffusion, MLP and ResNet emulators train through a differentiable mixed chain, with direct-solver references and Jacobi physics steps.The loss compares the second-step output against a reference; differentiation is implicit in 1D and unrolled in 2D and 3D.Source: §F.2 · pp. 171, 173

F.3 BURGERS EQUATION - AUTOREGRESSIVE NEURAL EMULATOR

  • The Burgers ResNet uses mixed-chain training over two steps, while PRDP refinement is guided by validation at t = 5, favoring longer-rollout generalization.The equation uses a periodic boundary condition, and the iterative solver is GMRES with restart 2.Source: §F.3 · p. 175

F.4 NAVIER STOKES - NEURAL-HYBRID CORRECTOR LEARNING

  • The Navier–Stokes neural-hybrid corrector learns on a coarse source manifold against a fine solver, and PRDP refines both physics executions in its two-step loss.The source solver has half the spatial resolution of the reference solver; PRDP is controlled using x-velocity validation at t = 5.Source: §F.4 · pp. 175–176

G.1 PRDP SAVINGS IN TERMS OF COMPUTE TIME

  • In 3D heat emulation, an 81% reduction in inner iterations aligned with 78% wall-clock savings; savings become more noticeable as iterative solves dominate training cost.For smaller 1D problems, other compute-graph overhead dominates, so fewer inner iterations do not translate into wall-clock savings.Source: §G.1 · p. 178, Figure 24 · p. 178

G.2 NEURAL NETWORK PERFORMANCE - PRDP VS. BASELINE

  • Validation accuracy for neural and neural-hybrid emulators is reported as last-epoch mean and standard deviation across 10 random seeds.Table 1 summarizes the validation results; the supplied passages do not provide its numeric cell values.Source: §G.2 · p. 178, Table 1 · p. 178

G.3 PRDP SAVINGS VS. NEURAL NETWORK EXPRESSIVENESS

  • Increasing the 1D heat MLP’s parameter count by an order of magnitude improved accuracy at approximately n^-1/5 convergence, while PRDP retained around 80% solver-iteration savings.The largest model had slightly higher savings, likely because it triggered one fewer refinement level; the study found the effect of expressiveness on iteration savings minimal.Source: §G.3 · p. 179

G.4 PRDP BASED ON A VALIDATION METRIC VS. ON TRAINING LOSS … A.4.1 Derivation of the Closed-Form Optimizer for the Linear Case

These sections examine how solver fidelity, training objectives, and emulator structure shape neural PDE emulator performance, including comparisons of PRDP guidance and cases where emulators outperform imperfect solvers. They use Fourier-mode analyses across linear and nonlinear PDE settings to study errors and rollout behavior, and derive a closed-form optimizer for linear emulators under single-mode training.

G.4 PRDP BASED ON A VALIDATION METRIC VS. ON TRAINING LOSS

  • Validation-loss-guided PRDP gives smoother convergence and is preferable for broader applications, while training loss may be unreliable for multimodal problems or those with spurious minima.The smoother progression is attributed to a more uniform refinement schedule.Source: §G.4 · p. 179
  • Training-loss-guided PRDP trains successfully but refines more slowly, yielding slightly higher PR savings while overall IC savings and final accuracy remain comparable.Training loss may keep decreasing after validation loss plateaus, which delays refinement.Source: §G.4 · p. 179, Figure 26 · p. 181

Abstract

  • Emulators trained purely on low-fidelity solver data can outperform that solver against a higher-fidelity reference, as regularization and favorable error accumulation can improve multi-step rollouts.The paper reports empirical evidence across PDEs and standard neural architectures.Source: Abstract · p. 184, §1 · p. 184

1 Introduction

  • Emulator superiority is not universal: it depends on the PDE, solver errors, architecture, training objective, evaluation metric, and time horizon.The authors argue that benchmarking against the same imperfect data source can unfairly penalize a more accurate emulator.Source: §1 · p. 185

2.2 Machine Learning Approaches for PDE Solving

  • Autoregressive emulators are trained to mimic a chosen solver but can be evaluated on multi-step rollouts against a different, higher-fidelity solver, testing solver and temporal or state-space generalization.The training objective is commonly one-step supervised prediction from solver-generated trajectories.Source: §2.2 · pp. 186–187

2.3 Comparing Emulators and Emulator Superiority

  • The paper distinguishes state-space superiority, observable even at the first step, from autoregressive superiority, which requires an initially non-superior emulator to become superior after later rollout steps.These arise because training optimizes a particular loss and distribution, whereas evaluation can involve multi-step rollouts and different conditions.Source: §2.3 · p. 187

3 A Case Study for Superiority under Linear Conditions

  • The analysis proves state-space superiority in closed form for three prototypical linear PDEs and reports empirical state-space and autoregressive superiority for advection and Burgers across neural architectures.The introduction specifies theoretical results for advection, diffusion, and Poisson, and empirical tests using convolutional networks, dilated ResNets, FNOs, UNets, and Transformers.Source: §1 · pp. 184–185, §3 · p. 187

3.1 Advection Equation

  • For advection, scheme errors are systematic across Fourier modes, and the superiority analysis identifies a region where the emulator is better than its training simulator for modes ϕ > ψ.The mode-wise magnitude and phase errors depend on relative mode and CFL number.Source: Figure 3 · p. 189, §3.1 · p. 189
  • Linear advection superiority occurs at higher test modes than training modes when a low-capacity linear ansatz learns from a high-capacity imperfect solver and |γ1| is sufficiently large.This is forward generalization in Fourier space; the analysis also extends to multiple rollout steps and physics-informed training.Source: §3.1 · p. 190

3.2 Diffusion Equation

  • For diffusion, emulators trained on a backward-in-time central-in-space scheme outperform it at higher test modes, with stronger superiority as γ2 increases.The favorable convolutional inductive bias resembles a forward-in-time central-in-space scheme; superiority is observed against the analytical reference after one time step.Source: §3.2 · p. 190, Figure 4 · p. 190

3.3 Poisson Equation

  • For Poisson, an emulator can outperform an unconverged Richardson solver mainly at lower test modes than training modes, with the pattern depending on iteration count q.This backward superiority occurs where the iterative reference has more error and the emulator has a favorable inductive bias.Source: §3.3 · p. 190, Figure 4 · p. 190

4.1 Nonlinear Emulators for the Linear Advection Equation

  • Nearly all tested nonlinear architectures achieve autoregressive superiority after initial steps on unchanged initial-condition distributions, including when trained on a stable explicit scheme.There is no first-step superiority because the emulators are trained to mimic the solver’s one-step behavior on those initial conditions.Source: §4.1 · p. 191, Figure 5 · p. 191
  • With broader test initial conditions, only the local feed-forward ConvNet matches the linear case’s state-space generalization; FNOs and Transformers generalize less effectively.The experiment trains on the implicit scheme and tests extrapolation from single-mode training to multi-mode states.Source: §4.1 · p. 191, Figure 5 · p. 191

4.2 Nonlinear Emulators for the Burgers equation

  • A UNet trained only on smooth frames from a truncated implicit Burgers solver propagates shocks more accurately than its training data, matching the high-fidelity shock angle.The proposed mechanism is implicit correction of the truncated nonlinear solve and convolutional generalization from low-frequency training data to emergent high-frequency shocks.Source: §4.2 · p. 192, Figure 6 · p. 192

5 Limitations & Outlook

  • Superiority is regime-specific, and its intensity depends on training-solver inaccuracy; future work should map operating boundaries and test sensitivity to solver fidelity, training strategies, and complex multiphysics problems.The authors note that highly precise training solvers leave less margin for improvement, while regime-bound superiority may still be practically useful.Source: §5 · pp. 192–193

6 Related Work; 7 Conclusion & Impact

  • The work identifies pipeline conditions under which emulators outperform data containing deterministic structural errors and stresses that flawed testing references can obscure emulator performance.It frames superiority as arising from the interaction of inductive biases, training objectives, solver errors, and rollout dynamics.Source: §6 · p. 193, §7 · p. 193

A.1.2 Error of the Schemes

  • Fourier diagonalization reduces advection scheme errors to mode-wise multiplier errors, assessed by magnitude and phase against an analytical multiplier of unit magnitude.The appendix gives magnitude and phase error expressions for both explicit and implicit schemes.Source: §A.1.2 · p. 200

A.1.3 Closed-Form Emulator trained on the Implicit Scheme

  • Fitting the advection emulator to the implicit scheme yields closed-form coefficients for its complex-valued equation and an explicit Fourier multiplier.The ansatz has the explicit scheme’s upwinding bias and is intended for γ1 < 0.Source: §A.1.3 · p. 201

A.1.4 Closed-Form Superiority when using the Implicit Scheme as Baseline

  • Closed-form superiority expressions for the implicit-training advection case exist, but most resist easy simplification and require lengthy supplemental expressions.The analytical scheme is the reference and the implicit scheme is the baseline.Source: §A.1.4 · p. 201

A.1.5 More Combinations of References

  • Figure 7 shows superiority depends on training and baseline references: analytical-scheme training strengthens it, while the stronger explicit baseline largely removes it within stability.The explicit ansatz can nevertheless become superior beyond the stability limit near the trained mode; when its functional form matches the training scheme, it re-learns that scheme and has ξ[t] = 1.Source: Figure 7 · p. 202, §A.1.5 · p. 202

A.1.6 Influence of the Number of Test Rollout Steps

  • Magnitude superiority against the implicit baseline persists for many rollout steps but eventually vanishes, and it is lost faster for higher |γ1|.Within |γ1| < 1, superiority converges to 1 after many steps; phase superiority persists until arctan evaluation issues arise.Source: §A.1.6 · p. 203, Figure 8 · p. 203

A.2.1 Deriving the Schemes for the Diffusion Equation

  • For diffusion, central-difference schemes introduce no phase errors, so only magnitude errors are analyzed; FTCS is stable for γ2 ≤ 1, and the reported scheme is unconditionally stable for γ2 > 0.Within γ2 < 1, the explicit scheme has smaller error.Source: §A.2.1 · p. 204, §A.2.2 · p. 205, Figure 9 · p. 205

A.2.3 Closed-form Superiority Expressions for Diffusion

  • For diffusion, fitting the FTCS-like ansatz to the implicit solver yields a corrected γ2 that can better approximate the analytical solution and produce superiority.The fitted θ approaches γ2 when ψ is close to 0; Figure 10 compares training and baseline combinations against the analytical reference.Source: §A.2.3 · p. 206, Figure 10 · p. 206

A.3.1 Deriving the Schemes for the Poisson Equation

  • Poisson Richardson iterations converge to the direct finite-difference solution as q increases, provided the relative mode is neither 0 nor an integer multiple of 1/2.The direct method solves the discretized system, while the iterative method treats Poisson as the steady state of a pseudo-time diffusion equation.Source: §A.3.1 · p. 207
  • The Fourier-spectral Poisson solution inverts the continuous differential operator without spatial discretization error, assuming a sufficiently smooth periodic forcing; periodic solvability requires zero mean forcing.The zero Fourier mode of the solution is typically set to zero or otherwise determined to fix its mean.Source: §A.3.1 · pp. 207–208

A.3.2 Errors of the Poisson Schemes

  • Poisson-scheme errors are evaluated against the Fourier-spectral solution, with separate measures for direct finite differences, iterative-solver error, and iterative truncation error.The iterative truncation error measures the iterative scheme relative to the direct finite-difference scheme it approaches.Source: §A.3.2 · p. 208

A.3.3 Closed-form Superiority Expressions for Poisson

  • Emulator superiority depends on the training solver’s convergence level and on which baseline and high-fidelity reference define the comparison.The analysis varies the baseline and reference solvers, and illustrates how training on the iterative scheme’s q iterations affects superiority.Source: §A.3.3 · p. 209, Figure 11 · p. 209

A.4.1 Derivation of the Closed-Form Optimizer for the Linear Case

  • Under single-mode training, the linear emulator’s closed-form optimum is independent of the mode’s amplitude and phase distribution; multiple modes instead yield energy-weighted least squares.In the single-mode case, amplitude affects the loss only through a positive scaling factor, while phase is absent; with multiple modes, relative mode energies affect the optimum.Source: §A.4.1 · pp. 209–211

A.4.2 Proof that One-Step Supervised Training is equivalent to Discrete Residua … B.2.1 Implicit Time Integrators for the Burgers Equation

For linear systems, discrete-residual and one-step supervised training have identical optimizers when a single Fourier mode is active, while implicit multi-mode cases depend on spectral energy. The experimental sections specify emulator comparisons and use a one-step implicit Burgers solver for training against a fully converged reference.

A.4.2 Proof that One-Step Supervised Training is equivalent to Discrete Residua

  • With one active mode, supervised and discrete-residual losses have identical optimizers; they are also identical for explicit schemes, so the superiority analysis applies to both training paradigms.For implicit schemes with at least two active modes, the objectives cannot generally be related simply: their optimizers depend on modal energy, which the residual formulation additionally scales and rotates.Source: §A.4.2 · p. 212

B Experimental Details

  • The architecture comparison uses ConvNet, UNet, Dilated ResNet, FNO, and Transformer models with 27–33k parameters; unlike the convolutional and Fourier-spectral models, the Transformer lacks periodic-boundary inductive bias.The listed models span finite receptive fields for convolutional networks and global receptive fields for FNO and Transformer architectures.Source: B Experimental Details · pp. 212–213

B.1 Nonlinear Emulators on Linear Advection

  • Linear-advection experiments vary γ1, the training simulator, and initial-condition distributions to examine autoregressive and state-space generalization, using the analytical solver as the test reference.The domain has N = 100 points; training uses 300 one-step samples, and evaluation rolls out 10 initial conditions for 50 time steps.Source: §B.1 · p. 213

B.2.1 Implicit Time Integrators for the Burgers Equation

  • The Burgers setup discretizes the periodic unit interval into N = 60 points and uses upwind differentiation based on winds inferred from neighboring averages, allowing for sign changes.The nonlinear system is advanced by Picard iteration, which reassembles the system matrix at each iteration while keeping the right-hand side fixed.Source: §B.2.1 · pp. 214–215
  • The one-step (P1) implicit Burgers method is the training reference and evaluation baseline; its single Picard iteration leaves nonlinear residuals unconverged and can cause shock-propagation errors as gradients sharpen.It linearizes the nonlinear term at the previous time step and performs one matrix assembly and linear solve per step.Source: §B.2.1 · p. 215
  • Burgers evaluation instead uses a fully converged nonlinear implicit reference, iterating until the residual is below 10^-5 to obtain correct shock propagation.Both training and testing linear systems use LU decomposition to machine precision; unconverged iterative linear solvers could introduce additional error.Source: §B.2.1 · p. 215
  • Burgers training and testing initial conditions activate both the first Fourier mode and the mean mode.The experiments illustrate the effect of the unconverged Picard iteration on the solution.Source: §B.2.1 · p. 215

B.2.2 Experimental Details … C.3 Effect of Coarse Solver Fidelity by ablating the number of Picard Iterations

The experiments show that emulator superiority depends on structured solver error and architectural constraints: coarse training solvers and appropriately limited model capacity can improve on solver behavior, while higher solver fidelity removes that opportunity.

B.2.2 Experimental Details

  • The Burgers experiment used a UNet, viscosity γ2 = 0.2, convection difficulty δ1 = −8.0, and 60 grid points, with smooth Fourier-based initial conditions that developed shocks.Initial conditions used randomized phases in the first Fourier mode plus a nonzero mean-mode offset.Source: §B.2.2 · pp. 215–216
  • Training used one-step MSE prediction on 500 samples, while evaluation used 20-step autoregressive rollouts on 100 test samples measured by nRMSE.The low-fidelity training simulator truncated Picard iteration after one step; evaluation used the same scheme with full convergence as its reference.Source: §B.2.2 · pp. 215–216
  • Performance variability was assessed across 30 network initialization seeds while training and testing data were held fixed across seeds.Training used Adam with batch size 32 for 100,000 iterations; all seeds ran on one RTX3060 GPU in under five hours.Source: §B.2.2 · p. 216

C.1 Multi-Mode Case Study

  • Training on multiple modes still produced forward superiority when those modes were lower than the tested mode, including the numerical experiment testing on mode 5.With multiple training modes, the emulator’s closed-form parameters become a data-weighted linear least-squares problem rather than a single-mode equation.Source: §C.1 · p. 217, Table 1 · p. 217

C.2 Impact of Receptive Field in CNNs and Active Fourier Modes in FNO

  • A receptive field of 4 achieved the best ConvNet superiority, with ξ[t] of 0.73 after 10 time steps; both smaller and larger fields performed worse.The authors attribute the decline for overly large fields to excess capacity that can learn more of the solver’s non-physical behavior.Source: §C.2 · p. 217, Table 2 · p. 218
  • An FNO with 2 active modes achieved the best performance, with a superiority ratio of 0.59; 1 mode failed to learn, while 3+ modes weakened superiority.The authors link the 2-mode result to sufficient capacity for target dynamics and spectral truncation that filters solver artifacts.Source: §C.2 · p. 217, Table 3 · p. 218

C.3 Effect of Coarse Solver Fidelity by ablating the number of Picard Iterations

  • ξ[t] dropped to 0.85 with a very coarse Picard training solver, while superiority vanished as solver fidelity increased and the emulator replicated the accurate solver.The ablation varied the number of Picard iterations; one iteration was the original low-fidelity setup, and more iterations produced a more accurate solver.Source: Table 4 · p. 218, §C.3 · p. 218

C.4 Additional Architectures for Burgers Study

The Burgers study finds that superiority over a low-fidelity training solver depends on architecture: models with spatial or spectral inductive biases succeed, while the vanilla Transformer does not.

C.4 Additional Architectures for Burgers Study

  • Across 30 seeds, ConvNet, Dilated ResNet, FNO, and UNet achieved significant superiority over their low-fidelity training solver, while the vanilla Transformer failed and became progressively worse than the baseline solver.The reported comparison is median superiority; the study extends the Burgers experiment from section 4.2 to all architectures.Source: §C.4 · p. 219, Table 5 · p. 219

Related papers

Loading 2608.24547v1…