From Numerical Simulators of PDEs to Neural Emulators and Back

Felix Koehler

Source-linked AI summary

From Numerical Simulators of PDEs to Neural Emulators and Back

Felix Koehler

arXiv:2608.24547v1Aug 25, 2026cs.LG

TL;DR

PDE simulation is costly for fast or repeated evaluations, while neural emulators are often treated as opaque alternatives to the solvers generating their data. This thesis unifies both paradigms through solver-role analysis and Fourier-spectral reasoning, showing that their correspondence explains emulator superiority, training savings, and benchmark dependence on explicit assumptions.

Problem
PDE solvers remain costly for fast or repeated evaluations, while limited analysis leaves the relationship between solvers and neural emulators poorly understood.
Method
The thesis disentangles solver roles in emulator training and uses mode-wise Fourier analysis to interpret solver errors, architectural biases, and training objectives together.
Results
Neural emulators and numerical solvers are more alike than different, with inductive biases explaining emulator superiority, incomplete convergence, and architecture-dependent performance.
Takeaways & Limitations
Recognizing the shared structure enables insights to flow between neural emulators and classical numerical methods through a common analytical language.
Takeaways & Limitations
The PRDP method requires an iterative solver in the computational graph and its incomplete-convergence savings depend on problem structure.

Abstract

from arXiv · show

Simulation is central to modern engineering and science, but the cost of numerical solvers for partial differential equations (PDEs) remains a bottleneck whenever fast or many-query evaluations are required. Neural emulators trained on solver-generated data promise significant speedups, yet they are usually framed as opaque alternatives to the very methods that produce their training signal. This thesis argues the two paradigms are more alike than different: neural architectures mirror classical discretizations, their errors are amenable to the same spectral analysis, and insight flows profitably in both directions. We approach the relationship by disentangling the multiple roles a solver plays in the emulator learning pipeline. Mode-wise Fourier analysis then provides a common language in which solver errors, architectural inductive biases, and training objectives can all be read off simultaneously. Taken together, this allows synthesizing three contributions. (1) APEBench, a comprehensive benchmarking suite for autoregressive neural emulators of PDEs that uses fast differentiable pseudo-spectral solvers in JAX. (2) Progressively Refined Differentiable Physics, an investigation of the effect of unconverged solvers on surrogate training. (3) Neural Emulator Superiority, an analysis of the influence of numerical errors and architectural inductive biases.

1. Prof. Dr.-Ing. Nils Thürey · 2. Assistant Prof. Dr. William de Almeida Gilpin (schriftliche Beurteilung) · 2. Prof. Dr. Riccardo Marin (mündliche Prüfung)

Die Dissertation wurde am 21.04.2026 an der Technischen Universität München eingereicht und am 02.07.2026 durch die TUM School of Computation, Information and Technology behandelt. Inhaltlich hebt der Abschnitt die Bedeutung von Simulationen, die Kosten numerischer PDE-Löser und das Potenzial neuronaler Emulatoren hervor.

2. Prof. Dr. Riccardo Marin (mündliche Prüfung): Die Dissertation wurde am 21.04.2026 bei der Technischen Universität München eingereicht.
2. Prof. Dr. Riccardo Marin (mündliche Prüfung): Die TUM School of Computation, Information and Technology behandelte die Dissertation am 02.07.2026.
Zusammenfassung: Simulationen sind ein zentrales Werkzeug in Ingenieurwesen und Wissenschaft.
Zusammenfassung: Die Kosten numerischer Löser für partielle Differentialgleichungen bleiben ein limitierender Faktor.
Zusammenfassung: Dies gilt insbesondere, wenn schnelle oder vielfach wiederholte Auswertungen erforderlich sind.
Zusammenfassung: Neuronale Emulatoren, die auf synthetischen Simulatordaten trainiert werden, versprechen erhebliche Geschwindigkeitsgewinne.

List of Publications

The dissertation examines numerical PDE simulators and neural emulators as closely related paradigms through spectral analysis and a holistic view of surrogate learning. Its publications develop benchmarking, progressively refined solver training, and analysis of when emulators can surpass their training data.

Cross-cutting perspective: The dissertation’s central premise is that numerical simulators and neural emulators share significant similarities, so understanding the full surrogate-learning pipeline can inform both directions.Neural emulators are commonly trained on synthetic data produced by conventional numerical methods, making the relationship between solver and emulator central to the publications.
Neural Emulator Superiority: Fourier-spectral error analysis explains why neural emulators succeed or fail and how they can sometimes exceed the fidelity of their training solver.This establishes a shared numerical-analysis perspective on solver errors and emulator behavior.
Progressively Refined Differentiable Physics: Progressively refined differentiable solvers show that full convergence is unnecessary for efficient surrogate training, enabling substantial computational savings.The approach exploits the solver’s role within the training loop rather than treating it only as a data generator.
APEBench: APEBench treats the solver as a first-class component for systematic evaluation across 46 PDE configurations, multiple architectures, and diverse training strategies.The framework uses fast differentiable pseudo-spectral solvers in JAX to benchmark autoregressive neural PDE emulators.

2. Foundations of Numerical Simulation

Numerical simulation advances discretized PDE states through evolution operators built from spatial discretization and time integration. The thesis emphasizes Fourier-mode error analysis and differentiable JAX implementations as foundations for efficient, scalable, and trainable solvers.

Method of lines: The method-of-lines pipeline discretizes a continuous PDE in space, then repeatedly applies a time-integration evolution operator to generate state trajectories.The operator Ph advances u[n] h by one time step ∆t from a discretized initial condition.
Differentiable solvers: JAX implementations make numerical time steppers batchable, accelerator-compatible, and differentiable, enabling sensitivity analysis, optimization, hybrid architectures, and physics-based regularization.JAX’s vmap vectorizes solver instances, lax.scan abstracts time-stepping loops, and Exponax generates large datasets in seconds without implicit-solve convergence tolerances affecting gradients.
Spatial discretization: Spatial discretizations differ in domain representation, derivative approximation, and boundary-condition treatment, with finite differences and Fourier spectral methods reviewed as two main approaches.Finite differences use Taylor-series-based stencils whose quadratic spatial error decays polynomially with ∆x, while boundary-compatible padding is required.
Spatial discretization: Spectral derivatives are exact for bandlimited states, but spectrally rich or discontinuous states can make them less efficient than finite differences because Fourier transforms dominate cost.Bandlimited states carry no energy above the Nyquist frequency N//2.
Fourier analysis: Fourier-mode analysis separates spatial and temporal error sources: BTCS always under-attenuates diffusion, whereas FTCS combines opposing errors whose net sign depends on γ2 and mode ϕ.The analysis considers the linear one-dimensional diffusion equation on a periodic domain and compares numerical multipliers with the analytical multiplier.
Solver analysis tools: Picardax supports controlled studies of how iterative solver convergence and selective gradient cuts affect training signals, while Chaotax evaluates Lyapunov properties for emulator evaluation horizons.These capabilities directly underpin the PRDP work and connect Lyapunov exponents to meaningful autoregressive-emulator horizons.

3. Neural Emulators for PDEs

Neural PDE emulators encode spatial structure through inductive biases that mirror classical discretizations, including finite-difference-like convolutions, spectral-like Fourier layers, multigrid-like UNets, and attention mechanisms. The chapter then examines how solver roles and numerical fidelity shape emulator training, including cases where emulators outperform their training solvers and training-loop solvers need not converge.

Architectural analogies: Neural emulator architectures mirror classical numerical methods through locality, Fourier multipliers, resolution hierarchies, and learned identity-plus-perturbation updates.Convolutional networks resemble finite-difference stencils, FNOs resemble spectral methods, UNets resemble geometric multigrid, and residual structures echo I + ∆t A time steppers.
Fourier Neural Operators: FNO layers apply learned complex multipliers mode-wise in Fourier space, retaining only approximately 12–20 low modes and thereby imposing an inherent low-pass filter.A full 1D parameterization requires N//2 + 1 complex multipliers per channel pair, whereas practical FNOs set higher-mode multipliers to zero.
Attention-based architectures: Transformers provide flexible nonlocal mixing but encode little physical inductive bias, lacking built-in locality, periodicity, and translation equivariance in their vanilla form.Shifted-window and hierarchical variants reintroduce some priors, while attention can extend to unstructured representations with positional embeddings.
Chapter objectives: An emulator trained on trajectories from a numerical solver can produce multi-step rollouts whose error against a higher-fidelity reference is strictly smaller than that of its training solver.The reference may be analytical in some cases, motivating analysis beyond a simple data-driven view of surrogation.
Chapter objectives: The chapter separates five numerical-solver roles in the emulator pipeline and studies configurations involving solver-surpassing emulators, unconverged training-loop solvers, and benchmarking consequences.These role configurations are formalized in Section 4.1 and connected to Sections 4.2–4.4.

4. The Solver-Emulator Relationship

The solver-emulator relationship is clarified by separating the solver’s roles, analyzing errors and inductive biases spectrally, and showing that unconverged physics can suffice for training. Neural emulators can sometimes outperform their numerical training sources, while benchmarks must expose solver roles rather than hide them in fixed datasets.

Solver roles and benchmarking: Numerical solvers may generate data, provide reference trajectories, operate inside training, or serve as comparison baselines, so benchmarks must identify which solver fills each role.Fixed benchmarks such as PDEBench can collapse roles (1), (2), and (5) into an opaque dataset, preventing assessment of what the emulator reproduces.
Neural Emulator Superiority: Inductive biases can implicitly regularize solver-generated artifacts, allowing an emulator to exceed the accuracy of its training source in certain frequency bands.For BTCS data, the learned multiplier can be closer to the analytical multiplier than the BTCS training data, whereas FTCS data recover θ = γ2 exactly.
Neural Emulator Superiority: Superiority requires mismatched inductive biases and solver errors, suitable training-testing differences or rollout dynamics, and sufficiently inaccurate training data.Autoregressive superiority can occur with identical training and test distributions, while state-space superiority was observed mainly for convolutional architectures with local receptive fields.
Progressively Refined Differentiable Physics: Progressive solver refinement is justified by a bilevel view in which full inner-solver convergence is unnecessary while network training remains far from convergence.Solver-induced gradient inaccuracy can be dominated by minibatch noise, optimizer normalization, and the neural model’s coarser final accuracy.
Progressively Refined Differentiable Physics: PRDP saves up to 86% of cumulative solver iterations relative to fully converged physics, reducing wall-clock and potentially inference costs.The appropriate refinement schedule depends on the PDE, discretization, iterative solver, and learning dynamics, motivating an adaptive algorithm.
Benchmarking: APEBench treats the solver as a first-class component and implements four design principles to make solver roles explicit rather than treating data as a black-box source.These principles respond to the need for benchmarks that go beyond fixed datasets and leaderboards.

5. Systematic Emulator Benchmarking

APEBench provides a systematic, solver-accessible benchmark spanning 46 PDE configurations and multiple experimental axes. Its findings link emulator performance to receptive fields, problem structure, training methodology, and data horizon, while exposing limitations of the pseudo-spectral setting.

Benchmark design: APEBench exposes PDE coefficients, spatial resolution N, time step ∆t, and initial-condition distributions for controlled emulator–solver experiments.Its JAX-based Exponax solver also supports differentiable physics losses and neural-hybrid correction tasks.
Benchmark design: APEBench provides 46 PDE configurations covering linear, nonlinear, and reaction-diffusion dynamics, with qualitative classifications and dimension availability.Linear periodic problems can be solved without spatial discretization error, while nonlinear and reaction-diffusion systems generate richer or high-frequency states.
Benchmark design: APEBench’s pseudo-spectral scope excludes problems such as high-Reynolds-number flows and Euler equations, but isolates competencies underpinning continuum modelling.The framework probes Laplacian, transport, incompressibility, polynomial nonlinearities, dispersion, and chaos in isolation and combination.
Key findings: Emulator accuracy degrades as the advection difficulty γ1 increases when an architecture’s receptive field fails to cover the grid cells traversed per time step.Local ConvNets and ResNets fail beyond their receptive fields, whereas architectures with larger receptive fields perform better; this mirrors the CFL condition.
Key findings: No architecture is universally best: ResNets are consistently safe, FNOs excel on global low-Reynolds-number Navier–Stokes structure, and FNOs struggle with high-frequency reaction-diffusion patterns.For 2D advection, diverted chains with B = 1, T > 1 tend to outperform supervised unrolling with B = T for local architectures.
Key findings: Training-data horizon matters most for multi-stage dynamics such as Burgers and KdV, while some attractor dynamics are less sensitive; the full suite costs approximately 900 GPU-hours.The experiments also report that some architectures converge with as few as five training trajectories, whereas FNOs require more data.

6. Discussion and Outlook

The discussion presents numerical solvers and neural emulators as closely related, while identifying accuracy, stability, generalization, and evaluation limits. It also outlines methodological implications, computational opportunities, and open questions for extending these findings beyond controlled PDE settings.

Accuracy and stability: Neural emulators are typically capped at ≈10−2 nRMSE, with the best cases reaching ≈10−3, unlike solvers that can converge to machine precision.Solver convergence does not guarantee accurate representation of the underlying PDE system; the relevant issue is the types of errors present.
Accuracy and stability: Autoregressive emulators eventually diverge on sufficiently challenging problems because errors compound at each step, although periodic or statistically stationary dynamics can support long stable rollouts.The severity depends strongly on the problem type, while non-periodic, non-stationary dynamics are especially challenging and may require only a finite deployment horizon.
Scope and limitations: The thesis is restricted to semi-linear PDEs with periodic boundaries on uniform Cartesian grids and fast reference solvers, enabling Fourier analysis and rapid hypothesis testing but excluding complex geometries and other boundary conditions.The controlled setting is both a genuine limitation and an asset because it isolates mechanisms that realistic scenarios could confound.
Scope and limitations: Full-field nRMSE does not ensure convergence of derived quantities, since spatial derivatives amplify high-frequency errors and surface integrals add quadrature errors.This limitation is particularly relevant when applications prioritize quantities such as stress, lift, or drag over state-field accuracy.
Methodological outlook: JAX implementations of established ETDRK and finite-difference methods add autodiff, vmap, and accelerator support, enabling fast data generation and neural-hybrid models.The outlook also envisions automated translation and comparison of CPU solvers across techniques such as FVM, FEM, and SPH.
Central lesson: The overarching correspondence explains why emulators can implicitly correct structured solver errors, why fully converged hybrid solvers may be unnecessary, and why numerically resembling architectures can suit particular PDEs.Current models remain tied to a single solver and PDE system, while APEBench does not yet cover Transformer-based architectures.

7. Summary of Publications … H.1 General Details

The thesis presents APEBench and related analyses as a unified study of numerical solvers, neural emulators, differentiable training, and spectral error behavior. Its publications show that solver refinement, architectural inductive bias, and evaluation design jointly determine emulator accuracy, efficiency, and interpretability.

D.2 Differentiable Physics in Learning Setups: PRDP treats differentiable-physics training as bilevel optimization and progressively increases solver iterations only when validation performance has plateaued.It supports both unrolled and implicit differentiation and automatically detects when further refinement yields no measurable improvement.
7.2 Progressively Refined Differentiable Physics: 62% training-time reduction and up to 86% fewer solver iterations are achieved with PRDP while final accuracy remains statistically indistinguishable from fully converged physics.The method was validated on Poisson inverse problems, heat, Burgers, and Navier–Stokes neural-hybrid settings.
7.3 Neural Emulator Superiority: Fourier analysis shows that linear emulator ansätze matching numerical schemes yield closed-form amplification factors, with inductive bias acting as implicit regularization.State-space superiority arises under distribution shift, whereas autoregressive superiority can emerge during rollout even with identical training and test distributions.
APEBench: Full Paper: APEBench is a JAX-based benchmark integrating efficient pseudo-spectral simulation, procedural data generation, and emulator interaction across 46 PDE dynamics in 1D, 2D, and 3D.It also introduces a dynamics taxonomy and identifiers tied to emulation difficulty.
2 From Classical Numerics to Learned Emulation: Learned emulators can outperform their numerical training solver over specific rollout regimes, making rollout metrics and high-fidelity references necessary to separate emulator and solver errors.The learned emulator may later diverge when inconsistent, while additional autoregressive training can improve long-term behavior.
4 Components of the APEBench benchmark: APEBench evaluates solver–emulator interaction through training methodologies, temporal rollout behavior, architectures, dataset sizes, and evaluation metrics.Its design complements fixed-dataset benchmarks by supporting neural-hybrid and differentiable training setups.
5 Experiments: Unrolled training generally improves long-term accuracy, while architecture suitability depends on receptive field, PDE dynamics, dimensionality, resolution, and frequency content.Local architectures often excel on local or high-frequency problems, whereas global architectures can better capture long-range structure.
6 Limitations: The framework relies on periodic uniform grids and has implementation limits including equal extents and discretization counts, real-valued PDEs, and constant time steps.Within these constraints, Fourier pseudo-spectral ETDRK methods are efficient on GPUs and integrate naturally with automatic differentiation.

H.2 Motivational Experiment … I.2 Optimization Configuration Ablation

The appendix reports experimental configurations across the benchmark studies and ablations, showing that optimization, rollout horizon, architecture, and resolution choices materially affect emulator evaluation. It also finds that shorter unrolling can improve substantially under matched compute, while longer unrolling remains superior overall and the default peak learning rate of 0.001 is generally reasonable.

H.2 Motivational Experiment: The motivational experiment uses γ1 = 0.75, N = 30, D = 1, K = 5, five training initial conditions, and 200 analytical-solver time steps.Stencil optimization uses full-batch Newton updates initialized with the FOU stencil, and evaluation uses 50 new initial conditions over 200 time steps.
H.3 Bridging Experiment: The bridging experiment varies γ1 ∈{0.5, 2.5, 10.5} and modifies only feedforward convolutional-network depth, with depth 10 matching the default.Errors are reported across architectures and difficulty, including ResNet training with more unrolled steps.
H.4 Diverted Chain Experiment: The diverted-chain experiment evaluates three nonlinear scenarios with default configurations and reports time-step 1 and 100 errors using medians and 50% IQR over 50 seeds.The ResNet uses its default 1D configuration.
H.5 Task and Rollout Training Experiment: The task-and-rollout experiment fixes advection_gamma=10.5, varies coarse_proportion in {0.0, 0.1, 0.5}, and compares default 2D ResNet and FNO configurations over 100 rollout steps.Results use the geometric mean of test rollout error.
H.6 Broad Comparison Experiment: The broad comparison trains all emulators with one-step supervised prediction and measures geometric-mean test rollout over 100 time steps, using reduced 3D resolution N = 323 and fewer test trajectories.The study uses 50 seeds in 1D and 20 seeds in 2D and 3D.
I Ablation Studies: The ablation studies use compute-efficient broad-comparison settings while retaining APEBench’s flexible controls and reasonable defaults.Aggregated results use medians over 50 initialization seeds with 50% IQR error bars or shaded areas; the KS 3D and all reaction-diffusion problems are under-resolved under the 10−5 threshold.
I.1 Unrolled Training Ablation: Under equal compute time, shorter-unrolled ResNet training substantially improves test rollout performance, but configurations with more unrolled steps remain superior overall.Even one-step supervised training achieves accuracy over a significantly extended duration, without changing the main conclusion.
I.2 Optimization Configuration Ablation: A peak learning rate of 0.01 is excessive for ConvNet, ResNet, and Dilated ResNet, whereas UNet and FNO generally improve stably except in advection; 0.001 is generally reasonable and 0.003 may also work.The ablation retains cosine decay with linear warmup ending at one-fifth of training and evaluates geometric-mean error over 100 rollout steps.

I.3 Ablation for Size of the Training Dataset · I.4 Parameter Scaling Ablation

Training-data size improves emulator performance until gains become minor, with architecture-dependent sample efficiency and dynamics-dependent benefits from longer temporal horizons. Increasing parameter counts generally helps, but excessive scaling can plateau or reduce performance when receptive fields and optimization settings do not adapt.

I.3 Ablation for Size of the Training Dataset: Performance improves consistently with more training samples, although gains become minor beyond a certain dataset size.Local ConvNet and ResNet architectures converge with as few as five training samples, ahead of global convolutions and FNO.
I.3 Ablation for Size of the Training Dataset: ResNet notably underperforms on Korteweg-de Vries, but unrolled training substantially improves its performance.The underlying cause of this failure mode remains unclear, consistent with the optimization-configuration findings.
I.3 Ablation for Size of the Training Dataset: Longer temporal horizons benefit multistage Burgers and Korteweg-de Vries dynamics most, whereas their effect is weaker for Kuramoto-Sivashinsky dynamics already within its chaotic attractor.An overly short horizon can omit physical regimes such as shock propagation in Burgers.
I.3 Ablation for Size of the Training Dataset: The default dataset of 50 initial-condition samples and a temporal horizon of 50 balances training size while preserving the relative ranking of architectures.This setting enables fair comparisons without changing the observed architecture ordering.
I.4 Parameter Scaling Ablation: Increasing hidden channels improves performance across architectures while keeping the default receptive field unchanged.The parameter ablation expands network capacity without altering settings that influence receptive-field size.
I.4 Parameter Scaling Ablation: Beyond problem-dependent parameter thresholds, some architectures plateau or decline when scaling fails to enlarge the receptive field or adapt optimization.Thus, parameter-count increases do not guarantee continued improvement.
I.4 Parameter Scaling Ablation: The default architectures lie within the gray parameter region used for fair comparisons in the 1D scenarios.Figure 18 identifies this shaded region as the parameter range of the default architectures.

J Datasheet … F.1 POISSON EQUATION - INVERSE PROBLEM

The paper presents APEBench as a procedurally generated, differentiable PDE benchmark and introduces PRDP to reduce differentiable-physics training cost through incomplete convergence and adaptive refinement. Across inverse problems and neural-emulator scenarios, PRDP achieves substantial iteration savings while remaining subject to problem-dependent limitations.

J Datasheet: The benchmark depends strongly on JAX and Equinox, although representative trajectories will be released for data-driven learning in ecosystems such as PyTorch or Julia.Scenario modification, diverted-chain training, and correction functionalities depend on the benchmark’s integrated ecosystem.
J Datasheet: APEBench tightly integrates a differentiable JAX-based solver suite for training pipelines in which neural emulators interact with numerical PDE simulators.The benchmark supports both purely data-driven training and more intricate solver-interacting configurations.
Any other comments?: APEBench procedurally regenerates reproducible training and test trajectories for more than 46 PDE scenarios across three spatial dimensions.Default datasets contain 50 training trajectories of 51 time steps and 30 test trajectories of 201 time steps, stored as structured Cartesian arrays.
PRDP: PROGRESSIVELY REFINED DIFFERENTIABLE PHYSICS: PRDP exploits the finding that differentiable physics and its Jacobian need not be fully converged during noisy-gradient network training.The method progressively refines solver iterations from a coarse initial level and adaptively increases refinement when training stagnates.
4 EXPERIMENTS: 33% savings were achieved with PRDP on the inverse Poisson problem using adaptive progressive refinement.The approach was designed to identify the sufficient refinement level without exhaustive searches over fixed iteration counts.
4.2 LINEAR NEURAL EMULATOR LEARNING: 48% savings were obtained for Jacobi-based linear neural-emulator training, while steepest-descent configurations achieved 43%.These savings held for both unrolled and implicit differentiation in the reported three-dimensional emulator setup.
4.4 NEURAL-HYBRID EMULATOR FOR THE NAVIER-STOKES EQUATION: 81% fewer iterations reduced wall-clock training time by 62% in the neural-hybrid Navier–Stokes experiment.The reduction combined 65% savings from incomplete convergence with 16% savings from progressive refinement.
6 LIMITATIONS AND OUTLOOK: PRDP achieves favorable outcomes across tested scenarios but remains to be evaluated for unstructured discretizations, higher dimensions, and multiple-physics systems.The conclusion emphasizes cheaper initial training with coarse physics followed by adaptive refinement toward higher accuracy.

F.2 HEAT DIFFUSION - AUTOREGRESSIVE NEURAL EMULATOR TRAINING … G.3 PRDP SAVINGS VS. NEURAL NETWORK EXPRESSIVENESS

The experiments train neural emulators and neural-hybrid correctors for heat, Burgers, and Navier–Stokes problems using differentiable physics and mixed-chain objectives. Supplementary results show that PRDP savings increase with computational scale while having little effect on emulator performance or dependence on expressiveness.

F.2 HEAT DIFFUSION - AUTOREGRESSIVE NEURAL EMULATOR TRAINING: Heat-diffusion emulators use discretizations with 30, 900, and 3D degrees of freedom, alongside MLP and ResNet architectures tailored to dimensionality.The 1D and 2D MLPs use three hidden layers, while the 3D ResNet uses six blocks with 32 hidden channels.
F.3 BURGERS EQUATION - AUTOREGRESSIVE NEURAL EMULATOR: Burgers emulation uses a 256-degree-of-freedom periodic discretization, GMRES with restart 2, and a six-block ResNet with 32 hidden channels.Training minimizes the nMSE after chaining the network’s first step with the physics operator’s second step against a direct-solver reference.
F.4 NAVIER STOKES - NEURAL-HYBRID CORRECTOR LEARNING: The Navier–Stokes neural-hybrid corrector learns on a coarse source manifold with 144 total degrees of freedom against a fine reference manifold with 291.The fine solver uses fully converged GMRES, while the model uses a three-block, 64-channel periodic ResNet and sums first- and second-step MSEs.
G.1 PRDP SAVINGS IN TERMS OF COMPUTE TIME: PRDP reduces inner solver iterations across experiments, but wall-clock savings emerge mainly for larger systems where linear solves dominate the training compute graph.For approximately 100 × 100 1D matrices, other compute-graph overhead masks iteration reductions; from 2D onward, training-time decreases approach iteration reductions.
G.2 NEURAL NETWORK PERFORMANCE - PRDP VS. BASELINE: PRDP targets lower end-to-end training cost without significantly affecting trained-network performance, with validation errors reported as means and standard deviations over 10 seeds.The supplementary comparisons include validation-accuracy measurements and examine how savings vary with network parameter count.
G.3 PRDP SAVINGS VS. NEURAL NETWORK EXPRESSIVENESS: Increasing the 1D Heat MLP parameter count by an order of magnitude improved accuracy with convergence rate approximately n−1/5, but had minimal impact on IC savings.The largest model showed slightly higher savings, likely because it triggered one fewer refinement level during training.

G.4 PRDP BASED ON A VALIDATION METRIC VS. ON TRAINING LOSS … A.4.1 Derivation of the Closed-Form Optimizer for the Linear Case

The paper establishes that neural emulators can outperform low-fidelity PDE solvers in specific regimes through interactions among inductive biases, training objectives, solver errors, and rollout dynamics. It develops this claim theoretically and empirically across linear and nonlinear PDEs, while showing that PRDP can use training loss but validation metrics generally provide smoother, more reliable refinement.

G.4 PRDP BASED ON A VALIDATION METRIC VS. ON TRAINING LOSS: PRDP guided by training loss successfully trains networks but refines more slowly because training loss can continue decreasing after validation loss plateaus.This yields slightly higher PR savings, while overall IC savings and final accuracy remain comparable.
G.4 PRDP BASED ON A VALIDATION METRIC VS. ON TRAINING LOSS: Validation loss is preferred for PRDP because it provides smoother convergence and may be more reliable than training loss on complex problems with multi-modality or spurious minima.The smoother progression is attributed to a more uniform refinement schedule.
Abstract: Emulator superiority occurs when neural networks trained on low-fidelity solver data achieve higher accuracy than those solvers against a higher-fidelity reference.The paper challenges the conventional assumption that emulator fidelity is bounded by training-data fidelity.
5 Limitations & Outlook: Superiority is regime-dependent rather than universal, with occurrence and intensity depending on the PDE, solver errors, architecture, objective, evaluation metric, and time horizon.Its practical relevance includes applications operating within defined boundaries, receding horizons, or near particular trajectories.
3 A Case Study for Superiority under Linear Conditions: Fourier analysis provides theoretical proof of state-space superiority for linear advection, diffusion, and Poisson equations, including forward and backward spectral mechanisms.The derivations identify mode-dependent error behavior and conditions under which a low-capacity ansatz can outperform its reference.
4 Extension to Nonlinear Settings: Nonlinear experiments confirm state-space and autoregressive superiority for advection and Burgers equations across diverse neural architectures.For advection, only the local feed-forward ConvNet generalizes to unseen modes as extensively as the linear case, whereas FNOs and Transformers are limited.
7 Conclusion & Impact: The analysis attributes superiority to interactions among emulator inductive biases, training objectives, solver error characteristics, and multi-step rollout dynamics.These interactions can let emulators implicitly correct structured numerical errors during inference.
A.4.1 Derivation of the Closed-Form Optimizer for the Linear Case: For single-mode linear training, the optimization problem has a closed-form solution independent of the mode-amplitude and phase distributions.With multiple modes, the loss becomes weighted least squares and the optimum depends on relative modal energy.

A.4.2 Proof that One-Step Supervised Training is equivalent to Discrete Residua … B.2.1 Implicit Time Integrators for the Burgers Equation

The section establishes when discrete PDE-residual training and one-step supervised training yield equivalent emulator optima, then details experiments spanning linear advection and Burgers equations. The Burgers setup contrasts a one-step unconverged implicit training reference with a fully converged high-fidelity testing reference.

A.4.2 Proof that One-Step Supervised Training is equivalent to Discrete Residua: For linear systems with a single active mode, discrete PDE-residual and one-step supervised losses have the same optimal emulator parameters.This extends the superiority analysis to the stated physics-informed training paradigm.
A.4.2 Proof that One-Step Supervised Training is equivalent to Discrete Residua: The residual loss is a spectrally weighted supervised loss, with identical objectives for explicit schemes and equivalent minimizers when only one mode is active.For explicit schemes, A = I; with one active mode, the relevant Fourier multiplier contributes only a constant scaling factor.
A.4.2 Proof that One-Step Supervised Training is equivalent to Discrete Residua: For implicit schemes with at least two active modes, supervised and residual-loss optimizers depend on modal energy distributions and are not easily related.Residual training additionally scales and rotates modal energy through element-wise multiplication by the Fourier multiplier of A.
B Experimental Details: Experiments use JAX and APEBench components, compare neural architectures with 27-33k trainable parameters, and train with MSE while testing with nRMSE.The listed architectures include ConvNet, UNet, and Transformer variants.
B.1 Nonlinear Emulators on Linear Advection: Linear-advection experiments vary γ1, the training simulator, and initial-condition distributions to test autoregressive and state-space generalization against analytical references.The domain uses N = 100 points, with implicit and explicit first-order upwind training references and testing distributions activating either the first mode or modes one to five.
B.1 Nonlinear Emulators on Linear Advection: These advection emulators train on 300 one-step trajectories and are evaluated on 10 initial conditions rolled out for 50 time steps.Optimization uses Adam with batch size 32, 10,000 iterations, and a warmup cosine-decay learning-rate schedule.
B.2 Burgers Superiority: The Burgers superiority evaluation uses the one-step (P1) implicit method as its training reference and baseline, linearizing the nonlinear term at the previous time step with one matrix solve per step.Because the nonlinear residual is not fully converged, this method produces shock-propagation errors, especially near sharp gradients.
B.2.1 Implicit Time Integrators for the Burgers Equation: Testing uses a fully converged nonlinear implicit method with residual tolerance 10^-5, while LU decomposition solves both training and testing linear systems to machine precision.The Burgers discretization uses N = 60, and the initial condition activates the first Fourier mode and the mean/zero mode.

B.2.2 Experimental Details … C.3 Effect of Coarse Solver Fidelity by ablating the number of Picard Iterations

The ablation studies evaluate neural emulators under controlled solver, data, receptive-field, spectral-mode, and Picard-iteration settings. They show that superiority depends on training-mode distribution, architecture-induced regularization, and structured error from coarse solver fidelity.

B.2.2 Experimental Details: The Burgers experiment uses a UNet on a 60-point grid in a strongly nonlinear regime with γ2 = 0.2 and δ1 = −8.0.Training uses one Picard iteration per time step, while evaluation uses the fully converged scheme as the high-fidelity reference.
B.2.2 Experimental Details: Fourier-based initial conditions concentrate on the first mode with randomized phases and a nonzero mean-mode offset, then develop shocks through Burgers dynamics.The experiment ran on a single NVIDIA RTX3060 GPU and completed all seeds in less than 5 hours.
B.2.2 Experimental Details: Training uses one-step MSE prediction on 500 samples, while evaluation performs 20-step autoregressive rollouts on 100 samples using nRMSE.The network is optimized with Adam for 100,000 iterations and evaluated across 30 initialization seeds.
C.1 Multi-Mode Case Study: When training includes multiple modes, forward superiority persists when the training modes are lower than the tested mode.The numerical study trains on combinations of modes and tests on mode 5, while the corresponding parameter analysis becomes a distribution-weighted linear least-squares problem.
C.2 Impact of Receptive Field in CNNs and Active Fourier Modes in FNO: ConvNet superiority is stronger than FNO or Transformer superiority, consistent with local inductive bias regularizing against global low-fidelity error patterns.A receptive-field ablation finds that superiority is maximized when the receptive field matches the problem’s physical characteristics.
C.2 Impact of Receptive Field in CNNs and Active Fourier Modes in FNO: FNO performance reflects a capacity–regularization trade-off: 1 active mode fails to learn, whereas 2 active modes achieves optimal performance.The ablation varies the number of active Fourier modes to examine how spectral truncation influences superiority.
C.3 Effect of Coarse Solver Fidelity by ablating the number of Picard Iterations: The Burgers ablation varies the number of Picard iterations in the training solver to relate coarse-solver fidelity to autoregressive superiority.The study again uses a UNet and reports the superiority-ratio rollout for different iteration counts.
C.3 Effect of Coarse Solver Fidelity by ablating the number of Picard Iterations: 0.85 is the ξ[t] ratio achieved with a very coarse training solver, while increasing solver fidelity reduces structured error and the potential for superiority.The results link stronger superiority to greater structured error in the training data, rather than identifying a universal fidelity threshold.

C.4 Additional Architectures for Burgers Study

The Burgers’ equation analysis was extended across all studied architectures, and the results robustly confirm that superiority is broadly associated with model inductive biases, especially spatial or spectral ones.

Additional architectures: The Burgers’ equation experiment was extended to all architectures from the study, with results reported as median superiority over 30 seeds.Table 5 extends the experiment of figure 6.
Additional architectures: The superiority phenomenon is not limited to a single architecture.The passage describes this conclusion as robustly confirmed across the extended architectural analysis.
Additional architectures: Superiority is closely tied to model inductive biases, particularly spatial or spectral biases.The passage identifies ConvNet and a truncated architecture name beginning with “Dil” among the relevant architectures.