PaperlayerView exact revision on arXiv

Source-linked AI summary

When Known Physics Helps Neural PDE Models: Residual Constraints Out-Regularize Generic Priors for Nonlinear Dynamics

Zahra Farazpay, Aniruddha Bora

arXiv:2609.34012v1cs.LGcs.AI

TL;DR

The paper asks whether known structure improves neural PDE prediction beyond a strong from-scratch baseline, rather than merely supplying generic regularization. It evaluates equation residuals and other priors under controlled comparisons, finding a durable residual advantage for nonlinear operators when the equation is resolved, but not under nonlinear under-resolution or in the tested transfer regime.

  • Problem

    Structural-prior gains are hard to interpret when comparisons use weak references that do not separate physics-specific information from generic regularization or pretraining.

    Source: §1 · p. 1

  • Method

    The study compares structural priors with a matched from-scratch neural operator and validation-selected generic regularizers, including a known-equation residual.

    Source: §1 · pp. 1–2

  • Results

    The residual advantage persists and grows with capacity for nonlinear PDEs but collapses toward or below parity for linear controls, while under-resolution reverses its benefit and transfer does not beat the baseline in the studied regime.

    Source: §4 · p. 13, §6 · pp. 13–14

  • Takeaways & Limitations

    Known equations provide useful non-redundant constraints when the residual is valid, whereas invalid coarse-grid constraints can introduce structural bias.

    Source: §4 · p. 13, §6 · p. 14

    Under nonlinear under-resolution, unresolved modes induce closure terms, so the true projected trajectory violates the naive coarse residual and enforcing it introduces bias that more labels or model capacity cannot remove.

    Source: §3.4 · p. 7

Abstract

from arXiv · show

Neural PDE surrogates increasingly incorporate structural priors, yet it is often unclear whether their gains arise from physics-specific information or simply from regularization and training choices. We evaluate several such priors under a common protocol against a matched from-scratch neural operator baseline. Our central result is that a known-equation residual consistently outperforms the best generic regularizer at equal tuning budget. At fixed capacity this benefit appears across linear and nonlinear PDEs, but a capacity sweep reveals a sharp distinction: the advantage persists and grows for Burgers, KdV, and Allen-Cahn, while collapsing toward or below parity for linear heat and advection-diffusion. Thus, the durable value of the residual is specific to nonlinear operators. We further falsify a pre-registered hypothesis that the benefit is activated only by data sparsity: the residual remains advantageous even under full supervision. Its usefulness does, however, have a clear boundary. Under grid under-resolution, nonlinear coarse fields no longer satisfy the naive governing-equation residual, and enforcing it becomes actively harmful. In contrast, cross-family pretraining and in-context conditioning fail to outperform the strong from-scratch baseline in the regime studied. Together, these results identify when known physics provides non-redundant information to neural PDE models, when it does not, and when enforcing it introduces bias.

1 Introduction

The paper asks whether known structure improves neural PDE prediction beyond a strong from-scratch baseline, addressing comparisons that can confound physics-specific information with generic regularization. It finds that residual benefits depend on operator nonlinearity and equation validity, while cross-family transfer does not beat the baseline in the studied regime.

1 Introduction

  • Structural priors are difficult to evaluate against weak references because gains may reflect generic regularization or pretraining rather than the specific prior.The study therefore compares priors with a matched from-scratch neural operator and validation-selected generic regularization.Source: §1 · pp. 1–2
  • The known-equation residual improves held-out prediction at fixed width across five PDE families, but its capacity advantage persists and grows only for nonlinear families.For linear controls, the advantage collapses toward parity or below it.Source: §1 · p. 2
  • The residual remains beneficial at full supervision, falsifying the pre-registered hypothesis that its value is activated only by data sparsity.The main comparisons use equal tuning budgets against generic regularizers rather than unregularized networks.Source: §1 · p. 2
  • Under nonlinear under-resolution, unresolved scales make coarse fields violate the naive equation, so enforcing its residual can make the physics-penalized model substantially worse than the generic baseline.The paper identifies resolvable equation evaluation as a necessary condition for using the residual constraint.Source: §1 · p. 2
  • Cross-family pretraining and in-context conditioning do not beat the strong from-scratch baseline once more than one target trajectory is available in the studied regime.Relevance-gradient controls are consistent with optimization-basin compatibility mattering beyond source relevance alone.Source: §1 · p. 2
  • The contributions combine a controlled residual-versus-generic-prior comparison, a resolution boundary for residual validity, and a negative transfer result.The paper also uses pushforward, relevance-gradient, and resolution-versus-geometry controls to delimit which structural biases survive stronger baselines.Source: §1 · p. 2

2 Methods

The methods benchmark five one-dimensional PDE families with a matched neural-operator protocol, compare equation residuals against tuned generic alternatives, and test capacity, resolution, and transfer. A pre-registered sparsity hypothesis is evaluated under a statistical protocol, and the reported data falsify it.

2.1 PDE benchmark and neural operator

  • The benchmark uses five one-dimensional periodic PDE families, with heat and advection–diffusion as linear controls and Burgers, KdV, and Allen–Cahn as nonlinear systems.A residual-form Fourier Neural Operator is trained with matched one-step supervision for headline comparisons and evaluated by held-out rollout relative L2 error.Source: §2.1 · pp. 2–3

2.2 Physics residual and controlled baselines

  • The physics arm minimizes observed-data loss together with a pointwise Crank–Nicolson equation residual evaluated at all collocation points.Its coefficient is tuned against data-only training, weight decay, and input-noise augmentation using matched search budgets and seed-disjoint validation.Source: §2.2 · p. 3

2.3 Resolution, capacity, and transfer tests

  • Capacity tests vary model width at full supervision, while resolution tests coarsen the evaluation grid and transfer tests compare source-based initialization or conditioning with from-scratch models on the same target trajectories.A separate pushforward control reduces rollout error by 30–41% across three viscosity regimes, without adding equation information.Source: §2.3 · p. 3, Figure 1 · p. 4

2.4 Statistical protocol and pre-registration

  • Comparisons generally use five seeds and paired bootstrap intervals, and the pre-registered hypothesis that residual advantage would vanish at dense supervision was falsified.Effects are treated as resolved when the corresponding 95% interval excludes zero for differences or one for ratios.Source: §2.4 · p. 4

3 Results

Across the tested PDE families, a known-equation residual beats the best equally tuned generic regularizer, with capacity-persistent gains for nonlinear operators but not linear controls. The results also identify under-resolution as a failure boundary and find no transfer advantage over a strong from-scratch baseline in the studied regime.

3.1 Pushforward training strengthens rollout stability

  • Multi-step pushforward training reduces rollout error relative to one-step training by 34%, 41%, and 30% across the mild, sharp, and steep viscosity settings.This is a physics-free rollout-stabilization control; the headline residual comparison keeps both arms on the same one-step protocol.Source: §3.1 · p. 4

3.2 A known-equation residual out-regularizes generic priors

  • At the largest width, residual advantages persist for Burgers (3.1×), KdV (3.5×), and Allen–Cahn (2.5×), while heat approaches parity and advection–diffusion falls below it.At fixed width, the residual improves held-out prediction at every observation density, including full supervision; the pre-registered sparsity-gating hypothesis is therefore falsified.Source: §3.2 · pp. 4–5, Figure 2 · p. 5

3.4 The benefit reverses under nonlinear under-resolution

  • Under nonlinear under-resolution, Burgers and KdV physics-to-generic advantage ratios fall to roughly 0.16–0.17, as the true coarse-field residual rises from roughly 10^-7 to 10^-1.Unresolved scales create a closure term, so enforcing the naive coarse-grid residual introduces bias that more labels or model capacity cannot remove; resolve the dynamics or model the missing closure.Source: §3.4 · pp. 6–7, Figure 5 · p. 8

3.3 Non-redundant operator content predicts the capacity-persistent gain

  • Figure 4 reports that non-redundant content ρnr predicts the capacity-robust advantage Egeneric/Ephysics at largest width across five families (Pearson 0.90), while the two linear families at ρnr = 0 straddle parity.Heat is just above parity and advection–diffusion below. The authors call the correlation suggestive rather than conclusive and say it does not resolve Burgers-versus-KdV ordering.Source: §3.3 · p. 6, Figure 4 · p. 7

3.5 Cross-family transfer does not beat the strong baseline

  • By M = 2, from-scratch FNO and U-Net models overtake in-context FiLM and attention-over-support, whose held-out error curves remain nearly flat as target trajectories increase.Transformer meta-training converges, so the result is not explained simply by a visibly untrained conditioner.Source: §3.5 · pp. 7–8, Figure 6 · p. 9

3.6 Source relevance alone does not predict positive transfer

  • For relevant-physics pretraining, held-out error at M = 16 is 0.186 versus 0.094 from scratch, and related pretraining is significantly worse on KdV and Allen–Cahn too.Rich non-physical pretraining is inert to marginally helpful, while structureless pretraining is destructive; the pattern is more consistent with basin compatibility than source relevance alone.Source: §3.6 · pp. 9–10

3.7 Resolution invariance does not imply geometry generalization

  • Trained at N = 128, the 1D operators change rollout error by only -1.2% to -2.9% at N = 256 and 512, yet on unseen obstacle layouts the CNN beats FNO by 53%.The results show a task-dependent trade-off: resolution invariance does not imply geometry generalization, and the geometry margin can narrow or reverse in a harder regime.Source: §3.7 · p. 11, Figure 9 · p. 12

4 Discussion

The residual behaves as a non-redundant information channel when it remains valid, with capacity-persistent gains specific to nonlinear operators. Its usefulness is bounded by residual validity, and the paper’s broader interpretation remains empirical and scoped.

4 Discussion

  • The residual’s benefit persists under full supervision and increasing capacity only for nonlinear operators; under nonlinear under-resolution, closure terms make the naive residual encode the wrong coarse-grid relation.The authors interpret this as useful equation information when the constraint is valid and structural bias when it is not.Source: §4 · p. 13
  • The evidence is empirical and prediction-side, and does not establish a general loss-landscape mechanism.The interpretation is consistent with statistical-learning views of physics penalties as indirect data rather than ordinary regularization [Barajas-Solano, 2026].Source: §4 · p. 13
  • The residual-versus-generic comparison uses matched one-step training, while transfer and generalization experiments use multi-step supervision; the results are scoped to one-dimensional families and low source-family diversity.The discussion also distinguishes resolution invariance from geometry generalization: FNOs are effectively resolution-invariant in the tested one-dimensional setting, while a local CNN performs better on unseen obstacle geometry.Source: §4 · p. 13

5 Related Work

Related work spans equation-based constraints, discretization-flexible neural operators, and cross-family pretraining or adaptation. This paper focuses on controlled attribution against an equally tuned from-scratch baseline and connects nonlinear residual failure under under-resolution to spectral-truncation closure.

5 Related Work

  • Physics-informed neural networks and neural operators encode governing equations through residual penalties or operator-level constraints [Raissi et al., 2019, Li et al., 2024].Source: §5 · p. 13
  • FNOs target discretization-flexible mappings between function spaces [Li et al., 2020], while another line studies pretraining, foundation models, and in-context adaptation across PDE families [Subramanian et al., 2023, Herde et al., 2024, McCabe et al., 2024, Yang et al., 2023].Source: §5 · p. 13
  • The paper asks whether structural priors improve prediction beyond a strong, equally tuned from-scratch baseline, and links nonlinear residual failure under under-resolution to spectral-truncation closure.The full derivation and supporting experiments are in Appendix B.Source: §5 · p. 13

6 Conclusion

A known-equation residual outperforms equally tuned generic regularization on resolved problems, with a durable capacity advantage for nonlinear PDEs but not linear controls. Under-resolution can make the residual harmful, while cross-family transfer does not beat the strong from-scratch baseline in the studied regime.

6 Conclusion

  • The known-equation residual outperforms generic regularization under equal tuning budget and remains capacity-robust for nonlinear Burgers, KdV, and Allen–Cahn, unlike the linear controls.Source: §6 · pp. 13–14
  • Under nonlinear under-resolution, coarse fields no longer satisfy the naive governing equation, and enforcing that residual can become harmful.Source: §6 · p. 14
  • Cross-family transfer does not beat the strong from-scratch baseline in the studied regime, and source relevance alone does not predict successful transfer.Source: §6 · p. 14

A PDE families and data generation

The appendix specifies one-dimensional PDE data generation and a separate two-dimensional obstacle-geometry system. Headline and capacity experiments use fixed coefficients, while transfer and relevance experiments sample coefficients across trajectories.

A PDE families and data generation

  • Headline and capacity experiments use fixed coefficients within each family, with 64 train and 32 test trajectories of 40 stored snapshots.The stated coefficients are heat ν = 0.05, advection–diffusion c = 1.0 and ν = 0.03, Burgers ν = 0.02, KdV δ = 0.01, and Allen–Cahn ε = 0.002.Source: A PDE families and data generation · p. 15
  • Transfer and relevance experiments sample PDE coefficients trajectory-wise; the two-dimensional geometry experiment uses advection–diffusion on a 64 × 64 periodic grid with three random rectangular Dirichlet obstacles.The geometry system uses ν = 2×10^-3, c = (0.5, 0.3), and 20 stored snapshots.Source: A PDE families and data generation · p. 15

B Physics-residual formulas

The section defines a Fourier–Galerkin residual and shows why it is exact for linear dynamics but can be biased for nonlinear dynamics when unresolved scales create closure terms.

B Physics-residual formulas

  • On a resolved grid, the true trajectory satisfies the residual to the discretization floor; on a coarse nonlinear grid, omitted closure terms make enforcing r = 0 systematically biased.The residual is a valid training constraint only when the evaluation grid resolves the dynamics.Source: B Physics-residual formulas · p. 15

B.1 Consistency of the discrete constraint: a closure dichotomy

  • The dealiased Fourier–Galerkin closure term τN(u) := FN(PNu) − PNF(u) measures the difference between the truncated operator on the projected field and the projected full operator.Here PN retains resolved Fourier modes and QN = I − PN represents unresolved modes.Source: §B.1 · p. 16
  • For linear families, τN is exactly zero because the Fourier-multiplier linear operator commutes with truncation; nonlinear families instead have closure bounded by unresolved energy.The quadratic-flux bound scales with ∥QNu∥L2∥u∥Hs, while the cubic-reaction bound scales with ∥QNu∥L2 and a quadratic field factor.Source: §B.1 · p. 16
  • The true projected trajectory violates the one-step residual by −PNτN(ū) + O(∆t2), so increasing data or model capacity cannot remove this constraint-induced bias under nonlinear under-resolution.The passage states that resolving the dynamics is the way to remove the bias.Source: §B.1 · p. 16
  • For Burgers and KdV, the true projected solution’s residual defect comes from the closure term, which is zero for linear operators and nonzero under nonlinear under-resolution.This is the discrete constraint’s linear/nonlinear consistency distinction.Source: §B.1 · p. 16
  • Figure 11 verifies the closure bound: nonlinear families follow the predicted slope-1 relation, while linear families cluster at machine epsilon with τN ≡ 0.Fitted slopes are 0.99, 0.98, and 1.01 in the bound’s norm.Source: Figure 11 · p. 17
  • Under refinement, the closure term vanishes at the solution’s spectral-regularity rate; measured rates are 4.3 for Burgers and 3.7 for KdV, versus σ = 4.3 and 3.8.The passage notes that the studied viscous solutions are smooth and that the resolution boundary would sharpen in the inviscid, shock-forming limit.Source: §B.1 · p. 17

C Architectures, hyperparameters, selection, and cost

The experiments use matched neural architectures and tuning budgets, select configurations on a held-out validation ensemble, and measure the added cost of the physics penalty.

C Architectures, hyperparameters, selection, and cost

  • The 1D FNO and periodic-convolution UNet baseline are parameter-matched, while 2D FNO and CNN models use matched widths and depths with mask-conditioned inputs.The 1D FNO uses 16 modes, width 32, and three spectral-convolution layers; optimization uses Adam, warmup–cosine decay, clipping, and batch size 64.Source: C Architectures, hyperparameters,… · p. 17
  • The physics arm tests three penalty weights against a four-configuration generic grid of data-only training, weight decay, and input-noise augmentation.The two arms receive matched search budgets.Source: C Architectures, hyperparameters,… · p. 18
  • Each arm’s lowest-validation-error configuration is evaluated on a seed-disjoint test ensemble, with validation selection applied identically to both arms.The passage states that this selection protocol leaves absolute errors and the generic-to-physics error ratio unbiased.Source: C Architectures, hyperparameters,… · p. 18
  • The physics penalty adds 11–23% wall-clock time per training step, or 0.02–0.05 ms, depending on the number of spectral derivatives.The overhead is 11–12% for single-derivative families and 20–23% for families with two or three derivatives.Source: C Architectures, hyperparameters,… · p. 18

D Pre-registration record

The pre-registered sparsity and bandwidth hypotheses were tested against explicit criteria; the dense-supervision and bandwidth-crossover predictions were falsified, while the monotone sparsity trend was supported.

D Pre-registration record

  • The prediction that the physics advantage vanishes at full supervision was falsified: all three nonlinear families retain a physics win at ρ = 1.The registered rejection criterion was that the ρ = 1 interval exclude parity for any nonlinear family.Source: D Pre-registration record · p. 18
  • The predicted monotone sparsity trend was supported: the advantage increases as observation density ρ decreases..Source: D Pre-registration record · p. 18
  • The proposed bandwidth-set crossover was falsified because no inert dense regime appeared; the measured ρ⋆ was 0.156 for all families.The passage specifies m = 5 and N = 64 for this measurement.Source: D Pre-registration record · p. 18
  • The hypothesis that families separate by bulk bandwidth m99 was untestable in this suite because all five families share m99 = 5.The authors report the registered falsifications without reframing the headline as a sparsity law.Source: D Pre-registration record · pp. 18–19

E Per-condition results with confidence intervals

Across observation densities, the table reports held-out one-step relative L2 errors for the best generic and physics arms, with bootstrap intervals for their difference. The accompanying repository note identifies the code locations and table terminology.

E Per-condition results with confidence intervals

  • Every reported generic-minus-physics error difference has a 95% bootstrap interval excluding zero, so the generic-to-physics error ratio exceeds 1 significantly at every condition.The table covers five PDE families and six observation densities at N = 64.Source: E Per-condition results with… · p. 19
  • At full supervision, Burgers has Egeneric = 5.85×10−5 and Ephysics = 1.71×10−5, with a ratio of 3.41.These are the ρ = 1 values in the held-out one-step relative L2 table.Source: E Per-condition results with… · p. 19

G Multi-target relevance replication

Across Burgers, KdV, and Allen–Cahn, relevance-gradient replication again found relevant-physics pretraining significantly worse than from-scratch, while structureless sources were destructive.

G Multi-target relevance replication

  • Relevant-physics pretraining was significantly worse than from-scratch on the two new targets, while structureless sources were destructive.The rich non-physical operator was inert for KdV and marginally helpful for Allen Cahn.Source: G Multi-target relevance replication · p. 20
  • The replication tested Burgers, KdV, and Allen–Cahn, pretraining four source conditions and finetuning over M × reps × seeds (5 × 3 at each M).Table 2 reports the data-adequate endpoint at M = 16; full per-M curves appear in Fig. 7.Source: G Multi-target relevance replication · p. 20
  • For KdV at M = 1, relevant-physics pretraining had a transient head start over from-scratch that reversed by M = 4.The reported values were 0.525 [0.498, 0.550] versus 0.632 [0.567, 0.699] at M = 1.Source: G Multi-target relevance replication · p. 20

Related papers

Loading 2609.34012v1…