PaperlayerView exact revision on arXiv

Source-linked AI summary

Do Neural PDE Solvers Learn the Right Dynamics?

Haonan Li, Yue Song, Bin Yang, Kaihong Luo

arXiv:2610.06952v1cs.LGnlin.CD

TL;DR

Prediction scores measure agreement with reference solutions but provide limited evidence about error accumulation, nearby-state divergence, and extreme events. We evaluate those dynamics by evolving nearby-state ensembles against DNS, and find that favorable prediction scores can coexist with systematic dynamical errors.

  • Problem

    Pointwise and distributional scores quantify agreement with reference solutions but provide limited insight into error accumulation, nearby-state divergence, and extreme events.

    Source: §1 · p. 2

  • Method

    The framework evolves ensembles of nearby initial states with neural solvers and DNS to examine error formation, ensemble geometry, and extreme-event statistics.

    Source: §1 · pp. 2–3

  • Results

    In two-dimensional Kolmogorov flow, lower trajectory errors can reflect weaker amplification despite larger residuals, while matched ensemble spread and event frequency can conceal incorrect directions and conditional events.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Improved prediction accuracy does not necessarily imply greater dynamical fidelity; the framework provides criteria for assessing whether solvers represent underlying dynamics more faithfully.

    Source: §1 · pp. 2–3

    Experiments are limited to two-dimensional Kolmogorov flow, so the results do not establish how broadly the observed failures extend.

    Source: A DISCUSSION AND FUTURE WORK · p. 13

Abstract

from arXiv · show

Neural PDE solvers can achieve low prediction errors, but do they reproduce the dynamics of the systems they model? Prediction scores alone offer an incomplete answer: they measure agreement with reference solutions but provide limited insight into how errors accumulate, nearby states diverge, or extreme events arise. We propose an evaluation framework that directly examines these behaviors in deterministic and stochastic neural solvers. By evolving ensembles of nearby initial states and comparing them with direct numerical simulation, we assess three complementary aspects of learned dynamics: error formation, ensemble geometry, and extreme events. Experiments on two-dimensional Kolmogorov flow reveal limitations that conventional scores can obscure. Smaller trajectory errors can reflect weaker error amplification despite less accurate local updates. Models can match an ensemble's overall spread and effective dimension while failing to capture the spatial directions where nearby states diverge. Similarly, matching overall event frequencies can conceal failures to predict persistent extreme events. These findings show that improved prediction accuracy does not necessarily imply greater dynamical fidelity. Our framework makes this distinction measurable, providing concrete criteria for evaluating whether advances in neural PDE solvers better capture the underlying dynamics.

1 INTRODUCTION

Prediction scores leave open how errors form, how nearby states diverge, and whether extreme events occur under the right initial conditions. In two-dimensional Kolmogorov flow, the three probes reveal that favorable scores can coexist with systematic dynamical errors.

Motivation and contribution

  • Conventional scores quantify agreement with reference solutions, but leave how predictions evolve and which dynamical properties they preserve insufficiently examined.The investigation complements those scores by tracing error formation, testing nearby-trajectory geometry, and resolving extreme-event statistics by initial condition.Source: §1 · pp. 2–3

Evaluation approach

  • The common protocol evolves densely sampled ensembles of perturbed initial states with neural solvers and DNS, comparing their evolution.For stochastic solvers, members share a random path, with fresh randomness at successive steps, so differences reflect initial-state responses under the same model randomness.Source: §1 · p. 2

Error formation

  • Larger local residuals can coexist with lower trajectory errors when existing discrepancies undergo weaker dynamical amplification.Early residuals also reduce ensemble-member differences relative to DNS advancement while increasing centered ensemble error against the DNS reference.Source: §1 · p. 2

Ensemble geometry

  • Models can match ensemble variance and effective dimension while misaligning substantially with the dominant DNS subspace at intermediate prediction times.The dominant DNS directions change as variance first concentrates into a few directions and later spreads across more.Source: §1 · p. 2

Extreme events

  • Matching overall extreme-event frequencies can conceal over- or underestimation of event probabilities within individual initial-state ensembles.These errors can cancel upon aggregation, concealing failures to predict events that persist under the reference dynamics.Source: §1 · p. 2

2 EXPERIMENTAL SETUP AND DYNAMICAL PROBES

The experiments use perturbed ensembles around sampled Kolmogorov-flow states to compare neural solvers with DNS. A shared set of initial states, data split, and prediction origin supports the three dynamical investigations.

Dynamical probes

  • All three investigations share the same initial states, data split, and prediction origin while comparing neural solvers and DNS.The ensembles probe collective responses to initial-state perturbations alongside individual prediction errors.Source: §2 · p. 3

Reference dynamics

  • The reference system is deterministic two-dimensional Kolmogorov flow, governed by incompressible Navier–Stokes equations with sinusoidal forcing and periodic boundaries.The flow exhibits chaotic evolution and develops localized intense vortices.Source: §2 · p. 3

Dataset construction and initial-state probes

  • Temporally decorrelated archive states define cases, and spatially structured perturbations of archive-state differences define nearby ensemble members.Perturbed test members have matched DNS references; training and validation use only unperturbed trajectories.Source: §2 · p. 3

Solvers and forecast evaluation

  • The evaluation includes configurations of U-Net, FNO, AFNO, DPOT, DySLIM, PDE-Refiner, ACDM, PFNO, and DYffusion, excluding configurations poor on conventional forecast metrics.Configuration labels specify input history length H, output block length P, and training unroll length K.Source: §2 · p. 3

3 DOES LOWER TRAJECTORY ERROR IMPLY MORE ACCURATE LOCAL UPDATES?

The section tests whether lower trajectory error signals more accurate updates and whether those updates preserve the evolution of nearby states. It finds that weaker error amplification can mask larger residuals, while residuals can contract ensembles and increase their centered error.

3 DOES LOWER TRAJECTORY ERROR IMPLY MORE ACCURATE LOCAL UPDATES?

  • The analysis separates update residuals, propagation of existing discrepancies, and their interaction to test what underlies trajectory error.Propagation can amplify or reduce existing error, while coupling can reinforce or offset the residual contribution.Source: §3 · p. 3, §3.1 · pp. 3–4

3.1 LOWER TRAJECTORY ERROR CAN CONCEAL LARGER RESIDUALS

  • U-Net H1 often has lower trajectory error than H4 despite larger cumulative residual injection, because existing discrepancies undergo weaker net amplification.Thus, lower forecast error does not indicate more accurate local updates.Source: §3.1 · pp. 3–4, Figure 2 · p. 4
  • FNO later-failing and non-crossing groups have similar median trajectory errors, although later-failing members already show substantially larger residual injection.Residual injection grows before propagation and coupling become insufficient to offset it, followed by spurious oscillations and runaway amplitudes.Source: §3.1 · p. 4, Figure 2 · p. 4

3.2 IMPROVED FORECAST ACCURACY CAN ACCOMPANY DISTORTED ENSEMBLE EVOLUTION

  • Across DPOT test cases, denoising residuals contract ensembles relative to DNS advancement while increasing centered paired error.Contraction can occur even when total ensemble spread continues to grow.Source: §3.2 · p. 5, Figure 3 · p. 5

4 DO SOLVERS PRESERVE THE GEOMETRY OF NEARBY TRAJECTORIES?

The section compares ensemble spread, effective dimension, and dominant spatial directions with DNS. Most tested solvers reproduce broad temporal trends in spread and dimension, but their dominant subspaces often misalign with DNS at intermediate times.

4 DO SOLVERS PRESERVE THE GEOMETRY OF NEARBY TRAJECTORIES?

  • The geometry analysis measures RMS spread, the number of POD modes explaining 90% of variance, and overlap between solver and DNS subspaces.Temporal reorientation compares each solver’s leading three-dimensional subspaces 10 steps apart.Source: §4 · p. 6, Figure 4 · p. 6
  • In DNS, ensemble variation first concentrates into a few dominant directions, then spreads across more directions, while the dominant subspace changes over time.Most tested solvers reproduce the broad trends in spread and effective dimension and exhibit temporal reorientation.Source: §4 · p. 6, Figure 4 · p. 6
  • Most tested solvers show low overlap with the dominant DNS subspace at intermediate times, even when spread and effective dimension are close to the reference.The misalignment occurs while variation remains concentrated in a few modes, affecting the dominant spatial patterns of trajectory separation.Source: §4 · p. 6

5 DO SOLVERS REPRODUCE EXTREME EVENTS UNDER THE RIGHT CONDITIONS?

Neural solvers can match aggregate extreme-event frequencies while mispredicting event-producing initial conditions, and recognizable vortices can still evolve with incorrect geometry. The section evaluates both event probabilities and vortex evolution against DNS.

5.1 CORRECT AGGREGATE FREQUENCIES CAN CONCEAL INCORRECT EVENT CONDITIONS

  • Events are defined by a same-sign connected region with |ω| ≥10 occupying at least 0.5% of the periodic domain, with probabilities estimated per case and time from ensemble members.Excess and deficit separately measure overprediction and underprediction relative to DNS; their signed difference is bias.Source: §5.1 · p. 7
  • U-Net H1’s 0.42% normalized bias coexists with 41.31% excess and 40.89% deficit, showing that aggregate agreement can result from cancellation.PDE-Refiner with shared noise likewise has a 0.44% bias and 73.04% combined absolute error.Source: §5.1 · p. 7, Table 1 · p. 7
  • 10 of 11 configurations predict no events in the DNS persistent-event group for case 1616; the remaining configuration also predicts events in the non-event group.The remaining configuration, U-Net H4 P4, predicts events with a frequency of 69.58% in the DNS non-event group.Source: Table 2 · p. 7, §5.1 · p. 8, Figure 6 · p. 8

5.2 RECOGNIZABLE VORTICES CAN DEFORM AND ROTATE INCORRECTLY

  • Predicted vortices remain recognizable but can deform and rotate differently from DNS, even when peak intensities are similar.AFNO retains a rounder core as DNS elongates, while U-Net H4 K8 and DPOT sustain elongation after DNS becomes rounder.Source: §5.2 · p. 8

6 IMPLICATIONS FOR EVALUATING NEURAL PDE SOLVERS

The paper argues that prediction scores should be assessed alongside the dynamics they may fail to capture. It recommends diagnostics tailored to physical systems and applications, while noting that training and benchmark choices can affect what is measured.

6 IMPLICATIONS FOR EVALUATING NEURAL PDE SOLVERS

  • The experiments identify three gaps: lower trajectory error can coexist with inaccurate local updates, matched ensemble spread can hide incorrect directions, and matched event frequencies can conceal missed event-producing conditions.The authors recommend evaluating local updates, ensemble geometry, and extreme events alongside conventional prediction scores.Source: §6 · pp. 8–9
  • Training perturbed and unperturbed states toward the same target may suppress physically meaningful differences, while multi-step rollout training can direct displaced states toward a reference continuation unlike their own.Perturbation scales, forecast intervals, and unroll lengths should be evaluated for effects on prediction accuracy and rollout state evolution.Source: §6 · pp. 8–9
  • Benchmarks should specify determinism, state sufficiency, relevant time scales, and physical outcomes, tailoring inputs, horizons, and criteria to the system and task.For hazard prediction, identifying event-producing conditions may matter more than a modest reduction in average field error.Source: §6 · p. 9
  • For stochastic solvers, predictive-distribution quality and the dynamical fidelity of generated trajectories require separate assessment.Other systems may require different diagnostics, but the aim is to identify which physical behaviors improve and which remain distorted.Source: §6 · p. 9

A DISCUSSION AND FUTURE WORK

The study is limited to two-dimensional Kolmogorov flow, and its diagnostics do not identify which model or training choices cause the observed departures. The discussion situates perturbed initial states as controlled probes of learned dynamics, distinct from operational ensembles used to represent initial-state uncertainty.

A DISCUSSION AND FUTURE WORK

  • Because the experiments cover only two-dimensional Kolmogorov flow, they do not establish how broadly the observed failures extend to other systems.Additional dynamical systems may reveal different relationships between prediction accuracy and dynamical fidelity.Source: A DISCUSSION AND FUTURE WORK · p. 13
  • The diagnostics identify departures from DNS but do not establish which architectural or training choices cause them.Controlled comparisons and tests across systems and operating conditions remain future work.Source: A DISCUSSION AND FUTURE WORK · p. 13

B.1 ENSEMBLE FORECASTING

  • In operational forecasting, perturbed states primarily represent uncertainty in the actual initial state, arising from measurement, coverage, and background-forecast errors.Ensembles have also been used to examine forecast skill and dynamical sensitivity, including extreme dissipation bursts in Kolmogorov flow.Source: §B.1 · p. 13
  • Here, dense ensembles of nearby states are deliberately constructed as controlled probes of learned evolution, not estimates of uncertainty about an unknown initial state.Their evolving errors, geometry, and extreme-event behavior reveal how learned dynamics respond to nearby states.Source: §B.1 · p. 13

B.2 EVALUATION OF NEURAL PDE SOLVERS

Prior evaluations of neural PDE solvers extend beyond pointwise prediction errors to benchmark suites, long-term statistics, and local dynamical properties. The paper distinguishes its finite-perturbation ensemble diagnostics from approaches based on tangent-space dynamics and local attractor properties.

B.2 EVALUATION OF NEURAL PDE SOLVERS

  • Benchmark suites cover equations, parameters, horizons, architectures, spatial scales, and rollout training, using measures such as field, conserved-quantity, spectral, and rollout errors.PDEBench, PDEArena, The Well, BLASTNet, and APEBench provide complementary coverage and diagnostics.Source: §B.2 · pp. 13–14
  • Other evaluations assess long-term statistics and dynamical invariants, including attractor statistics, Lyapunov measures, energy spectra, and distributional fidelity.Reported studies include methods that improve statistical accuracy, rollout stability, or long-term behavior in their tested systems.Source: §B.2 · p. 14
  • Local-dynamics studies assess tangent-space stability or attractor properties, whereas this paper follows finite initial-state perturbations and compares their evolving spread and dominant directions with reference dynamics.Prior approaches include covariant Lyapunov vectors, finite-time Lyapunov exponents, and recurrence-based dimension and persistence measures.Source: §B.2 · p. 14

C DNS, DATASET CONSTRUCTION, AND FORECAST PROTOCOL

The evaluation uses perturbed initial-state ensembles shared between DNS and neural solvers, with a common forecast origin and matched predictions. The data split separates conventional training and validation from dynamical ensemble evaluation.

DNS Settings

  • DNS resolves vorticity on a 128×128 Fourier pseudospectral grid and, with fixed forcing and viscosity, the full vorticity field specifies the reference dynamical state.Saved fields are separated by ∆t = 0.2; the internal time step is 0.002.Source: C DNS, DATASET CONSTRUCTION, AND… · p. 14

Dataset and Perturbations

  • 512 base states are drawn from an archive of 8,192 decorrelated stationary-flow states, and structured perturbations define nearby ensemble members shared across DNS and neural solvers.Each perturbation is rescaled to spatial mean square 10^-5; the diagnostic ensemble uses 256 perturbed members per test case.Source: C DNS, DATASET CONSTRUCTION, AND… · pp. 14–15
  • Training and validation use only unperturbed trajectories, while dynamical evaluation uses 256 perturbed members per test case and evaluates the unperturbed member separately.The cases are split into 416 training, 48 validation, and 48 testing cases.Source: C DNS, DATASET CONSTRUCTION, AND… · p. 15

Forecast Origin and Stochastic Sampling

  • Each solver receives ten DNS-evolved history steps and predicts from data step 11 onward, with outputs fed back for subsequent updates and predictions compared against DNS at matching times.Stochastic solvers share a random path across members within each case, using fresh draws at successive updates; one trajectory is generated per initial state.Source: C DNS, DATASET CONSTRUCTION, AND… · p. 15

D SOLVER CONFIGURATIONS AND FORECAST EVALUATION

The study evaluates selected solver configurations using conventional relative L2 forecast error and then probes their dynamics. The configurations span different architectures, training procedures, and stochastic forecasting methods.

D SOLVER CONFIGURATIONS AND FORECAST EVALUATION

  • Twelve configurations are retained for dynamical evaluation after screening by conventional relative L2 error; scores are case medians on unperturbed test trajectories.PFNO H10 P10, DYffusion H1 P16, FNO H1, and DPOT H10 trained from scratch are excluded during screening.Source: D SOLVER CONFIGURATIONS AND FORECAST… · p. 15
  • The evaluated methods include U-Net, FNO, AFNO, and DPOT architectures, while DySLIM is a distribution-matching training framework rather than a distinct architecture.The passage describes U-Net skip connections, Fourier-mode kernels in FNO, Fourier-space channel mixing in AFNO, and DPOT's autoregressive denoising pretraining.Source: D SOLVER CONFIGURATIONS AND FORECAST… · p. 15
  • Multi-step U-Net and AFNO settings use different gradient procedures, distinguishing pushforward training from full differentiation through a rollout.U-Net H4 K8 advances seven updates without gradients before differentiating the eighth-step loss; AFNO H4 K2 differentiates through both updates.Source: D SOLVER CONFIGURATIONS AND FORECAST… · p. 15
  • Training configurations vary in history, prediction block, and unroll length, and validation losses select checkpoints within each setup rather than provide comparable forecast scores across objectives.The table reports parameter counts and case-median relative L2 field errors on test trajectories.Source: Table 3 · p. 16, Table 4 · p. 16
  • FNO H10 K10 remains in the geometry comparison but is excluded from physical-event comparisons because of numerical instability.PFNO H10 P10 diverged under sampled autoregressive recursion; several other listed configurations had weaker rollout performance than retained settings.Source: D SOLVER CONFIGURATIONS AND FORECAST… · p. 16

E.1 TRAJECTORY ERROR

The trajectory-error analysis separates error introduced at an update from the propagation and interaction of existing discrepancies. It also examines how residuals affect ensemble evolution relative to DNS advancement from current states.

E.1 TRAJECTORY ERROR

  • The squared-error identity at successive updates requires neither linearization nor independence of the existing discrepancy and model residual, and a negative coupling term can offset a nonnegative squared residual.Averaging memberwise quantities yields the aggregate decomposition used in the analysis.Source: §E.1 · p. 17
  • A later-failing trajectory is defined by peak absolute vorticity exceeding ten times the maximum peak absolute vorticity of the aligned DNS ensemble for case 4232.The comparison uses all 50 threshold-crossing members and a fixed control group of 50 non-crossing members.Source: §E.1 · p. 17

E.2 RESIDUAL EFFECTS ON ENSEMBLE EVOLUTION

  • The same residual can reduce ensemble dispersion while increasing paired error because the two effects use different reference vectors in their coupling terms.These sums isolate residual effects relative to DNS advancement from current states, not endpoint changes that also include propagated ensemble differences.Source: §E.2 · p. 17

F.1 EVENT DEFINITION AND AGGREGATE PROBABILITY ERRORS

The methods define event-probability comparisons, persistent-event groups, vortex geometry measures, and ensemble-subspace diagnostics. Together, these procedures specify how the paper quantifies event statistics and the evolving geometry of nearby trajectories.

F.1 EVENT DEFINITION AND AGGREGATE PROBABILITY ERRORS

  • An event is a same-sign, connected component of |ω| ≥ 10 occupying at least 0.5% of the domain; aggregate probability errors compare solver and reference event probabilities over cases and stored times.Positive and negative probability-error parts are reported separately as Excess and Deficit, with their signed difference as Bias.Source: §F.1 · pp. 17–18

F.2 PERSISTENT-EVENT GROUPS

  • Persistent-event groups are selected by event or non-event outcomes under reference dynamics over forecast steps 90–100, and their frequency averages occupancy across members and stored times.This within-group statistic differs from the time-weighted aggregate probability errors.Source: §F.2 · p. 18

F.3 VORTEX GEOMETRY

  • Vortex geometry is characterized by peak absolute vorticity, core axis ratio, and accumulated major-axis rotation, with near-circular cores omitted from orientation tracking.Orientations with χ70 < 1.05 are omitted because their major axes are poorly determined; the reported rotation is relative to the DNS median.Source: §F.3 · p. 18

G ENSEMBLE GEOMETRY DIAGNOSTICS

  • Ensemble geometry is quantified through RMS spread, the rank capturing 90% of variance, and subspace overlap and reorientation based on leading POD modes.Solver–DNS comparison uses the smaller r90 at each case and time, while temporal reorientation uses a fixed rank of three.Source: G ENSEMBLE GEOMETRY DIAGNOSTICS · p. 19

Related papers

Loading 2610.06952v1…