Source-linked AI summary
Learning data driven discretizations for partial differential equations
Yohai Bar-Sinai, Stephan Hoyer, Jason Hickey, Michael P. Brenner
TL;DR
Resolving fine PDE features directly is often computationally infeasible, while deriving reliable coarse-grained equations is difficult. The paper learns low-resolution discrete models from high-resolution solutions, using neural networks for spatial derivatives and physical constraints for time integration. The resulting methods accurately integrate several nonlinear one-dimensional PDEs at substantially coarser resolutions, although speed and stability remain important challenges.
Problem
PDE simulations may require unresolved fine-scale physics and prohibitively many grid points, while constructing reliable effective coarse-grained equations is challenging.
Method
Data driven discretization learns low-resolution discrete models directly from high-resolution solutions, using neural networks to parameterize spatial-derivative approximations with physical constraints.
Results
The learned methods outperform baseline schemes at low resolution and achieve accuracy comparable to increasing baseline resolution by about 8x on a larger inference domain.
Takeaways & Limitations
Data driven discretizations can solve several nonlinear one-dimensional PDEs accurately at substantially lower resolution while incorporating interpretable, physics-compatible numerical structure.
Takeaways & Limitations
Low training loss does not guarantee stable time integration, and the neural-network models require many more convolution operations than finite-difference schemes.
Abstract
from arXiv · showhide
The numerical solution of partial differential equations (PDEs) is challenging because of the need to resolve spatiotemporal features over wide length and timescales. Often, it is computationally intractable to resolve the finest features in the solution. The only recourse is to use approximate coarse-grained representations, which aim to accurately represent long-wavelength dynamics while properly accounting for unresolved small scale physics. Deriving such coarse grained equations is notoriously difficult, and often \emph{ad hoc}. Here we introduce \emph{data driven discretization}, a method for learning optimized approximations to PDEs based on actual solutions to the known underlying equations. Our approach uses neural networks to estimate spatial derivatives, which are optimized end-to-end to best satisfy the equations on a low resolution grid. The resulting numerical methods are remarkably accurate, allowing us to integrate in time a collection of nonlinear equations in one spatial dimension at resolutions 4-8x coarser than is possible with standard finite difference methods.
1 Data driven sub-grid scale modeling
Data driven discretization replaces difficult coarse-grained equation derivation with learned low-resolution approximations based on high-resolution solutions. The method targets unresolved physics while preserving accurate dynamics around sharp features such as shocks.
- Motivation: Standard discretizations require grid spacing smaller than the smallest feature, but resolving such features can be computationally intractable.Direct simulation of turbulent flow at Reynolds number about 1,000 used roughly 5 × 10^11 grid points.
- Core idea: Data driven discretization learns low-resolution discrete models directly from high-resolution solutions instead of deriving and discretizing an approximate coarse-grained continuum model.The learned models aim to encapsulate unresolved physics through approximations to the solution manifold.
- Illustration: Polynomial interpolation produces overshoots near shocks that compound during time integration, whereas neural-network interpolation closely matches the exact solution and reduces poor predictions.The neural network also makes much smaller errors in high-curvature regions corresponding to shocks.
- Learned discretization: The method learns equation-dependent spatial-derivative coefficients that can vary across space, allowing different regions such as shocks and smooth areas to use different approximations.Training uses high-resolution data and learns discrete derivative approximations on a coarse grid.
- Burgers’ equation: For Burgers’ equation, shock formation creates sharp local structure governed by relationships among shock height, width, and velocity.This structure motivates using equation-specific approximations rather than polynomial interpolation alone.
2 Models for time integration
The time-integration models use neural networks to learn pseudo-linear derivative filters while imposing conservation, flux, and accuracy constraints. These models improve coarse-resolution integration, but training loss alone does not ensure stable long-time evolution.
- Model parameterization: The network parameterizes spatial derivatives as pseudo-linear filters whose coefficients depend on neighboring field values through space and time.The representation generalizes finite differences and can adapt coefficients to local solution features.
- Model parameterization: Formal polynomial accuracy is imposed through a fixed affine transformation; the reported models use linear accuracy with a 6-point stencil.The authors describe this form as a generalization of ENO and WENO that replaces heuristic filter selection with optimized quantities.
- Physical constraints: Finite-volume modeling improves stability by evolving cell averages under the conservation law, with boundary fluxes determining each cell’s change rate.The network reconstructs boundary derivatives, computes the flux, and obtains each cell-average derivative from left-right flux differences.
- Physical constraints: Separating reconstruction, flux calculation, and time-derivative evaluation enables conservation and other physical constraints while keeping the learned component focused on reconstructing cell-boundary solutions.The intermediate quantities also have clearer physical interpretations.
- Training objective: The loss optimizes time-derivative accuracy rather than spatial-derivative accuracy, using mean squared error against derivatives obtained by coarse-graining fully resolved simulations.This directly targets the final prediction while incorporating physical constraints during training.
- Stability: Low training loss is necessary but insufficient for accurate, stable time integration, especially for low-dissipation equations and models without conservative constraints.For one constant-coefficient Burgers’ model, including the time-integrated solution in the loss was needed to avoid poor performance across resample factors.
- Interpretation: Learned coefficients differ from polynomial baselines near shocks and can show an upwind bias opposite the flow direction, consistent with a standard strategy for constraining temporal causality.The reported neural-network coefficients do not appear to use one-sided stencils near shocks.
3 Results
The learned discretizations accurately propagate nonlinear PDE dynamics on coarse grids, outperforming standard methods and generalizing from small training domains to much larger ones. Tests across Burgers’, KdV, and KS equations show strong low-resolution performance, with failure at sufficiently large resampling factors.
- Burgers’ equation: Learned Burgers’ models dramatically outperform the baseline at low resolution, while practically eliminating shock-induced ringing and numerical instabilities.The comparison includes constant-coefficient and solution-dependent learned models.
- Burgers’ equation: Training on [0, 2π] generalizes well to a domain of size [0, 20π], consistent with the model’s local, translation-invariant convolutional structure.The model also performs well in regions not represented in the training dataset.
- Burgers’ equation: The full neural network achieves mean absolute error comparable to increasing the baseline resolution by about 8x on a ten-times larger inference domain.The error is averaged over space, time less than 15, and multiple forcing realizations.
- Cross-equation evaluation: The method was evaluated on Burgers’, KdV, and KS equations using equation-specific estimators trained from high-resolution simulations.KdV provides a soliton-bearing example, while KS represents a classically chaotic PDE.
- Cross-equation evaluation: Valid simulation time measures when the low-resolution solution first exceeds a prescribed deviation threshold from the cell-averaged high-resolution solution.The authors use this metric to compare performance across different equations.
- Cross-equation evaluation: The neural network models have comparable or better median valid simulation times than all other methods across the tested equations and resolutions.Their advantage is particularly pronounced at low resolutions, while sufficiently large resampling factors eventually cause failure.
4 Discussion and conclusion
The paper frames data-driven discretization as learning numerical weights from PDE solutions, while identifying interpretability, speed, and dimensional scaling as remaining challenges.
- Data-driven discretization uses training data to parametrize the PDE solution manifold and solve equations accurately on low-resolution grids.
- Neural networks can reveal physically sensible local structure, such as an upwind bias, but extracting abstract scaling relations remains difficult.
- Neural-network models require many more convolution operations than finite differences, making speed a deployment challenge.The paper gives 322 = 1024 convolutions with a five-point stencil between its second and third layers.
- Scaling beyond one-dimensional regular grids remains challenging because real-world problems often use higher dimensions and irregular or adaptive meshes.
- In two and three dimensions, computational gains could scale like the square or cube of the resampling factor.
Appendix I: Neural network model
The appendix describes a convolutional model that predicts derivative-related quantities from coarse values, incorporates physical constraints, and is trained with specified optimization procedures.
- The model uses three fully convolutional layers, each with 32 filters and kernel size five, with physical constraints included.
- During deployment, network outputs are combined with coarse-grained values to estimate spatial derivatives, calculate fluxes, and obtain temporal derivatives.
- Training minimizes either time-derivative error or future evolved-state error, with the latter used only for constant-coefficient Burgers’ models.
- Models were trained with Adam for 4.0000 × 10^4 steps, reducing the learning rate by a factor of 10 after 2.0000 × 10^4 steps.
- Some models showed highly variable performance across training runs because of randomness and missing numerical-stability guarantees.
- Godunov numerical flux was used for the convective term v^2 in some models, with separate left- and right-sided estimates.
Appendix II: PDE parameters
The appendix specifies the one-dimensional PDE setup, randomized forcing or initial conditions, training-domain parameters, and dataset-generation procedures.
- The experiments solve three one-dimensional PDEs in conservative form with different fluxes, forcing functions, and initial conditions.
- All equations use periodic boundary conditions, with training-domain lengths L = 2π for Burgers’ equation, L = 32 for KdV, and L = 64 for KS.
- Randomness is introduced through forcing for Burgers’ equation and through initial conditions for KdV and KS.
- The random signal is a sum of long-wavelength sinusoidal functions whose parameters are independently and uniformly sampled from specified ranges.
- Figure S1 depicts training on high-resolution snapshots and inference through repeated application of the optimized model with the method of lines.
- The training dataset contains 8000 high-resolution solutions per equation sampled from 800 numerical integrations.
- Training and evaluation datasets use adaptive third-order Runge–Kutta integration, while Burgers’ training uses a fixed-step midpoint method.
Appendix III: Polynomial accuracy constraints
The appendix formulates spatial derivatives as stencil-weighted combinations and constrains the learned coefficients to retain polynomial accuracy.
- Spatial derivatives at x0 are represented as linear combinations of function values at N neighboring grid points.
- The neighbor offsets h_n may describe uniform or arbitrary meshes, while the main text uses h_n = n ∆x.
- Polynomial accuracy up to degree m is enforced through affine constraints on the stencil coefficients, provided m ≤ N − ℓ.
- Standard finite-difference coefficients arise by solving Taylor-series constraints that cancel terms through the desired order.
- Figure S2 shows five random training samples for Burgers’ equation, KdV, and KS.
- Removing lower-order constraints makes the coefficient system under-determined, so solutions combine a fixed formula with a null-space component.
Appendix IV: Interpolation model
The interpolation model predicts intermediate high-resolution values from an 8x-subsampled grid, using a neural network trained with accuracy constraints and curvature-informed evaluation.
- The interpolation network uses the same training datasets and a similar structure as the time-integration models, but predicts subsampled function values.
- The model predicts seven interpolated values between each adjacent pair of seed points on an 8x-subsampled grid.Training uses high-resolution simulations and mean squared error at each interpolated point.
- Curvature is defined as the maximum |∂2v/∂x2| over each interpolated interval and indicates whether interpolation crosses a shock front.The heuristic resembles smooth-region identification used by WENO methods.
Appendix V: Additional plots of coefficients
The appendix examines learned coefficient structures for field reconstruction and derivatives, showing equation- and resolution-dependent departures from standard centered schemes, including upwinding.
- The neural network learns upwinding when reconstructing the Burgers’ field value v, as it does for the first derivative.
- Optimized constant coefficients often differ significantly from standard centered coefficients, particularly for lower-order derivatives.The comparison covers Burgers’, KdV, and KS equations at resample factors where optimization improves performance.
- For Burgers’ and KS with Godunov flux, optimized coefficients use upwinding, especially at coarser resolutions.The figure shows left-ward-facing zeroth-derivative coefficients alongside standard upwinded finite-volume coefficients.
Appendix VI: Results for KdV and KS equations
Additional KdV and KS experiments show that neural-network discretizations outperform baseline finite-difference methods and generalize from the training domain to a domain ten times larger.
- Neural-network models significantly outperform baseline finite-difference methods for KdV and KS across the examined resample factors.The comparisons use particular solution realizations at multiple resolutions.
- The KdV and KS models generalize to a spatial domain ten times larger than the training domain.The 8x-resample-factor models are evaluated on both the training-sized domain and the larger validation domain.
- Mean absolute error is evaluated over short times because divergent models make long-time means undefined.The appendix averages over 100 training-domain realizations and 10 larger-domain realizations.
Appendix VII: Ablation study
The ablation study compares neural-network architectures with progressively stronger physical constraints, finding that constrained coefficient models are most consistent and that added Godunov flux does not directly improve predictions.
- Model variants: The ablation compares time-derivative, flux, space-derivative, coefficient, constrained-coefficient, and Godunov-constrained-coefficient models.These variants impose increasing levels of physical structure on the learned discretization.
- Model variants: Finite-difference models can work but generally perform worse than the finite-volume models used in the main experiments.
- Additional diagnostics: The KdV and KS solution plots compare baseline, optimized-constant-coefficient, neural-network, and exact coarse-grained solutions, with blank regions marking solver divergence.
- Ablation results: Consistent results across equations and resolutions were achieved only with constrained coefficients and Godunov constrained coefficients.For Burgers’ and KS, these models achieved good results on all training runs.
- Ablation results: Physical constraints dramatically improve performance on fine-resolution grids but have less influence at low resolutions.Fine-resolution models use smaller timesteps and are applied more times over the same integration interval.
- Ablation results: Godunov flux does not directly improve neural-network predictions because the relevant equation is simple enough for the network to learn.