PaperlayerView exact revision on arXiv

Source-linked AI summary

Evolutional Deep Neural Network

Yifan Du, Tamer A. Zaki

arXiv:2103.09959v1physics.comp-phcs.LGphysics.flu-dyn

TL;DR

Predicting PDE dynamics far beyond a network’s training horizon remains challenging. EDNN represents spatial structure with a neural network and advances the solution by marching its parameters. Across several PDE examples, its results agree with analytical or reference solutions, including long-time chaotic-flow predictions.

  • Problem

    Predicting PDE dynamics over horizons far beyond the training horizon remains a challenge for neural-network approaches.

    Source: §1 · p. 1, §2 · p. 2

  • Method

    EDNN represents the spatial solution with a neural network and advances it by marching the network parameters using an evolution equation.

    Source: §1 · p. 2, §4 · p. 16

  • Results

    EDNN solutions agreed with analytical or reference spectral solutions across several PDEs, with accurate long-time predictions and statistics for chaotic turbulent flow.

    Source: §4 · pp. 16–17

  • Takeaways & Limitations

    EDNN maintains explicit time dependence and causality and can simulate very long chaotic PDE evolutions that are difficult for other neural-network methods.

    Source: §4 · p. 17

    For the inviscid Burgers equation, simulations were limited to evolution before formation of the N-wave.

    Source: §3.2 · p. 10

Abstract

from arXiv · show

The notion of an Evolutional Deep Neural Network (EDNN) is introduced for the solution of partial differential equations (PDE). The parameters of the network are trained to represent the initial state of the system only, and are subsequently updated dynamically, without any further training, to provide an accurate prediction of the evolution of the PDE system. In this framework, the network parameters are treated as functions with respect to the appropriate coordinate and are numerically updated using the governing equations. By marching the neural network weights in the parameter space, EDNN can predict state-space trajectories that are indefinitely long, which is difficult for other neural network approaches. Boundary conditions of the PDEs are treated as hard constraints, are embedded into the neural network, and are therefore exactly satisfied throughout the entire solution trajectory. Several applications including the heat equation, the advection equation, the Burgers equation, the Kuramoto Sivashinsky equation and the Navier-Stokes equations are solved to demonstrate the versatility and accuracy of EDNN. The application of EDNN to the incompressible Navier-Stokes equation embeds the divergence-free constraint into the network design so that the projection of the momentum equation to solenoidal space is implicitly achieved. The numerical results verify the accuracy of EDNN solutions relative to analytical and benchmark numerical solutions, both for the transient dynamics and statistics of the system.

1 Introduction

Neural networks offer tools for PDE solutions, but predicting dynamics far beyond the training horizon remains a challenge. EDNN addresses this by representing spatial dependence with a network and evolving its parameters to predict time-dependent solutions.

1 Introduction

  • Long-time PDE prediction remains difficult when the desired horizon far exceeds the network’s training horizon.The introduction identifies this as the key challenge motivating EDNN.Source: §1 · p. 1
  • Neural networks can approximate functions given sufficient neurons, samples, and computational resources, while automatic differentiation supports gradient calculation and optimization.The passage also describes convergence-rate estimates with respect to network depth and width.Source: §1 · p. 1
  • Operator-learning methods map equation inputs such as initial or boundary conditions to PDE solutions, often using data from independent simulations.Generating enough simulations to span the space of interest can be computationally expensive, although evaluation after training is efficient.Source: §1 · pp. 1–2
  • Solution-representation methods train a network over spatio-temporal coordinates, but PINNs can become costly over long horizons and their optimization can be frustrated by loss weighting.PINNs represent behavior across space and time in one network; time-parallel and high-order time-discrete variants are also noted.Source: §1 · p. 2
  • EDNN represents spatial dependence with a neural network and predicts time evolution by marching the network parameters.The introduction also previews embedded boundary and divergence-free constraints and investigations of convergence and long-time prediction.Source: §1 · p. 2

2 Methodology

EDNN represents a PDE solution spatially at one instant and advances it by marching network parameters under the governing equation, rather than training over the full space-time domain. Linear constraints, including boundary conditions, are embedded in the network so they hold exactly along the trajectory.

2 Methodology

  • EDNN needs network capacity for a spatial solution at one instant; PDE-driven parameter updates then produce the trajectory, potentially indefinitely, while conventional PINNs face rising complexity over longer horizons.The method treats network discretization in space and numerical marching in time as equivalent to discretizing the PDE and marching its solution.Source: §2 · pp. 2–3, Figure 1 · p. 3, Figure 2 · p. 3

2.1 Evolutional network parameters

  • Network weights are initialized by training on initial data, then their time derivative is chosen to match the network’s time derivative to the PDE operator and numerically integrated.The derivative is computed using the neural-network gradient and PDE operator at collocation points; forward Euler or fourth-order Runge–Kutta can advance the weights.Source: §2.1 · pp. 4–5

2.2 Embedded constraints

  • For linear constraints, EDNN constructs an auxiliary operator and network class so the resulting solution satisfies the constraint exactly throughout its trajectory, unlike penalty-based enforcement.The paper applies this framework to periodicity, homogeneous Dirichlet conditions, and divergence-free flow.Source: §2.2 · p. 5

2.2.1 Periodic boundary conditions

  • For periodic solutions on [0, 2π], the auxiliary construction makes the periodicity operator applied to the network output equal zero.The paper notes that replacing coordinates with sin(x) and cos(x) is another example of the general constraint framework.Source: §2.2.1 · pp. 5–6

2.2.2 Dirichlet boundary conditions

  • The Dirichlet construction cancels boundary values to impose homogeneous conditions at machine-zero error on arbitrary geometry; an inhomogeneous condition can be added through a boundary-valued function.The Green’s-function construction also gives smoothness but requires an expensive integral evaluation; the proposed efficient construction is demonstrated in two dimensions and extends to higher dimensions.Source: §2.2.2 · pp. 6–7

2.2.3 Divergence free

  • For divergence-free constraints, an auxiliary network is mapped to a velocity field satisfying zero divergence; in two-dimensional fluid mechanics, the auxiliary function is a stream function.The framework describes constructions for both two- and three-dimensional domains.Source: §2.2.3 · p. 7

3 Numerical results

Across heat, transport, chaotic, and incompressible-flow examples, the results examine EDNN accuracy against analytical or spectral references and the effects of spatial and temporal resolution. EDNN also predicts long-time Kolmogorov-flow statistics in a turbulent regime.

3.1 Parabolic equations

  • Heat-equation predictions agree with the analytical solution, and errors decay monotonically; larger networks and lower initial-state error improve accuracy throughout evolution.After a short transient (t > 0.2), the error decays exponentially as expected.Source: §3.1 · p. 8, Figure 5 · p. 9

3.2 Hyperbolic equations

  • For linear waves, EDNN agrees with the analytical solution and errors remain low without amplifying; for Burgers, it captures the developing steep, nearly discontinuous N-wave profile.The Burgers prediction is compared with a reference solution at early and late stages.Source: §3.2 · pp. 9–10, Figure 6 · p. 10, Figure 7 · p. 10

3.3 Kuramoto-Sivashinsky equation

  • With ∆t = 10^-3, KS case 3k agrees with the spectral reference; Runge-Kutta marching and smaller steps delay error amplification, though chaotic trajectories ultimately diverge exponentially.The initial state has O(10^-3) relative error, which is propagated and magnified along the chaotic trajectory.Source: Figure 9 · p. 11, §3.3 · p. 12

3.4 Incompressible Navier-Stokes equations

  • For incompressible Navier-Stokes, EDNN maintains a divergence-free solution so the projection onto solenoidal space is realized implicitly during parameter evolution.The parameter derivatives are divergence-free, and the least-squares solution of the associated linear system realizes the projection.Source: §3.4 · pp. 12–13

3.4.1 Taylor-Green vortex

  • Taylor-Green energy decay compares favorably with the analytical solution, while increasing spatial resolution or reducing ∆t lowers prediction error until it saturates below ∆t = 10^-4.The divergence-free condition is satisfied to machine precision.Source: §3.4.1 · p. 14, Figure 11 · p. 14

3.4.2 Kolmogorov flow

  • For n = 2 Kolmogorov flow, EDNN agrees with spectral long-time statistics at coarser resolution over 4×10^5 time units.The reference spectral simulation ran for 6×10^5 time units; EDNN also predicts the laminar trajectory and reaches a turbulent state in the reported tests.Source: §3.4.2 · p. 16, Figure 14 · p. 16

4 Conclusions

EDNN represents spatial dependence with a neural network and predicts PDE evolution by marching its weights, while embedding boundary conditions and other linear constraints. Across several PDEs, its predictions agree with analytical or reference solutions, and the method also captures long-time chaotic behavior; its main computational weakness is a dense linear solve.

4 Conclusions

  • EDNN represents spatial dimensions with a neural network and advances PDE evolution by marching its weights, enforcing boundary conditions and linear constraints through auxiliary functions and operators.The marching direction can be adapted, such as using the streamwise direction for boundary-layer flows.Source: §4 · p. 16
  • EDNN solutions agree with analytical or reference spectral solutions across heat, wave, Burgers, Kuramoto-Sivashinsky, and Navier-Stokes problems.For incompressible Navier-Stokes, an embedded divergence-free constraint realizes the projection step automatically.Source: §4 · p. 16
  • For Navier-Stokes, EDNN accurately predicts early laminar and long-time chaotic turbulent Kolmogorov flow, highlighting its ability to simulate long horizons.Accuracy improves monotonically as network structure and spatio-temporal resolution are refined.Source: §4 · p. 17
  • Unlike methods optimized over the whole spatio-temporal domain, EDNN represents an instantaneous solution snapshot and retains explicit time dependence and causality.The authors note that this can make its structural complexity significantly smaller for a specific PDE problem.Source: §4 · p. 17
  • EDNN's main computational cost is forming and inverting a dense, nonsparse linear system; domain decomposition is proposed to create a block-sparse system and enable parallelism.The method's flexibility includes simple implementation for differential operators, no need for complex geometric grids, and dynamic refinement of collocation points.Source: §4 · p. 17

Related papers

Loading 2103.09959v1…