Source-linked AI summary
Data Driven Governing Equations Approximation Using Deep Neural Networks
Tong Qin, Kailiang Wu, Dongbin Xiu
TL;DR
The paper addresses learning unknown governing equations from trajectory observations without requiring time-derivative data. It develops ResNet-based one-step and multi-step DNN approximations, including recurrent and recursive variants, and reports accurate finer-grid predictions while identifying error and data assumptions.
Problem
The paper seeks to create accurate models of unknown governing equations from solution-trajectory data collected at different time instances.
Method
The framework uses ResNet blocks as integral-form approximators, with recurrent RT-ResNet and recursive RS-ResNet multi-step constructions.
Results
RT-ResNet produces accurate predictions on finer time grids than the observation interval, while the proposed methods have no temporal discretization error.
Takeaways & Limitations
The methods can recover discrete dynamical systems from coarsely distributed trajectory data without requiring time derivatives.
Takeaways & Limitations
The general error bound does not provide structure-specific estimates, and the setup assumes a constant time lag for all data pairs.
Abstract
from arXiv · showhide
We present a numerical framework for approximating unknown governing equations using observation data and deep neural networks (DNN). In particular, we propose to use residual network (ResNet) as the basic building block for equation approximation. We demonstrate that the ResNet block can be considered as a one-step method that is exact in temporal integration. We then present two multi-step methods, recurrent ResNet (RT-ResNet) method and recursive ReNet (RS-ResNet) method. The RT-ResNet is a multi-step method on uniform time steps, whereas the RS-ResNet is an adaptive multi-step method using variable time steps. All three methods presented here are based on integral form of the underlying dynamical system. As a result, they do not require time derivative data for equation recovery and can cope with relatively coarsely distributed trajectory data. Several numerical examples are presented to demonstrate the performance of the methods.
1. Introduction.
The paper frames unknown dynamical-system learning as a function-approximation problem and introduces ResNet-based DNN structures designed for this task. These methods are developed for numerical evaluation across linear and nonlinear differential equations.
- The paper targets approximation and learning of dynamical systems using deep neural networks.
- ResNet blocks serve as the fundamental building blocks of the proposed DNN structures for dynamical-system learning.The paper distinguishes this construction from work viewing ResNet as Euler forward time stepping.
- A ResNet block can be interpreted as a one-step time integrator that is exact in temporal integration, with error arising from neural-network approximation.
- The paper introduces recurrent and recursive variations of the ResNet structure for multi-step approximation.
- Numerical examples cover linear and nonlinear differential equations to demonstrate the effectiveness of the proposed algorithms.
2. Setup.
The setup treats the governing equation as unknown and seeks to learn it from paired trajectory states observed at two time instances. With a constant time lag, these observations form input-output measurements of the lagged flow map.
- The governing equation f: R^n → R^n is unknown, and the goal is to construct an accurate model from solution-trajectory data.
- Each data pair contains solution states from one trajectory at two different time instances.
- The time lag Δ_j separates the two states in pair j, with potential measurement noises included in the observations.
- For notational convenience, all pairs use the same time lag, Δ_j = Δ.
- The resulting dataset is an input-output representation of the Δ-lag flow map.
3. Deep Neural Network Approximation.
The paper develops ResNet-based approximations of unknown dynamical systems from trajectory observations. Its one-step, recurrent uniform-step, and recursive adaptive-step constructions use integral-form dynamics, avoiding temporal discretization error and derivative data.
- Neural-network construction: A fully connected feedforward network approximates an R^n → R^n input-output map and serves as the core neural building block.The network has M ≥3 layers, including M −2 hidden layers, with n1 = nM = n.
- One-step ResNet approximation: ResNet explicitly adds the identity operator, training the network to approximate the effective increment rather than the full input-output map.The identity connection reintroduces the input and yields the residual mapping used for one-step approximation.
- One-step ResNet approximation: The one-step ResNet marches states by y(k+1) = y(k) + N(y(k); Θ) on a uniform grid, with temporal integration described as exact.The remaining error comes from approximating the effective increment and depends on data quality and network training.
- Data requirements: Because the constructions use the integral form of the governing equation, they require no time-derivative data and can accommodate relatively coarse trajectory sampling.The time lag ∆ therefore need not be made exceedingly small for numerical differentiation.
- Multi-step recurrent ResNet (RT-ResNet): RT-ResNet repeatedly applies one shared ResNet block K times, producing a uniform-step approximation with δ = ∆/K and no temporal discretization error.It can generate states on finer uniform grids even when training data span the larger interval ∆.
- Multi-step recursive ResNet (RS-ResNet): RS-ResNet recursively applies K distinct ResNet blocks with parameter sets Θk, approximating smaller data-determined increments through variable intermediate intervals.Its adaptive intervals are not explicitly known, so K applications are required to cover the training interval ∆; unlike RT-ResNet, it does not directly produce smaller uniform-step trajectories.
4. Theoretical Properties.
The analysis establishes continuity and composition properties of dynamical-system flow maps, providing theoretical support for ResNet-based approximations and general error-growth guidance.
- 4.1. Continuity of Flow Map.: Under Lipschitz continuity of f on D, the flow map Φ_t is Lipschitz continuous on D_τ for t ∈ [0, τ].This continuity follows from the dynamical system’s dependence on initial data.
- 4.3. Error Bound.: The Lipschitz continuity result supports arbitrary-accuracy neural-network approximation of the flow map and enters the subsequent error analysis.The paper notes that more specific error bounds for each network structure require future work.
- 4.2. Compositions of Flow Maps.: The flow map Φ_Δ can be represented as a K-fold composition of the shorter-time flow map Φ_δ.The representation follows from the flow-map composition property for autonomous systems.
- 4.2. Compositions of Flow Maps.: The residual flow map Φ_δ − I has norm O(Δ) for ResNet and O(Δ/K) for RT-ResNet with K > 1.As Δ becomes reasonably small, the flow map is close to identity, supporting approximation of its residual by a neural-network operator.
- 4.3. Error Bound.: A general error bound is derived for solution approximation using the DNN operator, serving as a guideline for error growth.The bound is presented generally rather than as a structure-specific estimate for each network.
5. Numerical Examples.
The numerical examples train ResNet, RT-ResNet, and RS-ResNet models from trajectory pairs separated by a relatively coarse Δ=0.1 interval, then test longer-time predictions against reference solutions. Across linear and nonlinear systems, the models produce accurate approximations, while RT-ResNet also resolves a finer time step and RS-ResNet is visually most accurate in one linear example.
- Training data: Training pairs are generated by sampling initial states uniformly over a computational domain and advancing each state by Δ=0.1 with a high-accuracy ODE solver.The coarse lag avoids relying on accurate numerical time differentiation because the methods use the integral form of the governing equation.
- Linear ODEs: For linear examples, the trained models are marched to t=2, where all three methods produce accurate predictions for Example 1.Example 1 uses D=[0,2]^2 and compares predictions with the reference solution shown in Figure 5.1.
- Linear ODEs: RT-ResNet resolves the finer step δ=Δ/K=1/30 with K=3 and agrees very well with the reference solution through t=2.This demonstrates prediction resolution finer than the training-data interval, although multi-step training can require more data and converge more slowly.
- Linear ODEs: In Example 2, RS-ResNet is visually more accurate than RT-ResNet, which is more accurate than the one-step ResNet model.The paper attributes this expected ordering to the multi-step methods and the adaptive nature of RS-ResNet.
- Nonlinear ODEs: For the damped pendulum, all three models show excellent agreement with the reference solution through t=20.The experiment uses α=8.91, β=0.2, and initial condition x0=(-1.193,-3.876).
- Nonlinear ODEs: For the genetic toggle-switch DAE, all three models produce accurate approximations during the long-time simulation through t=20.The experiment starts from x0=(19,17) on D=[0,20]^2.
6. Conclusion.
The paper presents ResNet-based DNN structures for approximating unknown dynamical systems from trajectory data, including one-step and two multi-step variants. These integral-form methods avoid time-derivative data and can handle coarsely distributed observations.
- 6. Conclusion.: The proposed DNN structures use ResNet as a one-step method and introduce recurrent RT-ResNet and recursive RS-ResNet as multi-step variations.The conclusion identifies all three as discrete dynamical-system approximations of the unknown governing system.
- 6. Conclusion.: All three methods are based on the integral form of the underlying system rather than requiring time derivatives of trajectory data.
- 6. Conclusion.: The methods can work with coarsely distributed trajectory data while approximating the underlying unknown governing equations.
- 6. Conclusion.: Numerical examples, construction details, and theoretical justifications are used to demonstrate the effectiveness of the proposed methods.