PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient Approximation of Solutions of Parametric Linear Transport Equations by ReLU DNNs

Fabian Laakmann, Philipp Petersen

arXiv:2001.11441v1math.NAcs.NAmath.APmath.FA

TL;DR

High-dimensional transport solutions with non-smooth initial data are difficult to approximate, and direct approximation faces a curse of dimension. The paper uses ReLU networks to represent solutions through characteristic-flow composition and presents approximation results for several transport-equation cases. The supplied results also identify a boundary: dimension-independent rates are not retained for the stated parameter-dependent initial-data setting.

  • Problem

    High-dimensional transport solutions with non-smooth initial data are difficult to approximate, and direct approximation faces a curse of dimension.

    Source: §1 · p. 1

  • Method

    The paper leverages the compositional form of transport solutions, in which the initial condition is composed with flow along characteristic curves, for ReLU-network approximation.

    Source: §4 · p. 10

  • Results

    The paper presents approximation results for solutions in each of the transport-equation cases it discusses.

    Source: §4 · p. 10

  • Takeaways & Limitations

    The results support the feasibility of approximating transport solutions with deep networks when solution samples are available despite unknown equation data.

    Source: §1.1 · p. 2

    When the initial condition depends on parameters, the stated Sobolev setting yields an approximation rate depending on parameter dimension D.

    Source: §5 · p. 18

Abstract

from arXiv · show

We demonstrate that deep neural networks with the ReLU activation function can efficiently approximate the solutions of various types of parametric linear transport equations. For non-smooth initial conditions, the solutions of these PDEs are high-dimensional and non-smooth. Therefore, approximation of these functions suffers from a curse of dimension. We demonstrate that through their inherent compositionality deep neural networks can resolve the characteristic flow underlying the transport equations and thereby allow approximation rates independent of the parameter dimension.

1 Introduction

The paper studies ReLU-network approximation of parametric linear transport solutions, whose high-dimensional, low-regularity form challenges classical methods. It uses the compositional structure induced by characteristic flows to motivate dimension-independent approximation rates and relates this approach to applications and prior approximation methods.

1 Introduction

  • High parameter dimension and low initial-data regularity make direct approximation of transport solutions extremely challenging for classical methods.The introduction frames the target as a high-dimensional function of low regularity.Source: §1 · p. 1
  • Deep ReLU networks exploit the compositional structure of characteristic flows to approximate solutions efficiently, with rates independent of parameter dimension in the prescribed setting.The paper contrasts this with classical regularity-based approximation rates.Source: §1 · pp. 1–2

1.1 Applications and relevance

  • Deep networks adapt to characteristic-curve regularity and benefit from smoother curves, unlike prior specialized systems tied to linear or C2-regular curves [14] and.The paper presents this adaptation as relevant to approximation theory and as a higher-order method.Source: §1.1 · p. 2
  • For measured solution samples with unknown initial data, source, and velocity, the transport problem becomes supervised learning, while classical PDE solvers lack the needed f and V.The authors say their approximation results show this learning approach is feasible.Source: §1.1 · p. 2
  • The approach requires little structure in parameter dependence: smooth dependence of V and f suffices, unlike linear reduced-order models that typically need additional assumptions.The cited empirical comparison with reduced-order models is reported in [21].Source: §1.1 · p. 3

1.2 Related work

  • Prior work establishes ReLU approximation rates comparable to classical approximation tools, while high-dimensional rates independent of ambient dimension require exploitable structure.The paper situates transport solutions among structured high-dimensional PDE approximation problems.Source: §1.2.1 · p. 3, §1.2.2 · p. 3, §1.2.3 · p. 4

2 Neural Networks

This section formalizes ReLU neural networks as weight sequences with associated realizations, distinguishing the network from the function it represents. It collects composition, parallelization, sum, and approximate multiplication operations, alongside a standard smooth-function approximation result.

2 Neural Networks

  • A neural network is treated as a sequence of matrix-vector tuples, with an associated realization defined through layerwise ReLU operations and possible skip connections.The framework distinguishes the network’s weights from its realized function.Source: §2 · p. 5

2.1 Standard operations on neural networks

  • Standard constructions compose or parallelize networks and form sums; multiplication is emulated only approximately, with accuracy controlled by ε.The section also introduces an operation for summing previously approximated functions.Source: §2.1 · pp. 5–6

2.2 Approximation of smooth functions

  • The section invokes a standard result for approximating smooth functions by neural-network realizations, with its constant depending on smoothness and dimension.For balls of larger radius, the constant also depends on the radius while the asymptotic behavior in ε remains unchanged.Source: §2.2 · p. 7

3 Linear Transport Equations

The section formulates parametric linear transport equations and uses characteristic curves to describe their solutions. It covers standard equations, weak solutions for rough initial data, and extensions with source terms or conservative form.

3 Linear Transport Equations

  • The standard problem evolves u by the vector field V, with initial data u0 and parameters η; the solution is constant along characteristic curves.The characteristic flow maps a point back to the origin of its curve, where the initial data determine the solution.Source: §3 · p. 8
  • Under smoothness and growth assumptions on V, characteristic curves exist globally and have the regularity needed to bound the flow map X.The flow-map bound is established to quantify the complexity of its neural-network approximation.Source: §3 · pp. 8–9

3.1 Solutions of the standard linear transport equation

  • For differentiable initial data, the standard equation has a unique solution given by composing u0 with the characteristic flow evaluated at time zero.The solution has regularity C^min{s,k} under the theorem's assumptions.Source: §3.1 · p. 9
  • For bounded initial data, the equation admits a global weak solution, also represented by composing the initial condition with the characteristic flow.The weak formulation is introduced for initial data that need not be differentiable.Source: §3.1 · p. 9

3.2 Solutions of extensions of the parametric linear transport equation

  • With a source term, solution existence and representation extend to non-homogeneous equations; the conservative formulation also has a corresponding solution representation.The source-term result allows a continuous source in the weak setting when the initial data are continuous.Source: §3.2 · pp. 9–10
  • The conservative equation's solution is given by a characteristic-flow representation, and this formulation reduces to the standard equation when div_x V = 0.The conservative result also has a weak-solution counterpart.Source: §3.2 · p. 10

4 DNN Approximation of Solutions of Linear Transport Equations

The section approximates transport-equation solutions by exploiting their composition of initial data with a characteristic flow. It extends the approach to weak solutions, source terms, and conservative form, with approximation guarantees under stated regularity and approximability assumptions.

4 DNN Approximation of Solutions of Linear Transport Equations

  • The approach exploits the compositional solution form by approximating the characteristic flow and initial data separately, then composing their networks.The paper presents approximation results for each transport-equation case considered in the preceding section.Source: §4 · p. 10, §4.1 · p. 12

4.1 Standard linear transport equations

  • For standard equations, approximation guarantees cover weak solutions with Lipschitz-continuous initial conditions, beyond the continuously differentiable case.The weak-solution extension is needed when the initial condition is not continuously differentiable.Source: §4.1 · p. 11
  • For non-smooth solutions, the stated construction gives a weight bound proportional to (ln(1/ε) + 1)ε^-1, versus (ln(1/ε) + 1)ε^-d for direct approximation.This comparison is presented for the setting where u0 is not in W^2,∞.Source: §4.1 · p. 12

4.2 Non-vanishing source term

  • For non-vanishing source terms, approximation results require either an η-independent source with specified regularity or an η-dependent source with higher regularity.The paper states that less regularity suffices in the η-independent case.Source: §4.2 · p. 13
  • The source-term construction approximates the source antiderivative and the transported initial data separately, then combines the resulting networks.The construction uses a network for the antiderivative of f with respect to the first coordinate.Source: §4.2 · pp. 14–15

4.3 Conservative form

  • For the conservative formulation, the construction approximates the flow Jacobian and transported initial data, then combines them to approximate the solution.The Jacobian is approximated before multiplication with the network for the composed initial data.Source: §4.3 · p. 16

5 Extensions

The extensions discuss damping or amplification, parameter-dependent initial data, discontinuous initial conditions, and nonlinear transport. The authors outline where their approximation results extend and where their argument is constrained.

Non-linear transport equations

  • For nonlinear transport, nonsmooth initial data can leave characteristic curves nonsmooth, undermining the smooth-flow composition underlying the approximation argument.The authors therefore consider similar results unlikely without overly restrictive assumptions.Source: §5 · p. 17

Damping/amplification

  • Adding a damping or amplification factor is a straightforward extension, with the solution represented using the characteristic flow and an integral of the coefficient a.The approximation construction combines a neural-network approximation of the integral, the exponential function, and the transported initial condition.Source: §5 · p. 18

Parameter dependence of initial condition

  • If the initial condition depends on parameters, the main results extend directly when it depends on only a few parameters or has a suitable decoupled form.For dependence on t parameters, the stated weight count is O(ε^−(n+t)/s), rather than O(ε^−n/s) when approximating u0(x).Source: §5 · p. 18

Weak solutions with discontinuous initial condition

  • For discontinuous initial conditions, uniform-norm approximation is unavailable because neural-network realizations are continuous, while Lp approximation for finite p can handle piecewise-regular functions.The passage points to [46] for this setting.Source: §5 · p. 18

A Proof of Proposition 3.4

The proof bounds the characteristic flow and its derivatives by iterating estimates based on the flow equation, growth assumptions, and Gronwall’s inequality.

Zeroth-order bound

  • The proof first bounds the flow itself using the growth condition and Gronwall’s inequality, obtaining ∥X∥C0 ≤ (|K| + CT) exp(CT).The argument then derives first- and second-derivative bounds and identifies the pattern for higher derivatives.Source: A Proof of Proposition 3.4 · p. 19

Derivative bounds

  • First- and second-derivative estimates follow by differentiating the flow equation, applying the Leibniz rule, and using Gronwall’s inequality.The proof drops η-dependence for simplicity, noting that derivatives with respect to η are easy to compute and bound.Source: A Proof of Proposition 3.4 · pp. 19–20

Higher derivatives and bound sharpness

  • Iterating the derivative estimates yields a kth-derivative bound, with factors 2^k and T^(k−1) arising from the product rule and repeated time-integral bounds.The authors caution that the resulting Ck bound may not be sharp, but only requires a data-dependent constant for the approximation result.Source: A Proof of Proposition 3.4 · p. 20

B Construction of a NN emulating the left Riemann sum

This appendix constructs a neural network that emulates a left Riemann sum for integrating a function over time. The construction uses shifted networks and indicator networks to assemble the sum.

Network construction

  • The construction samples a network Φ at grid times and combines the shifted evaluations with indicator networks that transition across each time interval.The indicators are 0 before t_i, increase linearly to 1 between t_i and t_(i+1), and remain 1 afterward.Source: B Construction of a NN emulating the… · p. 21

Network-size bound

Left Riemann sum

  • The left Riemann sum approximates the integral of f over its first argument from 0 to t using samples f(t_i,x) at t_i=iT/N.The passage states this setup for f in C1([0,T] × [−M,M]^n; R).Source: B Construction of a NN emulating the… · p. 22

Related papers

Loading 2001.11441v1…