PaperlayerView exact revision on arXiv

Source-linked AI summary

Weak Adversarial Networks for High-dimensional Partial Differential Equations

Yaohua Zang, Gang Bao, Xiaojing Ye, Haomin Zhou

arXiv:1907.08272v3math.NA

TL;DR

High-dimensional linear and nonlinear PDEs on arbitrary domains are difficult for mesh-based methods because of dimensionality and irregular geometry. WAN uses a weak formulation to turn solution finding into operator-norm minimization, parameterizing the solution and test function as primal and adversarial networks. Across reported experiments, it solves high-dimensional elliptic problems with relative errors below 1% in several Dirichlet cases and 2.03% and 1.31% in the tested Neumann cases, while showing stable progress but sensitivity to network size and sampling.

  • Problem

    Solving general high-dimensional PDEs on arbitrary domains remains a long-standing challenge, while mesh-based finite difference and finite element methods suffer from the curse of dimensionality.

  • Method

    WAN converts weak-solution finding into operator-norm minimization and parameterizes the weak solution and test function as primal and adversarial neural networks.

  • Results

    WAN reports relative errors of 0.44%–0.69% for tested d = 5, 10, 15, 20, 25 nonlinear Dirichlet cases, and 2.03% and 1.31% for d = 5 and 10 Neumann cases.

  • Takeaways & Limitations

    The method is presented as fast, stable, mesh-free, and applicable to high-dimensional linear and nonlinear PDEs on arbitrarily shaped domains.

  • Takeaways & Limitations

    Accuracy depends on sufficient network capacity, with the five-neuron case failing to show the same relative-error decay seen in other tested architectures.

Abstract

from arXiv · show

Solving general high-dimensional partial differential equations (PDE) is a long-standing challenge in numerical mathematics. In this paper, we propose a novel approach to solve high-dimensional linear and nonlinear PDEs defined on arbitrary domains by leveraging their weak formulations. We convert the problem of finding the weak solution of PDEs into an operator norm minimization problem induced from the weak formulation. The weak solution and the test function in the weak formulation are then parameterized as the primal and adversarial networks respectively, which are alternately updated to approximate the optimal network parameter setting. Our approach, termed as the weak adversarial network (WAN), is fast, stable, and completely mesh-free, which is particularly suitable for high-dimensional PDEs defined on irregular domains where the classical numerical methods based on finite differences and finite elements suffer the issues of slow computation, instability and the curse of dimensionality. We apply our method to a variety of test problems with high-dimensional PDEs to demonstrate its promising performance.

1 Introduction

High-dimensional PDEs remain difficult for mesh-based numerical methods, motivating WAN as a weak-formulation approach for linear and nonlinear equations on arbitrary domains.

  • Scope: The method is designed for general high-dimensional PDEs, including linear and nonlinear equations, on arbitrarily shaped domains.The introduction also states that it is fast, stable, and completely mesh-free.
  • Motivation: Finite difference and finite element methods discretize PDEs with mesh grids or triangulations but suffer from the curse of dimensionality.They construct basis functions and solve the resulting discrete coefficient system.
  • Proposed approach: WAN parameterizes the weak solution and test function as primal and adversarial neural networks trained using sampled interior and boundary collocation points.The method uses network evaluations and gradients without spatial discretization.
  • Paper organization: The paper extends the approach from stationary PDEs to time-dependent equations and presents numerical results for efficient and accurate high-dimensional solutions.The stated workflow includes weak formulations, a saddle-point reformulation, training details, and numerical experiments.

2 Related Work

Prior machine-learning approaches either assist classical solvers or directly approximate PDE solutions, while variational and weak-formulation methods provide alternative formulations.

  • Machine-learning approaches: Deep-learning PDE methods are broadly divided into approaches assisting classical numerical methods and approaches directly approximating PDE solutions.The direct-approximation category may be advantageous for high-dimensional problems.
  • Direct approximation: Direct neural PDE solvers have used boundary-satisfying decompositions, residual minimization, single-network approximation, PINNs, and stochastic representations.The cited methods differ in how they encode boundary conditions and train against PDE information.
  • Variational approaches: Variational approaches parameterize functions with neural networks and optimize variational or energy formulations of the corresponding PDEs.The deep Ritz method targets PDEs that admit equivalent energy minimization problems.

3 Proposed Method

The proposed method reformulates PDE solution as operator-norm minimization induced by a weak formulation, then learns the solution and test function with adversarial neural networks.

  • 3 Proposed Method: For boundary value problems, the weak solution is posed as an operator norm minimization problem derived from the PDE’s weak formulation.The method is later extended to initial-boundary value problems involving time.
  • 3 Proposed Method: The weak solution and test function are parameterized as deep neural networks whose parameters are learned through adversarial training.The training is governed by the weak formulation rather than a strong-form residual alone.
  • 3 Proposed Method: The approach provides implementation details for boundary value problems and extends the framework to time-dependent initial-boundary value problems.

3.1 PDE and weak formulation

The paper uses weak solutions because classical solutions may lack sufficient regularity or fail to exist, defining solutions through weak formulations against admissible test functions.

  • 3.1 PDE and weak formulation: A classical C2(Ω) solution may require sufficient regularity and may not exist, so the paper considers a weak formulation instead.The formulation is obtained by multiplying the PDE by a test function.
  • 3.1 PDE and weak formulation: The test functions lie in H1_0(Ω), the Sobolev space of functions with square-integrable values and weak partial derivatives and vanishing boundary trace.This boundary condition removes the boundary terms after integration by parts.
  • 3.1 PDE and weak formulation: A function satisfying the weak formulation for every admissible test function is called a weak solution, which may exist even when a classical solution does not.This lets the method address boundary value problems without requiring classical solvability.

3.2 Induced operator norm minimization

The paper recasts finding a weak PDE solution as minimizing an operator norm induced by the weak formulation, subject to boundary conditions. A theorem establishes equivalence between this minimization problem and the original weak-solution problem.

  • 3.2 Induced operator norm minimization: The residual A[u] is treated as a linear functional on test functions, whose induced operator norm measures the largest normalized weak-form residual.The norm is defined by maximizing the weak-form pairing over nonzero test functions, normalized by their L2 norm.
  • 3.2 Induced operator norm minimization: A weak solution satisfies the boundary condition and has zero induced operator norm.The norm is nonnegative, so zero residual norm characterizes vanishing weak-form residuals.
  • 3.2 Induced operator norm minimization: Theorem 1 states that a boundary-satisfying function is a weak solution if and only if it solves the associated minimization problems and has zero operator norm.The theorem provides the formal equivalence underlying the proposed optimization approach.
  • 3.2 Induced operator norm minimization: The minimax problem is well-defined because the relevant maximization and minimization domains are closed and the weak-form functional is continuous.The proof establishes existence of the maximum over normalized test functions and the minimum over boundary-constrained functions.
  • 3.2 Induced operator norm minimization: The operator-norm formulation lets the method seek the weak solution by minimizing the worst normalized weak-form residual.This follows directly from the theorem’s equivalence between the weak solution and the minimax formulation.

3.3 Weak adversarial network for solving PDE

WAN parameterizes the weak solution and test function with separate neural networks and trains them adversarially using sampled interior and boundary objectives. The resulting saddle-point objective combines weak-form residuals with boundary-condition error.

  • 3.3 Weak adversarial network for solving PDE: The solution network uθ minimizes the operator norm while the adversarial test-function network ϕη maximizes the normalized weak-form pairing.The two networks play the primal and adversarial roles induced by the minimax formulation.
  • 3.3 Weak adversarial network for solving PDE: The interior objective is formed from the weak-form pairing evaluated on collocation points, with a logarithmic reformulation used for convenience.The logarithm preserves optimization order because it is strictly increasing.
  • 3.3 Weak adversarial network for solving PDE: Boundary constraints are enforced with sampled boundary losses for either Dirichlet or Neumann conditions.Dirichlet conditions use solution-value error, while Neumann conditions use the normal derivative.
  • 3.3 Weak adversarial network for solving PDE: WAN seeks a saddle point of a weighted sum of the interior and boundary objectives.The balancing parameter α controls the relative contribution of the boundary loss.
  • 3.3 Weak adversarial network for solving PDE: Although any positive α is theoretically compatible with a zero-loss weak solution, its value affects training performance in practice.The paper defers examples of this effect to its numerical experiments.

3.4 Training algorithm for the weak adversarial network

WAN is trained by estimating objective gradients from randomly sampled interior collocation points and alternating descent on the solution network with ascent on the adversarial network.

  • 3.4 Training algorithm for the weak adversarial network: The solution and adversarial parameters are optimized by alternating gradient descent and gradient ascent, respectively.Both updates use gradients of the combined objective L(θ, η).
  • 3.4 Training algorithm for the weak adversarial network: The weak-form integrand I(x; θ, η) is introduced to derive gradients of the interior loss with respect to both network parameter sets.The construction applies to the elliptic PDE example through the weak formulation.
  • 3.4 Training algorithm for the weak adversarial network: Interior collocation points are sampled uniformly from the domain to approximate the gradient of the interior objective.The resulting gradient estimate averages contributions from Nr sampled points.
  • 3.4 Training algorithm for the weak adversarial network: The same sampling-based strategy approximates the remaining gradients, yielding the WAN algorithm.The paper states that gradients for the adversarial interior loss and boundary loss are handled similarly.

3.5 Efficiency and stability improvements of WAN

The paper introduces practical modifications intended to improve WAN’s efficiency and stability, including boundary-constrained adversarial functions and alternative loss forms. Algorithm 1 alternates updates using interior and boundary collocation samples.

  • 3.5 Efficiency and stability improvements of WAN: The adversarial function can be factorized as ϕη = w·vη, where a signed-distance function w vanishes on the boundary.This construction enforces the adversarial network’s boundary constraint throughout training.
  • 3.5 Efficiency and stability improvements of WAN: Precomputing and fixing w means training updates only the solution network uθ and the auxiliary network vη.The boundary condition for ϕη then requires no separate training constraint.
  • 3.5 Efficiency and stability improvements of WAN: Training uθ with the squared normalized weak-form pairing instead of the logarithmic term appears empirically to improve efficiency.The modification targets the solution-network objective.
  • 3.5 Efficiency and stability improvements of WAN: Algorithm 1 samples interior and boundary collocation points, then alternates updates of the weak-solution and test-function network parameters.The algorithm uses Ku solution-network updates and Kϕ adversarial-network updates per iteration.

3.6 Weak adversarial network for PDEs involving time

WAN extends to time-dependent PDEs through either sequential time discretization or joint treatment of space and time, using neural networks for weak solutions and test functions.

  • Approaches for time-dependent PDEs: WAN offers semi-discretization and joint space-time approaches for solving time-dependent PDEs.Semi-discretization solves a time-independent elliptical-type PDE at each time point, while the joint approach avoids time discretization.
  • Semi-discretization in time: The semi-discretization approach partitions [0, T] into N uniform segments and applies Crank-Nicolson time stepping.The resulting elliptical-type equation is solved at each time point using the static WAN procedure.
  • Semi-discretization in time: Starting from the initial condition, WAN sequentially computes u(x, tn) as a parameterized network for each later time point.Algorithm 1 serves as the subroutine for solving the elliptical equation at each step.
  • Joint space-time formulation: In the joint approach, the weak solution and test function are parameterized as deep neural networks over Ω×[0, T].The objective includes interior, initial-condition, and boundary-condition losses, with balancing parameters α and γ.
  • Joint space-time formulation: Algorithm 3 alternates updates of the weak-solution and test-function network parameters using region, boundary, and initial collocation points.The test function can be factorized so it vanishes on the boundary throughout training.

4 Numerical Experiments

The numerical experiments evaluate WAN on high-dimensional linear and nonlinear PDEs over regular and irregular domains using relative L2 error against exact solutions.

  • Experimental setup: The experiments cover high-dimensional linear and nonlinear PDEs defined on both regular and irregular domains.The proposed algorithms evaluated are Algorithms 1–3.
  • Accuracy measure: Accuracy is measured by the L2 relative error ∥uθ − u∗∥2/∥u∗∥2, using the exact solution as reference.The norm is defined through integration over the domain.
  • Evaluation procedure: For high-dimensional error computation, the evaluation points combine a 100×100 mesh over (x1, x2) with randomly sampled remaining coordinates.These points are fixed in advance and shared across comparison algorithms, separately from training samples.

4.2 Experimental results

WAN is evaluated on smooth and singular elliptic problems, nonlinear PDEs, Neumann boundaries, irregular domains, and time-dependent equations. Across these tests, it recovers accurate solutions in dimensions up to 25 and shows approximately linear runtime growth with dimension.

  • Singular problem: WAN captures the singularities in the first Poisson problem and faithfully recovers its unique weak solution, unlike strong-form methods tested there.The problem admits only a weak solution; PINN and another strong-form method show similar results.
  • Smooth problem: All methods achieve relative errors below 1% on the smooth five-dimensional Poisson problem, with PINN most accurate, WAN closely following, and DRM converging fastest.PINN requires second-order derivatives, making its computation more expensive than methods using first-order derivatives.
  • Nonlinear elliptic PDEs: 0.44%, 0.62%, 0.52%, 0.66%, and 0.69% are WAN’s relative errors after 20,000 iterations for nonlinear elliptic problems with d = 5, 10, 15, 20, and 25.The results support accurate solution of the tested high-dimensional nonlinear PDEs.
  • Neumann boundary condition: 2.03% and 1.31% are WAN’s relative errors after 20,000 iterations for Neumann-boundary problems with d = 5 and 10.The experiment reports effective performance for BVPs with Neumann boundary conditions.
  • Time-dependent PDEs: WAN reaches relative errors of 2.8% for the time-dependent problem and 0.78% and 0.66% for the nonlinear diffusion-reaction problem with d = 5 and 10.The first result uses Algorithm 2 at T = 1 after 10,000 iterations; the second uses Algorithm 3 after 20,000 iterations.
  • Stability and scalability: Runtime to reach 1% relative error increases approximately linearly with dimension d for the tested nonlinear elliptic problem.The scalability test covers d = 5, 10, 15, 20, and 25.

5 Concluding Remarks

WAN is presented as a weak-form saddle-point method for high-dimensional linear and nonlinear PDEs on arbitrary domains, with experiments examining sampling, architecture, and dimensional scalability.

  • WAN parameterizes the weak solution and test function as primal and adversarial networks, alternately updating their parameters during training.
  • Greater network depth or width usually accelerates relative-error decay, but can increase training time and cause overfitting; five neurons were insufficient in one test.
  • The method requires evaluations at randomly sampled interior and boundary collocation points, supporting mesh-free training on irregular domains.
Loading 1907.08272v3…