Source-linked AI summary
Deep least-squares methods: an unsupervised learning-based numerical method for solving elliptic PDEs
Zhiqiang Cai, Jingshuang Chen, Min Liu, Xinyu Liu
TL;DR
This paper studies unsupervised neural-network methods for solving elliptic PDEs, focusing on a first-order system least-squares functional. In the reported tests, performance depends on problem smoothness and activation choice, and the direct least-squares method has a stated regularity restriction.
Problem
Deep-network PDE methods use least-squares functionals, but the direct least-squares formulation is limited to sufficiently smooth solutions.
Source: §1 · p. 1, §2.2 · p. 4Method
The method approximates PDE solutions with a deep neural network and trains it using the FOSLS functional for a first-order system of scalar elliptic PDEs.
Source: §1 · p. 2Results
For the smooth Poisson solution, all three methods approximate accurately and deep LS performs slightly better, while on the interface problem deep FOSLS is significantly better than deep Ritz and deep LS fails to approximate well.
Source: §4.1 · p. 9, §4.3 · p. 12Takeaways & Limitations
The results indicate that activation choice is problem dependent, and the relative FOSLS functional value may guide that choice when the true solution is unknown.
Source: §4.3 · p. 12Direct deep LS requires a solution in at least H2(Ω), and estimating the neural-network approximation error remains an open problem.
Source: §2.2 · p. 4, §3.2 · p. 7
Abstract
from arXiv · showhide
This paper studies an unsupervised deep learning-based numerical approach for solving partial differential equations (PDEs). The approach makes use of the deep neural network to approximate solutions of PDEs through the compositional construction and employs least-squares functionals as loss functions to determine parameters of the deep neural network. There are various least-squares functionals for a partial differential equation. This paper focuses on the so-called first-order system least-squares (FOSLS) functional studied in [3], which is based on a first-order system of scalar second-order elliptic PDEs. Numerical results for second-order elliptic PDEs in one dimension are presented.
1. Introduction.
Neural networks approximate functions compositionally, prompting their application to PDEs, while existing least-squares and energy-based formulations have important constraints. This paper studies an unsupervised deep-network approach using the FOSLS functional, whose least-squares properties include stabilization and a posteriori error estimation.
1. Introduction.
- Neural-network PDE solving through least-squares minimization follows earlier work dating to 1994, with subsequent approaches using mesh samples and deeper networks.The paper situates its approach within this developing line of work.Source: §1 · p. 1
- Earlier PDE least-squares losses based on the original equation lead to a fourth-order minimization problem for second-order PDEs, and their interior and boundary integrals are unbalanced.Prior work explored both shallow and deeper networks, with random domain sampling noted as beneficial for higher-dimensional problems.Source: §1 · p. 1
- The deep Ritz method uses an underlying PDE energy functional and therefore applies to problems with a minimization principle.The cited example is a Poisson problem with Dirichlet boundary conditions.Source: §1 · pp. 1–2
- The paper applies deep neural networks to PDE solution by approximating solutions compositionally and optimizing parameters with least-squares functionals, focusing on FOSLS for a first-order system of scalar elliptic PDEs.The paper also reports one-dimensional numerical results for three test problems and a quadrature comparison.Source: §1 · p. 2
- Least-squares methods can naturally symmetrize and stabilize problems without minimization principles, while their functional values provide a posteriori error estimates for numerical-process control.The passage cites applications in solid and fluid dynamics, radiation transport, and magnetohydrodynamics.Source: §1 · p. 2
2. Problem Formulation.
The section formulates a scalar second-order elliptic problem with mixed boundary conditions and introduces first-order-system and direct least-squares formulations. It also discusses boundary-condition scaling and the regularity restriction on the direct least-squares method.
2. Problem Formulation.
- The problem is a scalar second-order elliptic PDE on a bounded Lipschitz domain, with Dirichlet and Neumann data on boundary portions and a uniformly positive definite coefficient matrix.The formulation permits a first-order differential operator X and assumes the stated data and coefficients lie in specified Sobolev and L2 spaces.Source: §2 · p. 2
2.1. Least-Squares Formulations.
- The FOSLS functional is based on that first-order system, and its homogeneous form is coercive and bounded, yielding a well-posed minimization problem with a unique solution.The cited result is established in.Source: §2.1 · pp. 2–3
- Introducing flux σ = −A∇u rewrites the second-order problem as a first-order system with corresponding Dirichlet and Neumann boundary conditions.The admissible solution and flux spaces encode the respective boundary data.Source: §2.1 · p. 3
2.2. Treatment of Boundary Conditions.
- The deep Ritz energy functional adds a boundary penalty to the interior energy, with a parameter β, to treat boundary conditions in the loss.The text notes that boundary and interior norms are on different scales, motivating a modified energy functional.Source: §2.2 · pp. 3–4
- For FOSLS, both Dirichlet and Neumann conditions are essential and are added to its functional with appropriate scaling; the resulting minimization is well-posed without strongly enforcing them.The passage attributes this well-posedness to coercivity and boundedness of the homogeneous functional.Source: §2.2 · p. 4
- The direct least-squares formulation balances the equation and boundary residuals using boundary norms and parameters for Dirichlet and Neumann conditions.Its minimization is posed over H2(Ω), and well-posedness is stated under H2 regularity of the solution.Source: §2.2 · p. 4
- The direct least-squares loss applies only when the solution is sufficiently smooth, at least in H2(Ω), and requires a non-piecewise-linear network activation.This restriction is stated for the formulation in (2.14)–(2.15).Source: §2.2 · p. 4
3. The Deep FOSLS.
The deep FOSLS method represents the solution pair with a deep neural network and minimizes a discretized FOSLS functional. Its numerical error includes network approximation and quadrature components, with estimating the former still an open problem.
3.1. Deep Neural Network Structure.
- A network is built by composing layers; fully connected layers apply a linear transformation followed by an activation function.The number of neurons specifies layer width, while weights and biases are trainable parameters.Source: §3.1 · p. 5
- Smooth sigmoid activation is essential for the deep LS method because networks with continuous piecewise linear activation functions are only in H1(Ω).Leaky ReLU is easier to compute than the nonlinear sigmoid function.Source: §3.1 · p. 6
3.2. Deep FOSLS.
- The method uses a deep neural network to approximate the FOSLS solution pair and numerically minimizes the functional using composite midpoint quadrature.The discrete formulation approximates the FOSLS Sobolev norms by weighted L2 norms with local weights h_E^-1/2.Source: §3.2 · pp. 6–7
- The total error separates deep-network approximation error from quadrature error, and estimating the former remains an open problem.Numerical quadrature error can be brought to a desired accuracy using uniform or adaptive partitions.Source: §3.2 · pp. 7–8
4. Numerical Experiments.
The numerical experiments apply deep FOSLS and related methods to one-dimensional elliptic test problems, comparing functionals, activations, and quadrature strategies. Results favor FOSLS for the interface problem and local quadrature refinement over global refinement.
4. Numerical Experiments.
- The experiments use separate neural-network branches for u and σ, and report one-dimensional Poisson, reaction-diffusion, and interface tests.The FOSLS, energy, and least-squares functionals are numerically evaluated on domain partitions.Source: §4 · pp. 8–9
4.1. Poisson Equation.
- With Leaky ReLU, 800 quadrature points are sufficient to accurately evaluate the FOSLS functional in the reported Poisson experiment.The experiment used a fixed learning rate of 0.0005 and 10,000 iterations.Source: §4.1 · p. 9, Table 4.1 · p. 10
- For the smooth Poisson solution, all three functionals approximate accurately, deep LS performs slightly better, and sigmoid is more accurate than Leaky ReLU.The authors suggest the activation difference may relate to the exact solution's exponential feature.Source: §4.1 · p. 9, Table 4.2 · p. 10
4.2. Singularly Perturbed Reaction-Diffusion Equation.
- For ε = 0.01 and α = 1, deep FOSLS is more accurate than deep Ritz with either tested activation, and sigmoid is more accurate than Leaky ReLU.All three methods accurately approximate the solution with interior layers after 20,000 iterations.Source: §4.2 · p. 10, Table 4.3 · p. 11
4.3. Interface Problem.
- For k = 10 and α = 1, deep FOSLS is significantly better than deep Ritz on the interface problem, while deep LS fails to approximate the solution well.Leaky ReLU performs better than sigmoid for this piecewise-polynomial solution; the authors suggest using relative FOSLS functional values to guide activation choice when the true solution is unknown.Source: §4.3 · p. 12, Table 4.4 · p. 12
4.4. Adaptive Numerical Quadrature.
- Local quadrature refinement yields a better relative FOSLS functional value than global refinement or a uniform partition in the reported comparison.All methods used 10,000 iterations; local refinement was repeated every 2,000 iterations, while global refinement occurred once after 5,000.Source: §4.4 · p. 13, Figure 4.4 · p. 13
5. Discussion and Conclusion.
The paper proposes deep FOSLS and balanced boundary-condition treatments for deep Ritz and deep LS methods. It also discusses how least-squares feedback and DNN representations relate to adaptivity and sharp-layer approximations.
5. Discussion and Conclusion
- The proposed deep FOSLS method uses DNNs to approximate PDE solutions, while the deep Ritz and deep LS methods are modified to treat boundary conditions in a balanced way.The authors state that deep Ritz applies to problems with an underlying minimization principle, while deep LS applies to smooth problems; they describe FOSLS as applicable to a much larger class.Source: §5 · pp. 13–14
- Values of the deep LS and FOSLS functionals provide feedback for controlling DNN structure and quadrature-point number and location; adaptive control of functional evaluation and then network structure remains future work.The paper also points to inequalities for choosing quadrature points with a fixed DNN and for guiding termination of the iterative process.Source: §5 · p. 14
- Deep neural network approximations of a sharp interior layer exhibit no overshooting or oscillation, and the authors attribute this performance to their meshless, “pointless” function class.The authors compare these defects with those of traditional mesh-based methods without strategies such as limiters.Source: §5 · p. 14