PaperlayerView exact revision on arXiv

Source-linked AI summary

An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications

Esteban Samaniego, Cosmin Anitescu, Somdatta Goswami, Vien Minh Nguyen-Thanh, Hongwei Guo, Khader Hamdia, Timon Rabczuk, Xiaoying Zhuang

arXiv:1908.10407v2stat.MLcs.LGmath.AP

TL;DR

The paper addresses approximating PDE solutions for computational mechanics when analytical methods are insufficient. It proposes using deep neural networks with mechanical energy as the loss function and demonstrates proof-of-concept solutions for several boundary value problems, while noting non-convex optimization challenges.

  • Problem

    The paper explores whether deep neural networks can approximate PDE solutions for computational mechanics, where analytical methods are generally insufficient and discretization is required.

  • Method

    The Deep Energy Method uses a mechanical system’s energy as the neural-network loss function and defines the approximation space through the network architecture, optimized with gradient-based methods.

  • Results

    The approach provides a proof of concept for using deep neural networks to approximate boundary-value-problem solutions in computational mechanics; one example reports relative prediction errors of 0.5% in displacement and 3.7% in energy norms.

  • Takeaways & Limitations

    Deep neural networks can serve as approximation spaces for several computational-mechanics boundary value problems using an energy-based formulation.

  • Takeaways & Limitations

    The resulting discrete optimization problems are non-convex, and the expressive neural-network approximation spaces can be difficult to analyze.

Abstract

from arXiv · show

Partial Differential Equations (PDE) are fundamental to model different phenomena in science and engineering mathematically. Solving them is a crucial step towards a precise knowledge of the behaviour of natural and engineered systems. In general, in order to solve PDEs that represent real systems to an acceptable degree, analytical methods are usually not enough. One has to resort to discretization methods. For engineering problems, probably the best known option is the finite element method (FEM). However, powerful alternatives such as mesh-free methods and Isogeometric Analysis (IGA) are also available. The fundamental idea is to approximate the solution of the PDE by means of functions specifically built to have some desirable properties. In this contribution, we explore Deep Neural Networks (DNNs) as an option for approximation. They have shown impressive results in areas such as visual recognition. DNNs are regarded here as function approximation machines. There is great flexibility to define their structure and important advances in the architecture and the efficiency of the algorithms to implement them make DNNs a very interesting alternative to approximate the solution of a PDE. We concentrate in applications that have an interest for Computational Mechanics. Most contributions that have decided to explore this possibility have adopted a collocation strategy. In this contribution, we concentrate in mechanical problems and analyze the energetic format of the PDE. The energy of a mechanical system seems to be the natural loss function for a machine learning method to approach a mechanical problem. As proofs of concept, we deal with several problems and explore the capabilities of the method for applications in engineering.

1. Introduction

Computational mechanics uses numerical approximations for PDE-based models of natural and engineered systems. This paper presents DNNs as function approximators and proposes using mechanical energy as a machine-learning loss for engineering-oriented PDE problems.

  • Realistic PDE models in computational mechanics generally require numerical methods to obtain approximate solutions in finite-dimensional spaces.
  • DNNs are treated as function approximation machines whose weights and biases are learned during training.
  • Earlier DNN-based PDE approaches generally use strong-form collocation and empirical losses, often outside direct engineering applications.
  • The paper proposes formulating mechanical PDEs variationally and using system energy as the machine-learning loss function.After finite-dimensional approximation, the variational problem becomes an optimization problem suited to machine-learning libraries.
  • Unlike data-driven surrogate models, the proposed approach uses machine learning to construct the approximation space for the PDE solution.
  • The paper develops the approach through generalized PDEs, DNNs, collocation and deep energy methods, implementation details, and computational-mechanics applications.

2. Mathematical modeling of continuous physical systems

The paper frames boundary value problems through generalized differential equations and their variational energy formulations. For mechanical systems, stationarity of the energy yields the weak form, while direct energy minimization provides the basis for the proposed discretization strategy.

  • A generalized PDE defines a boundary value problem through a differential operator acting on a function subject to Dirichlet and Neumann conditions.
  • For linear elasticity, displacement is the primal variable, with strain, stress, and momentum balance specifying the mechanical model.
  • The governing equation can be solved with a collocation-type method when the displacement field is sufficiently smooth.
  • The energy approach assumes that the variational energy has a unique minimizer corresponding to the PDE solution under Dirichlet constraints.
  • Setting the first variation of the energy to zero produces the weak form, which corresponds to virtual work in mechanics and underlies FEM discretization.
  • For linear elasticity, stored strain energy and boundary conditions define the solid's total energy.

3. Deep Neural Networks for PDE discretization

Deep neural networks map inputs to outputs through compositions of affine transformations and activation functions, with weights and biases defining the approximation. In this PDE setting, training data comes from evaluating governing equations and boundary information at selected points.

  • Artificial neural networks transform inputs into outputs through interconnected units called neurons, commonly represented as graph structures.
  • A DNN maps an input x to an output u_p(x) through compositions of affine mappings and element-wise activation functions.
  • DNN nonlinearity comes from activation functions, while layers organize the successive transformations in the network.
  • Weights and biases parameterize the network function, and the paper considers feed-forward fully connected architectures.
  • Once the architecture is fixed, training determines the weights and biases using PDE evaluations in the domain and selected boundary points.

4. Solution strategies

The paper contrasts strong-form collocation with the Deep Energy Method, which minimizes an energy-based loss for variational boundary-value problems. Training this DNN approximation is a non-convex optimization problem with practical boundary-condition and search-space challenges.

  • Collocation: Collocation evaluates the strong-form PDE at domain points and minimizes a loss that tends to enforce the governing equation there.Mean-square errors of the evaluated differential operator provide one example of the collocation loss.
  • Deep Energy Method: The Deep Energy Method uses the system energy as the DNN loss and approximates total energy with weighted energy densities at integration points.This formulation exploits the variational structure of suitable boundary-value problems.
  • Deep Energy Method: Dirichlet boundary conditions require a more involved treatment because DNN approximation functions do not satisfy the Kronecker delta property.Neumann conditions can in principle be imposed naturally in the variational formulation.
  • Optimization: After selecting the network architecture and empirical loss, training determines the weights and biases that define the approximation of the unknown field.The resulting objective is described as a non-convex finite-sum optimization problem.
  • Optimization: Gradient-descent variants are common, while L-BFGS can incorporate an approximation of second-order information through a quasi-Newton method.The paper notes stochastic selection, acceleration, and penalty terms as additional variants of gradient-based optimization.
  • Optimization: DNN function spaces may be highly expressive while their search-space topology creates optimization difficulties, although gradient-based methods can overcome them in some applications.The paper identifies this as an active research issue rather than a resolved property of the approach.

5. Implementation

The implementation builds a DNN solver from sampled domain points, initialized network parameters, boundary-condition-enforcing trial outputs, automatic differentiation, and energy-based optimization. TensorFlow and related open-source tools support this workflow from geometry generation through field prediction.

  • Workflow: The DEM implementation uses open-source machine-learning libraries to evaluate the integral defining the governing energy.The paper specifically identifies TensorFlow and PyTorch as implementation platforms.
  • Initialization: Training begins by generating uniformly spaced domain points, separating training points Xf from prediction points xpred, and initializing weights with Xavier initialization.The weights are initialized randomly from a Gaussian distribution.
  • Initialization: The network architecture stores layer weights and biases, with zero-initialized biases paired with Xavier-initialized weight matrices.The implementation appends these parameters for each adjacent pair of layers.
  • Boundary conditions: The displacement outputs are modified so that the Dirichlet boundary conditions are satisfied exactly.The implementation constructs trial fields from the neural-network outputs rather than using the raw outputs directly.
  • Network evaluation: Inputs are normalized using lower and upper coordinate bounds, then passed through hidden layers with the selected activation function before producing the network output.The example uses ReLU-based hidden-layer transformations and a final affine output.
  • Energy evaluation: Automatic differentiation computes displacement gradients and the variational energy at each domain point.The implementation differentiates the displacement components with respect to the spatial coordinates before assembling the energy density.
  • Training and prediction: Optimization minimizes the mean energy-loss error with Adam followed by L-BFGS, after which the trained network predicts field variables at xpred points.The workflow therefore combines first-order and quasi-Newton optimization stages.

6. Applications

The applications use DEM across continuum-mechanics problems, beginning with a one-dimensional analytical comparison and extending to elasticity, coupled fields, and other engineering models. The paper also relates ReLU DNN approximations to finite-element spaces while identifying interpretive and optimization limits.

  • Applications: The application section evaluates DEM on a one-dimensional problem against an available analytical solution before addressing linear elasticity, hyperelasticity, and coupled problems.The coupled examples include phase-field fracture and piezoelectricity, among other problems.
  • DNN and FE relationship: ReLU DNNs can reproduce linear finite-element spaces, providing an approximation-theoretic link between neural-network and finite-element representations.The paper defines ReLU as ReLU(x) = max(0, x).
  • DNN and FE relationship: A one-dimensional piecewise-linear finite-element approximation is constructed from linear hat functions with compact support around the discretization nodes.The hat functions satisfy Ni(xi) = 1 at their associated nodes.
  • DNN and FE relationship: The finite-element approximation can be expressed as a one-hidden-layer ANN with the coordinate x as input, uh(x) as output, and ReLU as the activation function.The formulation groups the finite-element increments into network weights.
  • DNN and FE relationship: The ANN approximation is parameterized by weights and node positions, with the latter corresponding to hidden-layer biases; fixing node positions makes the weights correspond to nodal values.This establishes the one-dimensional correspondence between DNN parameters and finite-element quantities.
  • Limitations: When node positions are also optimized, the problem becomes an adaptive finite-element problem but remains nonlinear and probably non-convex, complicating machine-learning optimization.For fixed node positions, the problem is equivalent to a standard finite-element problem.
  • Limitations: For general situations, assigning physical meaning to DNN approximation parameters is difficult, and collocation points should not be confused with finite-element mesh nodes.Finite-element nodes relate directly to approximation-space parameters, whereas collocation points serve as training inputs.

6.2. Linear Elasticity problem

The deep energy method (DEM) approximates linear-elastic displacement fields by minimizing stored elastic strain energy while enforcing boundary conditions. Across two- and three-dimensional benchmarks, the method produces low reported errors against analytical solutions.

  • Method: DEM obtains the displacement field by minimizing the stored elastic strain energy, with homogeneous Neumann conditions satisfied automatically.The objective uses the mean error of the energy functional at integration points, simplifying the loss by avoiding a boundary-loss component for homogeneous Neumann conditions.
  • Pressurized thick-cylinder: The thick-cylinder benchmark uses a quarter annulus under internal pressure, with the displacement solution constrained to satisfy all boundary conditions.The reported setup uses E = 1 × 10^5, ν = 0.3, and a neural network with 3 hidden layers of 30 neurons each on an 80 × 80 interior grid.
  • Pressurized thick-cylinder: 0.5% displacement and 3.7% strain-energy relative prediction errors are reported for the pressurized thick-cylinder example.Errors are measured relative to the analytical solution using displacement and energy norms.
  • Plate with a circular hole: 1.8% displacement and 3.19% strain-energy prediction errors are reported for the plate with a circular hole.The benchmark models one quarter of a finite plate domain under plane stress and compares the neural-network solution with the analytical solution.
  • Hollow sphere under internal pressure: 1.05% displacement and 4.44% strain-energy prediction errors are reported for the hollow sphere under internal pressure.Only one-eighth of the symmetric sphere is analyzed, and the predicted displacement field and strain energy are evaluated against the analytical solution.
  • Cube with a spherical hole subject to uniform tension: A cube-with-hole example reports a 5.3% strain-energy prediction error using the DEM approach.The section states that the L2 error is computed for strain energy to assess solution accuracy.

6.3. Elastodynamics

The elastodynamics example tests a space-time neural network on one-dimensional wave propagation using collocation terms for the governing equation and initial and boundary conditions. The computed displacement and velocity closely match the exact solution under the reported discretization.

  • Method: The one-dimensional wave-propagation problem uses a neural-network approximation depending on trainable parameters optimized through a collocation-based loss.The collocation points cover the interior, initial conditions, and time-dependent boundary conditions.
  • Method: The loss incorporates the equilibrium equation together with the initial and boundary conditions for the space-time displacement approximation.The network uses space and time as inputs and displacement as its output.
  • Results: 1.422382 · 10^-3 displacement and 5.954601 · 10^-3 velocity relative L2 errors are obtained.The results use Nint = 1992, Ninit = 199, and Nbnd = 200 collocation points.

6.4. Hyperelasticity

The deep energy method (DEM) solves hyperelasticity by representing admissible displacement fields with neural networks and minimizing the system’s potential energy. In a twisted 3D cuboid, DEM agrees well with finite-element reference results for displacement, while stress differences remain due to post-processing gradients.

  • Formulation: DEM formulates hyperelastic equilibrium variationally and uses potential energy as the neural-network loss function.Admissible trial functions satisfy the Dirichlet conditions before differential operators are applied.
  • Problem setup: A 3D Neo-Hookean cuboid with dimensions L = 1.25, W = 1.0, and depth = 1.0 is clamped on one surface and twisted by 60°.The material is isotropic and homogeneous, and the right Cauchy-Green tensor defines the constitutive formulation.
  • Implementation: The DEM setup uses 64000 equidistant points and a 5-layer 3–30–30–30–3 network with tanh hidden-layer activation.The inputs are nodal coordinates and the outputs are three unconstrained displacement components.
  • Results: DEM displacements generally agree well with a fine-mesh finite-element reference solution along the AB line.Stress differences are attributed to obtaining gradients during post-processing from the predicted displacement field.
  • Results: DEM also predicts displacement magnitude and Von Mises stress on the CDEF plane, while L2 and H1 comparisons report good results in two training setups.The two setups use 8000 points for 50 steps and 64000 points for 25 steps; the reference H1 seminorm is 0.51407.

6.5. Phase field modeling of fracture

The phase-field fracture examples use DEM to approximate displacement and damage fields by minimizing elastic and fracture energies. The method is compared with collocation in one dimension and then applied to crack initialization and tensile crack growth.

  • Model: The phase-field model represents cracks with a scalar field φ, where φ = 1 denotes cracked material and φ = 0 denotes undamaged material.The approach simultaneously solves displacement and fracture fields by minimizing total potential energy.
  • DEM formulation: DEM minimizes the combined elastic and fracture energy E = Ee + Ec subject to prescribed displacement boundary conditions.The networks approximate both the vector-valued elastic field u and scalar-valued phase field φ.
  • One-dimensional comparison: 70.6% relative L2 error is reported for collocation because it cannot capture the derivative discontinuity at the crack location x = 25.The one-dimensional bar has length 50 and a crack at 25 units.
  • One-dimensional comparison: Using the same network architecture and 8000 interior points, DEM is evaluated against the analytical one-dimensional phase-field solution.The DEM loss is defined from the energy functional rather than the collocation residual.
  • Crack growth: DEM requires a smaller network and fewer iterations than DCM for accurate crack initialization, while DCM needs explicit Neumann-boundary treatment and about ten times more iterations.The paper then uses DEM to study tensile crack growth by simultaneously solving the elastic and phase fields.

6.6. Piezoelectricity

DEM is extended to piezoelectricity by minimizing the total electro-mechanical energy under mechanical and electrical boundary conditions. A cantilever example demonstrates predictions under pure mechanical and pure electrical loading.

  • Coupled formulation: Piezoelectricity couples mechanical strain and stress with electric field, polarization, and displacement variables in non-centrosymmetric crystals.The constitutive description uses piezoelectric, dielectric, and elastic tensors.
  • DEM formulation: DEM obtains coupled field variables by minimizing total energy E = Ei + Wext subject to mechanical, electrical, and traction boundary conditions.Ei is bulk internal energy and Wext is the work of external forces.
  • Cantilever setup: The cantilever beam is modeled under plane strain with an isotropic, linearly elastic material and a fully connected network with three hidden layers of 150 neurons.The first two layers use tanh activation and the last layer uses linear activation.
  • Loading cases: Under pure mechanical loading, DEM predicts the deformed shape and induced electric potential caused by electro-mechanical coupling.The bottom surface potential is fixed to zero in the closed-circuit configuration.
  • Loading cases: Under pure electrical loading with V = −100 MV on the top surface, DEM predicts the cantilever’s deformed configuration.The paper presents both loading cases as demonstrations of solving coupled systems without classical numerical methods.

6.7. Kirchhoff Plate bending

For Kirchhoff plate bending, the paper applies neural-network collocation and deep energy methods to a fourth-order PDE whose traditional mesh-based treatment requires C1 continuity. The numerical examples report agreement with analytical solutions across multiple plate geometries, while IGA-FEM remains faster and more accurate in one comparison.

  • Problem and formulation: Kirchhoff plate bending is governed by a fourth-order PDE, and its weak formulation involves second-order derivatives of transversal deflection.Traditional mesh-based methods face significant C1-continuity challenges.
  • Neural formulations: The plate deflection is approximated by a feedforward neural network, with losses constructed from governing-equation and boundary-condition terms.The strong-form and energy approaches use the same neural approximation but different physical residual or energy formulations.
  • Boundary treatment: The energy method requires only Dirichlet boundary conditions, whereas deep collocation must approximate additional Neumann quantities such as twist moment and shear force.This difference makes the energy formulation easier for the stated plate-bending treatment.
  • Numerical results: Increasing hidden-layer depth and neuron count generally makes predicted maximum deflection more accurate and brings relative error curves toward a plateau.The trend is reported for circular-plate examples and is illustrated through deflection contours and error plots.
  • Autoencoder studies: Adding an autoencoder improves the deep-energy configuration’s ability to capture physical patterns, while two encoding layers can be sufficient for annular-plate bending.Increasing encoding layers and neurons makes deflection converge toward the analytical solution, with computational cost also considered.
  • Comparison and scope: Deep-energy predictions agree well with analytical deflections even for plates with holes, but IGA-FEM is faster and more accurate in the reported comparison.After training, the autoencoder-based network can rapidly predict deflection and stress across the plate.

7. Concluding remarks

The paper presents deep neural networks as approximation spaces for boundary value problems, using physical energy to construct the loss function. It demonstrates the approach as a computational-mechanics proof of concept while highlighting non-convex optimization and analysis challenges.

  • Deep neural networks can approximate boundary value problem solutions using concepts and tools from deep learning.
  • The method uses the physical body's energy as the loss-function basis and the neural-network architecture to define the approximation space.
  • Training produces a finite-sum non-convex problem solved with gradient-based optimization and standard machine-learning libraries such as TensorFlow.
  • The approach supports straightforward, transparent implementation of different mathematical models by defining their corresponding energies.
  • Even linear problems lead to non-linear, non-convex discrete problems, while expressive neural-network approximation spaces remain difficult to analyze.
Loading 1908.10407v2…