PaperlayerView exact revision on arXiv

Source-linked AI summary

An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications

Esteban Samaniego, Cosmin Anitescu, Somdatta Goswami, Vien Minh Nguyen-Thanh, Hongwei Guo, Khader Hamdia, Timon Rabczuk, Xiaoying Zhuang

arXiv:1908.10407v2stat.MLcs.LGmath.AP

TL;DR

DNNs have been explored as PDE approximators, often using collocation, while mechanical problems also have an energy-based variational formulation. This paper uses system energy as the DNN loss and reports proof-of-concept solutions for computational-mechanics boundary-value problems.

  • Problem

    DNNs offer a way to approximate PDE solutions, but prior explorations often use collocation; this paper examines an energy-based approach for mechanical problems.

    Source: §1 · p. 2, §4.1 · p. 5

  • Method

    The Deep Energy Method uses a mechanical system’s energy as the DNN loss and defines the approximation space through the network architecture.

    Source: §4.2 · p. 5, §7 · p. 48

  • Results

    Several computational-mechanics boundary-value problems are solved with DNNs, providing a proof of concept for DNN-based BVP approximation.

    Source: §7 · p. 48

  • Takeaways & Limitations

    The examples support considering DNNs as approximation spaces for computational-mechanics boundary-value problems.

    Source: §7 · p. 48

    The optimization problems are non-convex, and even linear problems lead to nonlinear, non-convex discrete problems.

    Source: §7 · p. 49

Abstract

from arXiv · show

Partial Differential Equations (PDE) are fundamental to model different phenomena in science and engineering mathematically. Solving them is a crucial step towards a precise knowledge of the behaviour of natural and engineered systems. In general, in order to solve PDEs that represent real systems to an acceptable degree, analytical methods are usually not enough. One has to resort to discretization methods. For engineering problems, probably the best known option is the finite element method (FEM). However, powerful alternatives such as mesh-free methods and Isogeometric Analysis (IGA) are also available. The fundamental idea is to approximate the solution of the PDE by means of functions specifically built to have some desirable properties. In this contribution, we explore Deep Neural Networks (DNNs) as an option for approximation. They have shown impressive results in areas such as visual recognition. DNNs are regarded here as function approximation machines. There is great flexibility to define their structure and important advances in the architecture and the efficiency of the algorithms to implement them make DNNs a very interesting alternative to approximate the solution of a PDE. We concentrate in applications that have an interest for Computational Mechanics. Most contributions that have decided to explore this possibility have adopted a collocation strategy. In this contribution, we concentrate in mechanical problems and analyze the energetic format of the PDE. The energy of a mechanical system seems to be the natural loss function for a machine learning method to approach a mechanical problem. As proofs of concept, we deal with several problems and explore the capabilities of the method for applications in engineering.

1. Introduction

The paper explores DNNs as function-approximation machines for PDE solutions in computational mechanics, using variational energy as a machine-learning loss and considering engineering applications.

1. Introduction

  • Engineering PDEs typically require numerical approximation, conventionally via FEM, mesh-free methods, or isogeometric analysis; the paper treats DNNs as trainable function approximators for their solutions.The approximation depends on learned weights and biases.Source: §1 · p. 2
  • TensorFlow and PyTorch provide implementation building blocks, while the paper explores DNN-based solvers through representative computational-mechanics applications.The authors say these platforms support readable implementations and describe code snippets in the implementation discussion.Source: §1 · pp. 2–3
  • Some DNN-based PDE work has generally addressed strong-form equations through collocation [9], minimizing empirical loss to fulfil governing equations; problems addressed have generally not been directly related to engineering applications.Source: §1 · p. 2
  • The authors propose variational energy as a natural loss for training DNN-based PDE approximations, turning the finite-dimensional variational problem into an optimization task.They also note that machine-learning libraries are oriented toward optimization and provide readable syntax that eases implementation.Source: §1 · p. 2
  • Unlike data-driven surrogate approaches using labeled simulation or experimental data, including replacement of constitutive models, this work uses machine learning to build the approximation space.The cited approaches use machine learning to build surrogate models.Source: §1 · pp. 2–3

2. Mathematical modeling of continuous physical systems

The section aims to enrich deep learning for computational mechanics with mathematical physics, introducing generalized PDEs and an energy approach.

2.1. Partial Differential Equations

  • A generalized strong-form BVP applies a differential operator to a function subject to Dirichlet and Neumann conditions, and sufficiently smooth solutions can be treated by collocation.For linear elasticity, displacement is the primal variable, with strain and stress used in the momentum balance equation; the example neglects inertia and body forces.Source: §2.1 · pp. 3–4

2.2. Energy Approach

  • The method minimizes variational energy directly, avoiding explicit derivation of the weak form; for mechanical problems, that weak form corresponds to virtual work and underlies FEM.The formulation assumes the energy problem has a unique solution identical to the PDE solution; the PDE is then the energy problem’s Euler–Lagrange equation.Source: §2.2 · p. 4

3. Deep Neural Networks for PDE discretization

The section presents neural networks as input-to-output functions and describes deep networks as layered functions parameterized by weights and biases. For PDE problems, the network parameters are trained using evaluations of the PDE in the domain and at boundary points.

3. Deep Neural Networks for PDE discretization

  • Artificial Neural Networks transform inputs into outputs through neurons whose states depend on their inputs and functional relations.They are usually represented as graphs, with neurons as nodes.Source: §3 · p. 4
  • Deep Neural Networks compose affine mappings across layers, with activation functions providing their nonlinearity.The activation function is applied element-wise.Source: §3 · p. 5
  • The networks considered are feed-forward and fully connected, with weights and biases parameterizing the input-output function.Convolutional neural networks are noted as another possible type.Source: §3 · p. 5
  • Training determines the network parameters after its layers, neurons per layer, and activation functions are fixed.Training data are generated by evaluating the PDE at points in the domain and at selected boundary points.Source: §3 · p. 5

4. Solution strategies

The section contrasts strong-form collocation with an energy-based loss for mechanical boundary-value problems, then describes the non-convex optimization used to train the DNN.

4.1. Collocation Methods

  • Collocation enforces the strong-form BVP by minimizing residuals N[u_p](x_i) at selected domain points, for example with a mean-square-error loss.The residual is evaluated at each point in a chosen set within the domain.Source: §4.1 · p. 5

4.2. Deep Energy Method

  • Following, the Deep Energy Method (DEM) uses system energy as the DNN loss, approximating total energy by a weighted sum of energy density at integration points.The loss uses displacement approximations at integration points and their corresponding weights.Source: §4.2 · pp. 5–6
  • Neumann conditions can be imposed naturally in the variational formulation, but Dirichlet conditions require a more involved treatment because DNN approximations lack the Kronecker delta property.The paper defers the treatment of these boundary-condition issues to Section 6.Source: §4.2 · p. 6

4.2.1. Optimization in machine learning

  • Training requires solving a non-convex finite-sum optimization problem over network weights and biases, commonly with gradient descent or variants such as SGD and L-BFGS.Variants include stochastic point selection, acceleration, parameter-penalty terms, and quasi-Newton approximations to second-order information.Source: §4.2.1 · p. 6
  • Although DNN function spaces can be expressive, their topology may hinder optimization, while gradient-descent-based methods appear to overcome such difficulties in some applications.The passage describes this as an active research issue.Source: §4.2.1 · p. 6

5. Implementation

The implementation uses machine-learning libraries to obtain PDE-governed field variables through an energy-based workflow, from network construction and boundary enforcement to optimization and prediction.

Implementation

  • The approach obtains field variables by computing the governing partial differential equation’s integral, using machine-learning libraries such as TensorFlow or PyTorch.The integral is identified as Eq. (10).Source: §5 · p. 7
  • The workflow generates uniformly spaced domain points for training and prediction, then initializes network weights randomly with Xavier initialization.The points are denoted Xf and xpred.Source: §5 · p. 7
  • The network outputs are transformed to satisfy Dirichlet boundary conditions exactly, using u = (1-x)*x*uNN and v = y*(y-1)*vNN.Here uNN and vNN are the network’s two outputs.Source: §5 · p. 7
  • The example neural network rescales spatial inputs using their lower and upper bounds and applies squared ReLU activations in hidden layers.The output layer is affine, with no activation shown.Source: §5 · p. 8
  • Automatic differentiation computes displacement gradients and variational energy at domain points, after which mean energy loss is minimized with Adam followed by L-BFGS.The optimized network then predicts field variables at xpred points.Source: §5 · p. 8

6. Applications · 6.1. DNN with ReLU activation functions and FE in 1D

The applications section explores DEM for continuum-mechanics PDEs, beginning with a one-dimensional example that relates ReLU neural networks to finite-element approximations.

6. Applications

  • The section’s applications progress from a 1D problem compared with an analytical solution to linear elasticity, hyperelasticity, and coupled problems involving fracture, piezoelectricity, and a Kirchhoff plate.Source: §6 · p. 8

6.1. DNN with ReLU activation functions and FE in 1D

  • ReLU networks can reproduce linear finite-element spaces, and the 1D example expresses a piecewise-linear FE approximation as a one-hidden-layer network.For fixed node positions and prescribed initial value, the weights correspond to the approximation’s values at the nodes.Source: §6.1 · pp. 8–9
  • Approximation-theory results extend the relationship to more general FE spaces using Sobolev norms; the one-dimensional illustration follows [20].Source: §6.1 · p. 9
  • With fixed node positions, energy minimization is equivalent to a standard FE problem; optimizing node positions instead gives an adaptive FE problem that may be nonlinear and non-convex.The authors note that these properties may cause difficulties for typical machine-learning optimization algorithms.Source: §6.1 · pp. 9–10
  • For general DNN approximations, parameters are difficult to interpret physically, and collocation points are training inputs rather than finite-element mesh nodes.The authors relate this distinction to the mesh-free character of DNN approximations.Source: §6.1 · p. 10

6.2. Linear Elasticity problem

The section applies a physics-informed neural network with the Deep Energy Method (DEM) to linear elasticity, finding displacement fields by minimizing stored elastic strain energy. It evaluates two- and three-dimensional problems against analytical solutions using relative L2 errors for displacement and strain energy.

6.2. Linear Elasticity problem

  • The DEM enforces homogeneous Neumann conditions automatically; the network minimizes energy error at integration points and boundary error for nonhomogeneous Neumann conditions, using Adam and L-BFGS.The section describes the method as minimizing the mean error of the energy functional at integration points.Source: §6.2 · p. 10

6.2.1. Pressurized thick-cylinder

  • For the pressurized thick cylinder, relative L2 errors are 0.5% for displacement and 3.7% for the strain-energy norm.The benchmark analyzes a quarter-annulus under internal pressure and plane stress; Fig. 2 shows displacement-field errors.Source: §6.2.1 · pp. 11, 13, Figure 2 · p. 13

6.2.2. Plate with a circular hole

  • For the plate with a circular hole under traction, prediction errors are 1.8% for u and 3.19% for strain energy.The problem is solved on a quarter of a finite plate, and Fig. 4 shows the computed solution and errors.Source: §6.2.2 · pp. 13–14, Figure 4 · p. 15

6.2.3. Hollow sphere under internal pressure

  • For the hollow sphere under internal pressure, prediction errors are 1.05% for u and 4.44% for strain energy.The model uses one-eighth of the sphere due to symmetry.Source: §6.2.3 · pp. 14, 16

6.2.4. Cube with a spherical hole subject to uniform tension

  • For the cube with a spherical hole under uniform tension, the strain-energy prediction error is 5.3%.The analysis uses one-eighth of the geometry due to symmetry.Source: §6.2.4 · pp. 16, 18

6.3. Elastodynamics

The section tests neural-network approximation on a time-dependent, one-dimensional wave-propagation problem. A space-time network trained with a collocation loss approximates displacement, with relative L2 errors reported for displacement and velocity.

6.3. Elastodynamics

  • The governing equilibrium equation models one-dimensional wave propagation, with displacement u(x, t) and prescribed Dirichlet and Neumann boundary data.The section uses this example to investigate neural-network approximation for time-dependent problems.Source: §6.3 · pp. 18–19
  • 1.422382 · 10−3 for displacement and 5.954601 · 10−3 for velocity are the reported relative L2 errors.Figure 8 compares computed and exact solutions for the example.Source: Figure 8 · p. 19, §6.3 · p. 20
  • The example takes wave speed c = 1 on a domain of length L = 4, with a sinusoidal load on the left, a fixed right side, and zero initial displacement and velocity.The left-side load is −sin(πt) for 0 ≤ t ≤ 1 and zero otherwise.Source: §6.3 · p. 19
  • The displacement network uses two space-time inputs, three hidden layers of 50 neurons each, and one displacement output, computing through T = 2.The network is trained with Nint = 1992, Ninit = 199, and Nbnd = 200 collocation points.Source: §6.3 · pp. 19–20
  • Its collocation loss incorporates the equilibrium equation and the initial and boundary conditions, using points in the interior, space domain, and time domain.The network approximation is determined by parameters optimized by minimizing this loss.Source: §6.3 · pp. 19–20

6.4. Hyperelasticity

The deep energy method (DEM) uses potential energy as a neural-network loss to solve a hyperelastic torsion problem. Its displacements agree well with a fine-mesh finite-element reference, while stresses show some differences.

Hyperelasticity

  • The method minimizes total potential energy over admissible trial functions, using the potential energy as the neural-network loss.A feedforward network parameterized by weights and biases supplies the trial solution, with boundary conditions imposed before operators are applied.Source: §6.4 · pp. 20–21
  • The example is a homogeneous, isotropic Neo-Hookean cuboid clamped on the left and twisted 60° counterclockwise at the right, with body and surface tractions applied.The cuboid dimensions are length 1.25, width 1.0, and depth 1.0; the body force is [0, −0.5, 0] and the traction is [1, 0, 0].Source: §6.4 · p. 22
  • The 64,000-point training setup uses a 5-layer 3−30−30−30−3 network with tanh hidden activations, L-BFGS learning rate 0.5, and 50 training steps.The points form a 40 × 40 × 40 equidistant grid over the domain.Source: §6.4 · p. 23
  • DEM displacements agree well with the fine-mesh finite-element reference along AB, while stresses differ slightly because they are computed from displacement gradients in post-processing.The network predicts displacements; the gradient fields used for stresses are obtained from that displacement field.Source: Figure 13 · p. 24, §6.4 · p. 24
  • At the CDEF plane, DEM predicts displacement magnitude and von Mises stress for the twisted cuboid.The study compares the DEM solution and reference solution using L2 norm and H1 seminorm in two setups.Source: §6.4 · pp. 24–25, Figure 14 · p. 25
  • Table 2 reports good DEM results for L2 norm and H1 seminorm across two setups using 8,000 or 64,000 training points.The reference norms were computed with a fine-mesh FEniCS program; the supplied passages do not provide the table’s individual norm values.Source: Table 2 · p. 25, §6.4 · p. 25

6.5. Phase field modeling of fracture

This section applies energy minimization with physics-informed neural networks to phase-field fracture, approximating displacement and damage fields. It examines a one-dimensional model and tensile crack growth in a plate.

6.5. Phase field modeling of fracture

  • The phase-field model represents cracks with φ = 1 and undamaged material with φ = 0, and recovers sharp cracks as the length scale l0 approaches zero.The model minimizes total potential energy to solve for the displacement and fracture fields, following Griffith theory for brittle fracture.Source: §6.5 · pp. 25–26
  • The DEM neural network approximates displacement u and phase field φ by minimizing total energy E = Ee + Ec subject to prescribed Dirichlet displacement.Ee is stored elastic strain energy and Ec is fracture energy.Source: §6.5 · p. 26

6.5.1. One-dimensional phase field model

  • The one-dimensional example compares DCM and DEM against an analytical phase-field solution for a bar of length 50 with a crack at 25 and l0 = 1.Both methods use the same network architecture; DEM uses 8000 interior integration points.Source: §6.5.1 · pp. 27–28
  • Collocation has 70.6% relative L2 error because it misses the derivative discontinuity at x = 25; DEM performs better under the same network parameters.The supplied passages do not give a numerical relative L2 error for DEM.Source: §6.5.1 · p. 28, Figure 16 · p. 29, §6.5.2 · p. 30

6.5.2. Single-edge notched tension example

  • For plate crack initialization, DEM predicts accurately with a smaller network, while DCM needs explicit Neumann-boundary treatment, more collocation points, and 10 times more iterations.The plate has a horizontal crack extending from the midpoint of the left edge to its center; the strain-history function initializes the crack.Source: §6.5.2 · pp. 28, 30, 32
  • To observe crack propagation in the plate, the study simultaneously solves the elastic and phase fields with a monolithic solver, and presents deformed configurations alongside corresponding phase-field plots.The computation applies constant displacement increments of Δv = 1 × 10−3 mm.Source: §6.5.2 · p. 32

6.6. Piezoelectricity

The section presents piezoelectricity as electromechanical cross-coupling and applies DEM energy minimization to a cantilever beam under mechanical and electrical loading.

6.6. Piezoelectricity

  • Piezoelectricity is linear electromechanical coupling in which strain induces polarization and an electric field can deform geometry, with stress, strain, electric field, and displacement linked by constitutive tensors.The passage identifies the effect as common in non-centrosymmetric crystals and names the piezoelectric, dielectric, and elastic tensors.Source: §6.6 · pp. 32, 34
  • DEM solves for the field variables by minimizing total energy, subject to prescribed displacement, electric-potential, and traction boundary conditions.Total energy comprises bulk internal energy and external-force work.Source: §6.6 · p. 34

6.6.1. Cantilever beam

  • The cantilever is modeled in plane strain as isotropic and linearly elastic, using a fully connected network with three 150-neuron hidden layers and ADAM followed by L-BFGS.The first two layers use tanh activation and the final layer uses linear activation.Source: §6.6.1 · p. 34
  • DEM predicts the cantilever’s deformed shape and induced electric potential under point-load mechanical loading, and deformation under an imposed top-surface potential of V = −100 MV.The authors state that both loading cases demonstrate DEM’s ability to solve an electromechanically coupled system without classical numerical methods.Source: §6.6.1 · pp. 34–35, Figure 22 · p. 35

6.7. Kirchhoff Plate bending

The section applies deep collocation and deep energy methods to Kirchhoff plate bending, using DNNs to approximate deflection despite the C1 continuity challenge in mesh-based methods. Across plate examples, the methods reproduce analytical or exact deflections, while deep energy simplifies treatment of essential boundary conditions, including for plates with cut-outs.

6.7. Kirchhoff Plate bending

  • Because Kirchhoff plate bending is governed by a fourth-order PDE whose weak formulation uses second derivatives, mesh-based methods face a C1 continuity requirement that DNN approaches avoid.Deep collocation minimizes governing-equation residuals with boundary conditions, while deep energy minimizes total potential energy and requires only Dirichlet boundary conditions rather than learned twist moments and shear forces.Source: §6.7 · pp. 35–37

6.7.1. Simply-supported square plate on Winkler foundation

  • For the simply-supported square plate on a Winkler foundation, deeper and wider networks improve maximum-deflection accuracy, and predicted plate deflections agree well with the analytical solution.The reported comparisons vary hidden-layer depth and neuron count; the passage notes improved accuracy even with two hidden layers.Source: §6.7.1 · pp. 38–39, Figure 24 · p. 39

6.7.2. Clamped circular plate

  • For the clamped circular plate, all tested networks achieve relative deflection error magnitude 1 × 10−4, and the predicted deflection agrees well with the exact solution.Increasing hidden layers and neurons improves the maximum-deflection prediction and brings relative-error curves toward the exact solution.Source: §6.7.2 · p. 40, Figure 26 · p. 41

6.7.3. Simply-supported square plate under a sinusoidally distributed load

  • For the sinusoidally loaded square plate, the proposed sine activation outperforms Tanh, and a two-encoding-layer autoencoder balances accuracy and efficiency while its deflection agrees with the analytical solution.More encoding layers and neurons yield more stable and accurate results, but the reported two-layer configuration meets both accuracy and efficiency aims.Source: §6.7.3 · pp. 42, 45, Figure 29 · p. 43, Figure 30 · p. 44
  • IGA-FEM is faster and more accurate than deep energy for the sinusoidally loaded square plate, although deep energy remains acceptable and trained autoencoders predict deflection and stress quickly.The comparison uses a deep-energy DNN with one 30-neuron encoding layer and increasing collocation points; its computational time refers to training time.Source: §6.7.3 · p. 42, Figure 31 · p. 44

6.7.4. Simply-supported annular plate

  • For the simply-supported annular plate, deep energy handles the free inner boundary by imposing only essential conditions, and predicted deflection agrees with the analytical solution despite the cut-out.The proposed activation predicts whole-plate deflection better than Tanh; suitable point sampling affects stability, and two encoding layers can suffice for plate bending.Source: §6.7.4 · pp. 45, 46, 48, Figure 37 · p. 48

7. Concluding remarks

The paper presents DNNs as a proof of concept for approximating solutions to boundary value problems in computational mechanics. Its energy-based loss and network-defined approximation space yield a nonconvex optimization problem, with implementation benefits and important analytical challenges.

Concluding remarks

  • Several computational mechanics examples demonstrate the feasibility of using DNNs to construct approximation spaces for boundary value problems.The authors describe these examples as a proof of concept for approximating BVP solutions with deep-learning concepts and tools.Source: §7 · p. 48
  • The method uses physical energy to define the loss and neural-network architecture to define the approximation space, producing a finite-sum nonconvex problem solved with gradient-based optimization.This approach permits use of standard machine-learning libraries such as TensorFlow.Source: §7 · p. 48
  • Machine-learning concepts and tools enable straightforward implementation and transparent energy-based loss definitions for trying different mathematical models.The paper describes near-mathematical notation for defining the loss and corresponding energy.Source: §7 · pp. 48–49
  • The approach remains challenging because its optimization problems are nonconvex and expressive approximation spaces can be difficult to analyze, even for linear problems.Even linear problems produce nonlinear, nonconvex discrete problems, posing challenges for the computational mechanics community.Source: §7 · p. 49

Related papers

Loading 1908.10407v2…