Source-linked AI summary
Neural Networks Based on Power Method and Inverse Power Method for Solving Linear Eigenvalue Problems
Qihong Yang, Yangtao Deng, Yu Yang, Qiaolin He, Shiquan Zhang
TL;DR
Direct loss minimization can fail to learn the smallest eigenvalue, while neural-network approaches have received little attention for largest-positive and interior eigenvalues. This paper combines neural networks with power and inverse power methods to approximate linear eigenvalues and eigenfunctions. Its experiments report accurate approximations across dimensions and better eigenvalue results than methods including DRM and FBSDE.
Problem
Direct loss minimization may fail to learn the smallest eigenvalue, and little neural-network research addresses largest-positive or interior eigenvalues.
Source: §1 · p. 2Method
PMNN and IPMNN combine power-method iterations with neural-network eigenfunction approximations, using automatic differentiation for the operator and specially defined losses.
Source: §1 · p. 2, §3.1.1 · p. 5, §3.1.2 · p. 6Results
Experiments report accurate eigenfunction approximations across dimensions and better eigenvalue results than DRM and FBSDE, with eigenvalues examined for d = 1, 2, 5, and 10.
Source: §5 · pp. 18–19Takeaways & Limitations
IPMNN can also solve for interior eigenvalues when prior knowledge α is given.
Source: §5 · p. 19The methods address linear differential operators, and may fail when the dominant or smallest eigenvalue has multiple values.
Source: §5 · p. 19
Abstract
from arXiv · showhide
In this article, we propose two kinds of neural networks inspired by power method and inverse power method to solve linear eigenvalue problems. These neural networks share similar ideas with traditional methods, in which the differential operator is realized by automatic differentiation. The eigenfunction of the eigenvalue problem is learned by the neural network and the iterative algorithms are implemented by optimizing the specially defined loss function. The largest positive eigenvalue, smallest eigenvalue and interior eigenvalues with the given prior knowledge can be solved efficiently. We examine the applicability and accuracy of our methods in the numerical experiments in one dimension, two dimensions and higher dimensions. Numerical results show that accurate eigenvalue and eigenfunction approximations can be obtained by our methods.
1. Introduction
Prior neural-network approaches to eigenvalue problems have limitations in higher dimensions, and direct loss minimization may fail to learn the smallest eigenvalue. The paper introduces PMNN and IPMNN for linear eigenvalue problems, using automatic differentiation and specially defined losses to approximate eigenvalues and eigenfunctions iteratively.
1. Introduction
- Earlier neural-network approaches DRM and PINN do not handle all problems, including higher-dimensional eigenvalue problems with λ ≠ 0; this work focuses on linear problems and leaves nonlinear problems for future work.The article also frames the work as a first step toward neural-network methods for discontinuous-interface K-eigenvalue problems in reactor physics.Source: §1 · pp. 1–2
- The introduction places eigenvalue solving in applications including nuclear reactor physics, elastic acoustics, and elastic viscoelastic composite structures [15].It notes that traditional power methods on discretized equations depend on mesh subdivision and become less efficient at large degrees of freedom.Source: §1 · p. 1
- DRM uses a variational Rayleigh quotient for the smallest eigenvalue, while PINN-based approaches construct a Rayleigh-quotient function without variation [18].The article defines the smallest eigenvalue as the minimum-modulus eigenvalue.Source: §1 · p. 2
- Regularization-based methods can be difficult to apply because the convergence point may be hard to identify and results can depend on hyperparameters; one cited regularization term depends strongly on constant c [25].Another approach reformulates the problem as a semigroup-flow fixed point and approximates the eigenfunction with a neural-network ansatz using FBSDEs.Source: §1 · p. 2
- Directly minimizing eigenvalue-related losses may fail to learn the smallest eigenvalue, and balancing their many terms is difficult; little neural-network research targets largest-positive or interior eigenvalues.The paper identifies this as motivation for new methods.Source: §1 · p. 2
- PMNN and IPMNN target the largest positive and smallest eigenvalues, plus interior eigenvalues when an approximation is given, by iteratively learning eigenfunctions with automatic differentiation and specially defined losses.The differential operator is not discretized into a matrix system.Source: §1 · p. 2
2. Preliminaries
The preliminaries formulate a linear eigenvalue problem with boundary constraints, then review power iteration for the dominant eigenvalue and inverse power iteration for the smallest eigenvalue in absolute value.
2.1. Eigenvalue Problems
- The eigenvalue problem is posed for a self-adjoint differential operator with a boundary constraint; this work considers Dirichlet and periodic conditions, and notes that a singular operator with smallest eigenvalue zero should be shifted.The differential equation defines an eigenfunction and corresponding eigenvalue, while the boundary condition constrains the problem.Source: §2.1 · pp. 2–3
2.2. Power Method
- Power iteration targets the dominant eigenvalue, the eigenvalue largest in absolute value, and its eigenvector; repeated multiplication and normalization make the dominant-eigenvector term prevail as k increases.The eigenvalue is typically estimated with the Rayleigh quotient u^T Au.Source: §2.2 · p. 3
2.3. Inverse Power Method
- Inverse power iteration targets the smallest eigenvalue in absolute value by making the corresponding eigenvalue of A^-1 dominant; iteration applies A^-1, normalizes, and estimates the eigenvalue with a Rayleigh quotient.The described iteration uses LU decomposition to apply A^-1 and stops when successive iterates are sufficiently close or the iteration limit is reached.Source: §2.3 · pp. 3–4
3. Methodologies
The paper represents eigenfunctions with neural networks and adapts power-method iterations through losses optimized over sampled points. PMNN targets the largest positive eigenvalue, while IPMNN targets the smallest and, with a shift based on prior knowledge, interior eigenvalues; boundary enforcement is described for Dirichlet and periodic conditions.
3.1.1. Power Method Neural Network
- PMNN represents the current eigenfunction with a neural network, applies the differential operator through automatic differentiation, normalizes the result, and trains toward it with a sampled-point loss.The loss allows iterative approximation without requiring the network to reproduce the normalized iterate exactly at each step.Source: §3.1.1 · pp. 5–6
3.1.2. Inverse Power Method Neural Network
- IPMNN targets the smallest eigenvalue by minimizing a loss that approximates the inverse-power update without explicitly computing the differential operator’s inverse.The network represents the current iterate, and the loss drives it toward the inverse-power relation.Source: §3.1.2 · pp. 6–7
- Shifting the operator by a prior value α lets IPMNN target an eigenvalue close to α and its corresponding eigenfunction.The shift rewrites the original problem using the shifted operator and eigenvalue.Source: §3.1.2 · pp. 6–7
3.2.1. Enforcement of Dirichlet Boundary Condition
- For Dirichlet conditions, the solution is constructed using a distance function and a smooth extension of the boundary data that matches it on the boundary.The paper limits its treatment to Dirichlet and periodic conditions; more complicated domains or boundary conditions are left for future work.Source: §3.2.1 · pp. 7–8, §3.2 · p. 7
3.2.2. Enforcement of Periodic Boundary Condition
- For periodic conditions, the network input is modified before its first hidden layer, and the network width increases from d to d + 2k − 1.The stated periodicity is along direction i with period P_i.Source: §3.2.2 · p. 8, Figure 1 · p. 8
4. Numerical Experiments
The experiments test PMNN and IPMNN on eigenvalue problems across dimensions, including comparisons with finite-difference and other methods. They report accurate eigenvalue and eigenfunction approximations, while noting that interior-eigenvalue recovery depends on prior knowledge.
4.1. An Example of Eigenvalue Problem implemented by PMNN
- For the PMNN problem, eigenvalue and eigenfunction errors decrease toward zero during iteration in one dimension, and eigenfunction densities match the exact densities for d = 1, 2, 5, and 10.The reported relative error is small across dimensions.Source: §4.1 · p. 9, Table 2 · p. 9, Figure 2 · p. 10
4.2.1. Harmonic Eigenvalue Problem
- For the harmonic eigenvalue problem, IPMNN finds the smallest eigenvalue and associated eigenfunction, with small relative error across dimensions and matching eigenfunction densities for d = 1, 2, 5, and 10.The study compares its higher-dimensional results with DRM.Source: §4.2.1 · p. 12, Table 3 · p. 12, Figure 9 · p. 14
4.2.2. Comparison with Finite Di
- In two dimensions, IPMNN approximates the eigenvalue and eigenfunction better than FDM, but requires much more training time; accuracy improves for both as point counts increase.The experiments also report IPMNN as suitable for high-dimensional problems, where traditional numerical methods are difficult to use.Source: Figure 10 · p. 15, §4.2.2 · p. 15
4.2.3. Fokker-Planck Equation
- For the periodic Fokker-Planck problem, IPMNN approximates the zero smallest eigenvalue accurately across dimensions, and its learned eigenfunction densities match the exact densities in the reported cases.The method uses a shifted operator for this zero-eigenvalue problem; higher-dimensional results are compared with FBSDE.Source: §4.2.3 · pp. 15–16, Table 5 · p. 16
4.2.4. Interior Eigenvalues Solved by IPMNN
- With prior knowledge α, IPMNN accurately solves interior eigenvalues and eigenfunctions in the one-dimensional example, but often fails when α is far from the target eigenvalue.The authors stress that knowing the eigenvalue distribution matters when selecting a specified eigenvalue.Source: §4.2.4 · pp. 16, 18, Table 6 · p. 18, Figure 14 · p. 18
5. Conclusions
The conclusions report accurate eigenfunction learning and better eigenvalue results than methods in the literature, while identifying limits in the methods’ current scope.
5. Conclusions
- PMNN and IPMNN use neural networks to learn eigenfunctions, automatic differentiation for differential operators, and power-method-inspired iterative loss functions.The methods enforce boundary conditions and approximate eigenvalues and eigenfunctions iteratively.Source: §5 · p. 18
- Numerical results show accurate eigenfunction learning and better eigenvalue results than DRM and FBSDE in the reported comparisons.The experiments include one-, two-, and higher-dimensional problems.Source: §5 · pp. 18–19
- IPMNN also solves interior eigenvalues when given prior knowledge α.Source: §5 · p. 19
- The methods address linear differential operators, may fail for multiple dominant or smallest eigenvalues, and need further study for nonlinear operators and complex geometries or materials.The authors also identify sampling algorithms and the number of sampling points as areas for further study.Source: §5 · p. 19