Source-linked AI summary
Kolmogorov Arnold Informed neural network: A physics-informed deep learning framework for solving forward and inverse problems based on Kolmogorov Arnold Networks
Yizheng Wang, Jia Sun, Jinshuai Bai, Cosmin Anitescu, Mohammad Sadegh Eshaghi, Xiaoying Zhuang, Timon Rabczuk, Yinghua Liu
TL;DR
Complex PDE boundary and initial conditions motivate approximate AI-based solvers, while equivalent strong, energy, and inverse formulations can differ computationally. The paper proposes KINN, replacing MLPs with KANs across these formulations, and reports higher accuracy and convergence speed than MLPs on most tested PDE problems, with weaker performance on complex geometries.
Problem
Complex boundary and initial conditions make exact PDE solutions difficult, while mathematically equivalent PDE formulations can differ in computational accuracy and efficiency.
Method
KINN replaces MLPs with KANs in strong, energy, and inverse PDE formulations for forward and inverse problems.
Results
KAN achieves higher accuracy and convergence speed than MLP in most tested PDE problems, including singularity, stress-concentration, nonlinear hyperelastic, and heterogeneous cases.
Takeaways & Limitations
KINN shows potential for more accurate and efficient AI-based PDE solutions across many computational solid-mechanics problems.
Takeaways & Limitations
KINN performs worse on complex geometries because rectangular KAN grids conflict with geometric complexity and cannot be enlarged indefinitely without increasing errors.
Abstract
from arXiv · showhide
AI for partial differential equations (PDEs) has garnered significant attention, particularly with the emergence of Physics-informed neural networks (PINNs). The recent advent of Kolmogorov-Arnold Network (KAN) indicates that there is potential to revisit and enhance the previously MLP-based PINNs. Compared to MLPs, KANs offer interpretability and require fewer parameters. PDEs can be described in various forms, such as strong form, energy form, and inverse form. While mathematically equivalent, these forms are not computationally equivalent, making the exploration of different PDE formulations significant in computational physics. Thus, we propose different PDE forms based on KAN instead of MLP, termed Kolmogorov-Arnold-Informed Neural Network (KINN) for solving forward and inverse problems. We systematically compare MLP and KAN in various numerical examples of PDEs, including multi-scale, singularity, stress concentration, nonlinear hyperelasticity, heterogeneous, and complex geometry problems. Our results demonstrate that KINN significantly outperforms MLP regarding accuracy and convergence speed for numerous PDEs in computational solid mechanics, except for the complex geometry problem. This highlights KINN's potential for more efficient and accurate PDE solutions in AI for PDEs.
Nomenclature
The nomenclature defines abbreviations for neural-network PDE methods, numerical methods, approximation functions, and related computational techniques.
- BINN denotes Boundary-Integral Type Neural Networks, and KINN variants apply KAN to BINN, CPINNs, DEM, or PINNs.
- PINNs denotes Physics-Informed Neural Networks, while CPINNs denotes the strong form of PINNs with subdomains.
- DEM denotes the Deep Energy Method, which uses the energy form of PDEs.
- FEM, IGA, NURBS, RBF, and FNO identify finite-element, isogeometric, spline, radial-basis, and Fourier-neural-operator concepts, respectively.
- MLP denotes a fully connected neural network, while KAN denotes a Kolmogorov-Arnold Network.
- Other listed abbreviations include Monte Carlo integration, neural tangent kernel, partial differential equations, physics-informed neural operator, and subdomain energy methods.
1. Introduction
The introduction motivates replacing MLPs with KANs across multiple PDE formulations because equivalent formulations can differ computationally and KANs may improve approximation properties.
- Complex boundary and initial conditions often make exact PDE solutions difficult, motivating approximate numerical and learning-based methods.
- AI for PDEs includes PINNs, operator learning, and physics-informed neural operators, with strong, weak, energy, and inverse formulations.
- Although PDE formulations are mathematically equivalent, they can differ in computational accuracy and efficiency.
- KANs learn activation functions using B-splines, whereas MLPs use fixed activation functions and conventional learned weights.
- The paper proposes KINN by replacing MLPs with KANs in strong, energy, and inverse PDE forms for forward and inverse problems.
- Experiments report KAN advantages for multi-scale, singularity, stress-concentration, nonlinear, and heterogeneous problems, but worse performance for complex boundary problems.
2. Preparatory knowledge
This section presents strong, energy, and inverse PDE formulations and explains how each becomes a neural-network optimization problem with distinct computational requirements.
- Strong form: A boundary-value PDE is expressed through domain and boundary operators, then transformed using weighted residuals and numerical integration.
- Strong form: Strong-form PINNs choose a specific weight function, numerically approximate the residual integrals, and optimize a neural approximation of the field variable.
- Energy form: The energy form uses the variation δu as a weight function and represents the problem through domain and boundary energy.
- Energy form: DEM requires an admissible function satisfying essential boundary conditions in advance, then minimizes the potential energy represented by the neural approximation.
- Energy form: Not all PDEs possess an energy form; mathematically, the energy formulation requires a linear self-adjoint operator with δ2L > 0.
- Inverse form: BINN derives boundary integral equations by transferring differential operators through weighted residuals and Gaussian integration, using fundamental solutions.
- Inverse form: BINN approximates unknown boundary quantities with a neural network, while singular and Cauchy-principal-value integrals require careful numerical treatment.
3. Method
KINN replaces MLPs with KANs in multiple PDE forms, optimizing B-spline-based activation coefficients on a formulation-specific loss using a virtual grid.
- KINN: KINN applies KAN instead of MLP to strong, energy, and inverse PDE forms for solving forward and inverse problems.
- KAN architecture: KAN activation functions are constructed from B-splines whose coefficients are trainable parameters determined by grid counts and spline orders.
- KAN architecture: Scaling factors, nonlinear activation, linear operations, and residual terms are combined to produce each KAN layer output.
- KINN training: After selecting grid size, grid range, spline order, and PDE form, KINN samples domain and boundary points to compute losses and optimize KAN parameters.
- KINN properties: KAN approximations resemble compositions of NURBS-based functions or finite-element shape functions, and solvable derivative order depends on activation and spline choices.
- KINN training: The virtual grid is determined by KAN grid size, while KINN optimizes B-spline coefficients through losses based on strong, energy, or inverse numerical formats.
4. Result
Across the tested PDE settings, KINN generally improves accuracy, convergence, or parameter efficiency over MLP-based methods, especially for multi-scale, singular, stress-concentration, and heterogeneous problems. Its advantages depend on hyperparameter choices and geometry, with limitations for high frequencies, noisy inverse data, and complex geometries.
- Multi-scale and multi-frequency problems: KAN almost avoids MLP’s spectral bias and solves some mixed-frequency problems that MLP cannot.The comparison uses identical optimization settings while changing only the network type.
- Multi-scale and multi-frequency problems: KAN’s spectral bias depends on grid size, and overly small grids fail on high-frequency problems.The reported relative error is evaluated across frequencies and grid sizes after 3000 iterations.
- Singularities: KINN improves error distribution and maximum absolute error over MLP-based CPINN, DEM, and BINN algorithms for the mode III crack.The KINN architecture used for CPINN and DEM has only 600 parameters.
- Singularities: KINN converges faster and reaches higher accuracy than corresponding MLP versions in the mode III crack problem.KINN-DEM performs best among the compared versions for the singular displacement derivative, although a jump appears at x = 0.5.
- Singularities: Triangular-grid integration produces smaller errors than uniform Monte Carlo integration in DEM and improves accuracy across numerical integrations.The triangular-grid approach reduces some of DEM’s meshless advantages while remaining adaptable to complex geometries.
- Other PDE problems: KINN achieves high accuracy and fast convergence for stress-concentration, nonlinear hyperelastic, heterogeneous, and inverse heterogeneous problems, but offers no significant improvement for complex geometries.For the plate with a central hole, KINN-DEM nearly matches FEM accuracy; noisy heterogeneous inverse data may be overfit by KAN.
5. Conclusion
The paper proposes KINN by replacing MLPs with KANs across strong, energy, and inverse PDE forms. KAN generally improves accuracy and convergence, but remains less effective for complex geometries and less efficient under equal epochs.
- KINN applies KANs to strong, energy, and inverse PDE formulations for forward and inverse problems.
- KAN achieves higher accuracy and faster convergence than MLP in most PDE problems, especially singularity, stress concentration, nonlinear hyperelastic, and heterogeneous problems.
- KAN has lower efficiency than MLP under the same epoch because the KAN algorithm lacks specific optimizations.
- KAN performs worse than MLP on complex geometric PDE problems, primarily because grid size conflicts with geometric complexity.
- For extreme heterogeneous inverse problems, KAN provides stronger fitting capability and higher accuracy than MLP.
- An overly large KAN grid can cause overfitting, so grid size must be selected according to problem complexity.
Appendix A. The requirement of PINNs to DEM
Appendix A establishes conditions under which a strong-form PDE problem can be transformed into an energy or extremum formulation. The transformation relies on linearity and self-adjointness of the differential operator, with boundary terms handled through integration by parts.
- The PDE is first expressed in Galerkin form as a general variational representation.
- The differential operator must be linear and self-adjoint for the stated transformation between strong and variational forms.
- Integration by parts transforms operator terms into boundary integral terms involving the trial functions.
- For a linear self-adjoint operator of even order 2m, the stationary-value problem can be transformed into an extremum problem used by DEM.
Appendix B. Integration strategies in BINN
Appendix B describes piecewise boundary integration and analytical transformations for singular integrals in BINN. Weakly singular and Cauchy-principal integrals are regularized before Gaussian quadrature is applied.
- The boundary is divided into segments with source points at segment centers, and integrals are evaluated piecewise.
- Integrals over segments not containing the source point are regular and are computed with Gaussian quadrature.
- Weakly singular integrals use a segment-local parameterization in which the regular factor includes boundary conditions, neural networks, and Jacobians.
- Integration by parts and Taylor expansion remove the singular behavior from the weakly singular terms.
- A substitution and Taylor expansion eliminate the strong singularity in Cauchy-principal integrals, leaving Gaussian quadrature for the remaining term.
Appendix C. Similarities between KAN and Finite Elements Method
Appendix C relates KAN representations to finite-element and isogeometric shape functions. Special KAN structures can reproduce finite-element approximations, while nested B-splines make KAN mathematically similar to nested FEM and NURBS functions.
- KAN and FEM equivalence: A one-dimensional KAN with order=2 and grid size=2 can completely encompass quadratic finite elements.
- KAN and FEM equivalence: In a special first-order structure, KAN is equivalent to a linear one-dimensional finite element.
- KAN and FEM equivalence: KAN and FEM differ in their two-dimensional functional forms, but multiplicative KAN activations can exactly match the FEM representation.
- Deep Finite Element Method: Adjusting KAN activations from additive to multiplicative combinations could produce a Deep Finite Element Method.
- KAN and FEM equivalence: The similarity between KAN and FEM arises because both use spline-based fitting functions, while deeper KAN structures are expected to provide stronger fitting capability.
- KAN and IGA: Because B-splines are a degenerate form of NURBS, KAN is mathematically similar to nested NURBS in isogeometric analysis.
Appendix D. The highest order of PDEs that KINN can solve
KINN’s ability to solve high-order PDEs depends on preserving non-zero higher-order derivatives through its activation and residual structure. The analysis and infinite-wedge example show why tanh and the residual term are essential.
- Applicability conditions: Higher-order PDEs require neural-network derivatives to remain non-zero, so KINN’s applicability depends on its network structure.The section analyzes these applicability conditions from KAN’s layered computation.
- Derivative construction: KAN derivatives are obtained by applying the chain rule to the network output, approximating the PDE’s differential operator.The derivation expands layerwise tensor products and B-spline terms.
- Derivative construction: tanh keeps first-order derivatives non-zero and, through smooth higher-order derivatives and the product rule, supports non-zero higher-order derivatives.This mechanism is stated explicitly for the single-layer KINN chain rule and extended to the full network.
- B-spline limitation: Using only third-order B-splines in a [2,5,1] KAN limits the maximum derivative order to 6, making higher derivatives zero.The reported bound is 3∗(3−1) = 6.
- Infinite-wedge verification: Removing tanh and the residual term causes a [1,1] KAN with spline order 3 to fail on the fourth-order infinite-wedge PDE.The result agrees with the theoretical derivative-order analysis.
- Infinite-wedge verification: Fig. D.27 compares analytical solutions, KINN predictions, and absolute errors for σr, σθ, and τrθ in the infinite-wedge problem.The panels are grouped into three analytical-solution, prediction, and error triplets.
Appendix E. Scale normalization in Deep Energy Method
Scale normalization keeps KINN inputs within the fixed B-spline grid and removes coordinate-size effects from derivatives. It also preserves the solution’s scaling with the simulated object size.
- Purpose: Scale normalization maps inputs into [−1, 1], matching the fixed B-spline grid and improving neural-network fitting.The authors state that this operation is especially essential for KAN.
- Size effects: Without normalization, large coordinates can produce excessively large coordinate gradients and normal strains during differentiation.The magnitude is mainly influenced by the size-dependent term x ∂ϕx(x;θ)/∂x.
- Normalization formulation: The proposed modification introduces the characteristic object size L into the trial expressions before differentiation.The resulting expressions are presented as the scale-normalized forms of the original equations.
- Scaling behavior: When the PDE simulation size increases by a factor of L, the solution scales by the same factor under the normalized formulation.Including the coordinate x in the expressions maintains this size relationship.
Appendix F. The Challenge to Solve Inverse Problems in Deep Energy Methods
Directly minimizing the Deep Energy Method energy form can make inverse elasticity optimization mathematically degenerate. The paper identifies the omitted external-force work and proposes displacement-error optimization instead.
- Inverse formulation: The inverse problem treats elastic modulus E and Poisson’s ratio υ as unknowns while displacement, force, and boundary-force fields are known.The prescribed displacement is defined on the Dirichlet boundary Γu.
- Energy-form degeneracy: With displacement pre-fitted, strain is fixed and non-negative strain energy makes E = 0 with any υ the formal minimum.The volumetric and shear strain-energy contributions are both non-negative.
- Energy-form degeneracy: This optimum is physically invalid because the elastic modulus cannot be zero, revealing an optimization error in the DEM energy form.The analysis attributes the error to ignoring external-force work in the variational optimization.
- Alternative strategy: A proposed DEM alternative initializes E and υ, solves the PDE for displacement, compares computed and given displacements, and optimizes the initial guesses.This approach is described as analogous to the method used in finite elements.