Source-linked AI summary
fPINNs: Fractional Physics-Informed Neural Networks
Guofei Pang, Lu Lu, George Em Karniadakis
TL;DR
The paper addresses the difficulty of applying PINNs to fractional PDEs, whose derivatives do not support the classical chain rule used by standard automatic differentiation. It introduces fPINNs, which combine automatic differentiation for integer-order operators with numerical discretization for fractional operators, and reports accurate forward and inverse solutions for fractional ADEs, including black-box forcing and noisy data.
Problem
Standard PINNs cannot be directly applied to fractional PDEs because the classical chain rule is not valid in fractional calculus.
Method
fPINNs construct fractional-PDE residuals with numerical differentiation formulas for fractional operators and automatic differentiation for integer-order operators.
Results
fPINNs solve forward and inverse fractional ADE problems accurately across 1D, 2D, and 3D settings, including black-box forcing and complex-geometry domains.
Takeaways & Limitations
The framework preserves high solution accuracy with black-box forcing and can handle high-dimensional complex-geometry domains without a mesh.
Takeaways & Limitations
fPINN convergence is not guaranteed because of optimization error, and its computational cost is generally larger than FDM when both methods are available.
Abstract
from arXiv · showhide
Physics-informed neural networks (PINNs) are effective in solving integer-order partial differential equations (PDEs) based on scattered and noisy data. PINNs employ standard feedforward neural networks (NNs) with the PDEs explicitly encoded into the NN using automatic differentiation, while the sum of the mean-squared PDE-residuals and the mean-squared error in initial/boundary conditions is minimized with respect to the NN parameters. We extend PINNs to fractional PINNs (fPINNs) to solve space-time fractional advection-diffusion equations (fractional ADEs), and we demonstrate their accuracy and effectiveness in solving multi-dimensional forward and inverse problems with forcing terms whose values are only known at randomly scattered spatio-temporal coordinates (black-box forcing terms). A novel element of the fPINNs is the hybrid approach that we introduce for constructing the residual in the loss function using both automatic differentiation for the integer-order operators and numerical discretization for the fractional operators. We consider 1D time-dependent fractional ADEs and compare white-box (WB) and black-box (BB) forcing. We observe that for the BB forcing fPINNs outperform FDM. Subsequently, we consider multi-dimensional time-, space-, and space-time-fractional ADEs using the directional fractional Laplacian and we observe relative errors of $10^{-4}$. Finally, we solve several inverse problems in 1D, 2D, and 3D to identify the fractional orders, diffusion coefficients, and transport velocities and obtain accurate results even in the presence of significant noise.
1 Introduction
Fractional PDE parameters are difficult to identify, and standard PINNs cannot directly encode fractional operators. The paper introduces fPINNs for forward and inverse fractional ADE problems, including black-box forcing and irregular high-dimensional domains.
- Motivation: Fractional PDEs model memory effects, spatial nonlocality, and power-law characteristics, with fractional orders requiring estimation from experimental data.For example, the time-fractional order can determine the long-term decay rate of a breakthrough curve.
- Motivation: Standard PINNs cannot be directly applied to fractional PDEs because the classical chain rule is not valid in fractional calculus.This prevents the usual automatic-differentiation treatment from being transferred directly to fractional operators.
- Contributions: fPINNs bypass interpolation for scattered black-box forcing by constructing equation residuals directly at measurement points.This addresses interpolation errors that can become large when forcing measurements are sparse.
- Contributions: fPINNs handle high-dimensional irregular domains using arbitrarily distributed training points rather than fixed meshes or grids.Auxiliary points are used to calculate fractional derivatives at training points.
- Contributions: The same code supports forward and inverse problems by adding unknown parameters to the optimized parameter list.No other code changes are required according to the paper.
- Scope: fPINNs solve 1D, 2D, and 3D fractional ADEs for both forward and inverse problems, including space-, time-, and space-time-fractional cases.The paper also attempts forward and inverse space-time-fractional ADEs in complex-geometry domains.
2 Fractional advection-diffusion equations (ADEs)
The paper formulates time-, space-, and space-time-fractional ADEs on bounded domains with black-box forcing and zero boundary conditions. Forward problems recover concentration fields, while inverse problems identify fractional orders, diffusion, velocity, and the field from final-time data.
- Equation and assumptions: The fractional ADE combines a Caputo time-fractional derivative, a directional fractional Laplacian, advection, and black-box forcing on a bounded domain.The solution is assumed zero outside the domain, and the forcing is known only at scattered spatio-temporal coordinates.
- Forward problem: For forward problems, the concentration field is solved given fractional orders, diffusion coefficient, flow velocity, forcing, and initial-boundary conditions.The forward formulation treats these quantities as specified inputs.
- Inverse problem: For inverse problems, fractional orders, diffusion coefficient, flow velocity, and concentration field are inferred from initial-boundary conditions, black-box forcing, and final-time concentration measurements.The study uses final-time data rather than scattered measurements in time.
3 Methodology
The methodology represents PDE solutions with neural networks and trains them by minimizing residual-based losses at selected spatio-temporal points. fPINNs extend this framework by combining automatic differentiation for integer-order operators with numerical discretization for fractional operators, supporting forward and inverse problems.
- Physics-Informed Neural Networks (PINNs): PINNs approximate the PDE solution with a neural network whose parameters are optimized to satisfy the equation and initial-boundary conditions.The network can directly serve as the surrogate solution, or the approximate-solution form can enforce boundary conditions automatically.
- Physics-Informed Neural Networks (PINNs): The PINN loss uses mean-squared equation-residual information from training points, with scattered points adopted for most examples.Training points may be lattice-like finite-difference-grid points or scattered points drawn from a quasi-random sequence.
- Physics-Informed Neural Networks (PINNs): Automatic differentiation computes the integer-order temporal and spatial derivatives, but fractional derivatives cannot generally use the classical chain rule.For deep networks, automatic differentiation applies the chain rule hierarchically; fractional derivatives may require infinite-series forms instead.
- Fractional PINNs (fPINNs): fPINNs split the differential operator into automatically differentiable and non-automatically differentiable parts, using numerical discretization for the latter.The non-AD component includes fractional temporal and spatial operators, and the paper focuses on finite-difference discretization.
- Fractional PINNs (fPINNs): The fPINN forward loss averages squared residuals over training points, and minimizing it yields optimized network parameters for predictions at arbitrary test points.If the non-AD operator vanishes, fPINNs reduce to PINNs.
- Inverse Problems: Inverse fPINNs jointly optimize network parameters and PDE parameters to recover fractional orders, diffusion coefficients, flow velocity, and the concentration field.The inverse loss adds a final-condition mismatch term and optimizes parameters such as α, γ, c, and v.
- Finite-Difference Schemes for Fractional Derivatives: The time-fractional L1 scheme makes each derivative evaluation depend on the current value and all previous time-step values, represented by auxiliary points.There are ⌈λt⌉ + 1 auxiliary points for a training point at time t, allowing scattered temporal and spatial training points.
4 Numerical examples
The numerical examples evaluate fPINNs across convergence, time-dependent forward, multidimensional, inverse, and noisy-data problems, while examining discretization, optimization, sampling, and network-approximation effects. fPINNs are especially competitive for black-box forcing and recover multidimensional solutions and parameters accurately, subject to optimization and computational limitations.
- Experimental setup: The experiments assess fPINNs using fabricated solutions, L2 relative error, and test points selected from finite-difference grids or Sobol sequences.The study covers forward and inverse problems, including noisy-data cases.
- Numerical convergence: Optimization error can limit convergence because the fPINN loss is high-dimensional and non-convex, with more than 1000 network parameters in a moderate example.The lowest reported fPINN MSE was 10^-14, compared with squared machine precision of 10^-32 for the FDM residual.
- Numerical convergence: For smooth solutions, fPINN error initially approaches FDM accuracy but can saturate as training points increase, while uncertainty increases.The saturation is attributed to the increasing complexity of the loss function.
- Numerical convergence: Network depth affects solution accuracy more than width: greater depth can slightly improve accuracy but also increase uncertainty, while extreme depth or width produces larger errors.The experiments indicate an optimal depth and width under fixed training and discretization conditions.
- Time-dependent forward problems: For black-box forcing, fPINNs outperform FDM especially with sparse observations because they avoid interpolation errors in the forcing term.Both methods become more accurate as the number of observation points increases, with the strongest fPINN advantage reported for N < 50.
- Multidimensional and inverse problems: Relative errors for 2D and 3D time-dependent problems range from 10^-4 to 10^-3, while time-dependent inverse problems recover both PDE parameters and concentration fields very well.Good initialization is used to search the parameter space efficiently, and noisy experiments show that 5% forcing noise can still yield nearly 1% forward relative error.
5 Summary and discussion
fPINNs extend PINNs to fractional PDEs by combining automatic differentiation for integer-order operators with numerical differentiation for fractional operators. The framework targets black-box forcing and complex, high-dimensional domains, while retaining limitations in convergence guarantees, computational cost, and sparse-forcing integration accuracy.
- Method: fPINNs encode time-, space-, and space-time-fractional advection-diffusion equations using a hybrid residual representation.Automatic differentiation handles integer-order operators, while numerical differentiation formulas represent fractional operators.
- Discretization: fPINNs use auxiliary points to approximate fractional derivatives, while training points may be scattered rather than restricted to a fixed mesh.The mixed representation is affected jointly by discretization, sampling, neural-network approximation, and optimization errors.
- Black-box forcing: Black-box forcing can be handled without interpolating the forcing term by constructing equation residuals at measurement points.This is intended to reduce interpolation errors when forcing measurements are sparse.
- Limitations: Sparse black-box forcing makes the integral evaluation less accurate, and fPINN convergence cannot be guaranteed because of optimization error.The paper also reports that fPINNs generally cost more computationally than FDM when both methods are available.
Appendix A: Chain rules for integer-order and fractional derivatives
The appendix contrasts classical and fractional chain rules, showing why fractional derivatives cannot be handled by standard automatic differentiation in the same way as integer-order derivatives.
- Integer- and fractional-order chain rules: The k-th derivative of a composite function is represented using combinations of derivatives of the inner function.The appendix introduces the classical higher-order chain-rule structure before presenting the fractional case.
- Integer- and fractional-order chain rules: The Caputo fractional chain rule includes an infinite series, making direct use in neural-network computations computationally prohibitive.The infinite-series structure distinguishes fractional differentiation from the finite classical chain-rule formulation.
- Integer- and fractional-order chain rules: For fractional Laplacians, a corresponding chain rule may not exist at all.This provides the strongest contrast with the classical chain rule used for standard neural-network differentiation.
Appendix B: Grunwald-Letnikov finite difference schemes
The appendix defines shifted Grunwald-Letnikov finite-difference operators on a stationary grid for approximating the one-dimensional fractional Laplacian.
- GL finite-difference schemes: The shifted GL finite-difference operator approximates the one-dimensional fractional Laplacian on a stationary grid.The grid is defined by x_j = (j − 1)Δx for j = 1, 2, · · ·, N.
- GL finite-difference schemes: The schemes shift by p step sizes to guarantee stability.The shift is introduced as a design choice in the finite-difference approximation.
- GL finite-difference schemes: First-, second-, and third-order GL formulas are provided for approximating the one-dimensional fractional Laplacian.The formulas use β = 1 − α/2 and correspond to k(1) = 4, k(2) = 5, and k(3) = 6.