Source-linked AI summary
Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations
Ben Moseley, Andrew Markham, Tarje Nissen-Meyer
TL;DR
PINNs are useful for mesh-free differential-equation modelling but struggle with large domains and multi-scale solutions because optimisation complexity and spectral bias increase. The paper introduces FBPINNs, which use overlapping subdomain networks, local normalisation, and parallel divide-and-conquer training. Experiments find accurate solutions across smaller, larger, and multi-scale problems, with FBPINNs generally outperforming standard PINNs in accuracy and computational resources required.
Problem
PINNs struggle to accurately and efficiently solve large-domain and multi-scale differential-equation problems as optimisation complexity and neural-network spectral bias become more challenging.
Method
FBPINNs use overlapping subdomain networks with separate input normalisation, flexible training schedules, mathematically enforced interface continuity, and parallel training.
Results
FBPINNs accurately solve the smaller, larger, and multi-scale problems studied, outperforming standard PINNs in accuracy and computational resources required.
Takeaways & Limitations
FBPINNs provide a scalable approach for applying PINNs to larger differential-equation problems while using smaller subdomain networks and parallel computation.
Takeaways & Limitations
For the studied wave-equation problem, PINN and single-threaded FBPINN training took about 10 hours on one GPU, compared with about 1 minute for finite-difference modelling on one CPU.
Abstract
from arXiv · showhide
Recently, physics-informed neural networks (PINNs) have offered a powerful new paradigm for solving problems relating to differential equations. Compared to classical numerical methods PINNs have several advantages, for example their ability to provide mesh-free solutions of differential equations and their ability to carry out forward and inverse modelling within the same optimisation problem. Whilst promising, a key limitation to date is that PINNs have struggled to accurately and efficiently solve problems with large domains and/or multi-scale solutions, which is crucial for their real-world application. Multiple significant and related factors contribute to this issue, including the increasing complexity of the underlying PINN optimisation problem as the problem size grows and the spectral bias of neural networks. In this work we propose a new, scalable approach for solving large problems relating to differential equations called Finite Basis PINNs (FBPINNs). FBPINNs are inspired by classical finite element methods, where the solution of the differential equation is expressed as the sum of a finite set of basis functions with compact support. In FBPINNs neural networks are used to learn these basis functions, which are defined over small, overlapping subdomains. FBINNs are designed to address the spectral bias of neural networks by using separate input normalisation over each subdomain, and reduce the complexity of the underlying optimisation problem by using many smaller neural networks in a parallel divide-and-conquer approach. Our numerical experiments show that FBPINNs are effective in solving both small and larger, multi-scale problems, outperforming standard PINNs in both accuracy and computational resources required, potentially paving the way to the application of PINNs on large, real-world problems.
1 Introduction
PINNs offer mesh-free forward and inverse differential-equation modelling, but their optimisation and computational limitations make large-domain and multi-scale problems difficult. FBPINNs are introduced as a domain-decomposition approach addressing these scaling issues through local networks, subdomain normalisation, and parallel training.
- PINNs provide mesh-free solutions and can solve forward and inverse differential-equation problems within one optimisation framework.
- Large-domain PINNs require larger networks and more training points, increasing optimisation complexity as solution complexity and domain size grow.
- Spectral bias makes high-frequency and multi-scale solutions difficult because neural networks learn higher frequencies more slowly than low frequencies.
- Domain decomposition replaces one large PINN optimisation problem with many smaller problems that can be trained in parallel.
- FBPINNs combine domain decomposition, separate subdomain normalisation, flexible training schedules, and parallel computation to target large, multi-scale differential-equation problems.
2 Physics informed neural networks (PINNs)
PINNs represent differential-equation solutions with neural networks trained by penalising equation and boundary-condition residuals. Strongly constrained ansätze can enforce boundary conditions by construction, whereas standard PINNs enforce them through the loss function.
- The governing problem combines a differential equation in the domain with boundary operators and boundary functions on portions of its boundary.
- PINNs approximate u(x) with a neural network and train it by minimising a loss containing physics and boundary terms.
- The physics loss promotes consistency with the differential equation, while the boundary loss matches prescribed boundary conditions and supports solution uniqueness.
- Training requires sufficient interior and boundary points so the network learns a consistent solution across the full domain.
- Weakly vs strongly constrained PINNs: A constructed ansatz can automatically satisfy boundary conditions, leaving only the physics loss and converting the optimisation problem into an unconstrained one.
3 A motivating example
A one-dimensional sinusoidal example shows that standard PINNs solve low-frequency solutions readily but struggle as frequency increases. The example motivates FBPINNs because higher frequency is equivalent to enlarging the domain after input normalisation.
- For ω = 1, a small fully connected PINN rapidly and accurately converges to the exact sinusoidal solution.The setup uses two hidden layers, 16 hidden units per layer, and 200 regularly spaced training points.
- For ω = 15, the same PINN captures only the first few cycles away from the boundary and fails to learn the solution accurately.
- Higher-frequency PINNs require more free parameters, converge more slowly, and achieve worse accuracy because of increased solution complexity, sampling demands, and spectral bias.
- After normalising inputs over the domain, increasing domain size by 15 times with ω = 1 presents the same optimisation problem as setting ω = 15.
- The motivating comparison anticipates FBPINNs solving the high-frequency case more accurately and efficiently than the studied PINNs.
4 Finite Basis PINNs (FBPINNs)
FBPINNs solve large and multi-scale differential-equation problems by decomposing the domain into overlapping subdomains, each containing a locally normalized neural network. Flexible schedules, interface continuity, and parallel training support scalable optimization while retaining a PINN-style loss.
- Overview: FBPINNs combine domain decomposition, subdomain normalization, and flexible schedules to turn one large PINN optimization problem into many smaller problems.The approach is inspired by finite element methods but uses the strong form of the governing equation rather than the weak form.
- Workflow overview: FBPINNs divide the problem domain into overlapping subdomains and place a separate neural network within each subdomain.The global solution is learned locally in subdomain centers and formed by summing network contributions in overlap regions.
- Mathematical description: Smooth differentiable window functions confine each subdomain network, while their summed contributions automatically produce a continuous global solution across interfaces.The construction avoids requiring additional interface terms in the loss.
- Parallel training: FBPINNs can train subdomain networks in parallel using independent threads, with communication required when neighboring networks share outputs in overlap regions.The method uses gradient descent and retains a loss function similar in form to that used for strongly constrained PINNs.
- Mathematical description: Separate input normalization maps each subdomain's inputs to [-1, 1], reducing the effective solution frequency seen by each local optimization problem.A common output unnormalization keeps each neural-network output within [-1, 1] and depends on the solution itself.
- Flexible training schedules: Flexible schedules update selected subdomain networks as active, fixed, or inactive models, enabling sequential training such as learning outward from a boundary.Only training points from active subdomains are required during each training step.
5 Numerical experiments
Across problems ranging from low-frequency one-dimensional equations to multi-scale and second-order cases, FBPINNs generally matched or outperformed PINNs in accuracy, convergence, and efficiency, with limitations in the hardest second-order setting.
- Experiment overview: FBPINNs accurately and efficiently solved the tested problems across increasing domain size, dimensionality, and multi-scale complexity.The experiments included smaller and larger problems, multi-scale solutions, Burgers equation, and a time-dependent wave equation.
- High-frequency case: At ω = 15, the FBPINN converged with very high accuracy in few training steps and required multiple orders of magnitude fewer forward-inference FLOPS than the PINN.The computational reduction came from using smaller networks within subdomains.
- Multi-scale case: For the multi-scale solution containing ω1 = 1 and ω2 = 15, the FBPINN reached higher accuracy in fewer training steps than the tested PINNs.The larger PINNs modeled all cycles, but their accuracy was nearly two orders of magnitude worse than the FBPINN.
- Second-order derivative case: In the second-order problem, an outward-from-the-boundary training schedule produced the best FBPINN solution, accurately modeling many cycles while retaining small edge errors.Without this schedule, both methods modeled the solution poorly away from the boundary despite learning more accurate second derivatives.
5.3 2D sinusoidal experiments
The 2D sinusoidal experiment tests FBPINNs on a high-frequency solution using 225 overlapping subdomains, where FBPINNs significantly outperform the tested PINN in accuracy and training efficiency.
- The experiment extends the motivating problem from 1D to 2D over x1, x2 ∈ [−2π, 2π].
- Results: FBPINNs significantly outperform the tested 5-layer, 128-hidden-unit PINN on the sinusoidal problem.The comparison uses L1 errors against the exact solution and convergence curves.
- Method: The high-frequency case uses ω = 15, with the FBPINN ansatz supplying the x2 dependence while networks learn the correction along x1.
- Results: The FBPINN converges to the solution with much higher accuracy and fewer training steps than the PINN.The PINN models all solution cycles but has accuracy over one order of magnitude worse.
5.4 (1+1)D Burgers equation
The experiments evaluate FBPINNs on viscous Burgers and time-dependent wave equations, including discontinuities, broadband propagation, and spatially varying wave speed. FBPINNs generally retain comparable or better accuracy while improving convergence robustness or computational efficiency.
- Burgers equation: The viscous Burgers benchmark uses ν = 0.01/π, producing a discontinuity at x = 0 as time increases.
- Burgers equation: With interfaces coinciding with the discontinuity, FBPINNs model the discontinuity but are slightly less accurate there than the PINN.Their overall convergence is more stable, while windowing and network summation make the discontinuity harder to represent locally.
- Burgers equation: Avoiding the discontinuity with six overlapping subdomains yields slightly higher overall accuracy than the PINN.
- Burgers equation: Both Burgers FBPINNs require multiple orders of magnitude fewer forward-inference FLOPS than the PINN.The reduction follows from using much smaller subdomain networks.
- Wave equation: The wave experiment models a broadband, oscillatory, dispersive point-source wave in a medium with spatially varying speed.The domain is 2D in space and time-dependent, with wave speed defined by a mixture of Gaussian distributions.
- Wave equation: Wave-equation FBPINNs and PINNs achieve similar accuracy, while FBPINNs require roughly half as many forward-inference FLOPS to train.A smaller 2-layer, 16-hidden-unit FBPINN does not converge on this problem.
- Wave equation: The time-marching FBPINN robustly learns the wave solution outward from t = 0, whereas the PINN remains close to zero after 20,000 training steps.
6 Discussion
FBPINNs scale PINNs to large and multi-scale problems through domain decomposition, separate subdomain normalisation, and flexible training schedules. They generally improve accuracy and data-efficiency, while requiring problem-specific configuration and further work for higher-dimensional and practical deployment settings.
- Numerical performance: FBPINNs outperform PINNs on larger-domain wave and high-frequency sinusoidal problems, while generally matching PINNs on smaller-domain cases.Across high-frequency tests, FBPINNs converge with higher accuracy and fewer training steps; for the wave equation, they converge more robustly.
- Computational efficiency: FBPINNs appear more data-efficient because smaller subdomain networks solve easier optimisation problems and require fewer forward-inference FLOPS during training.Total forward-inference FLOPS depends on subdomain network size rather than the number of subdomains, under the stated assumptions.
- Configuration sensitivity: Different problems require different FBPINN training schedules and network configurations to converge accurately.The Burgers equation also performs slightly worse when subdomain interfaces coincide with its solution discontinuity, so subdivision requires care.
- Open limitations: Higher-dimensional scaling remains challenging because FBPINNs retain the training-point burden of PINNs and sum more overlapping models as dimensionality increases.The authors plan to investigate FBPINN scaling to higher dimensions.
- Implementation constraints: Single-threaded FBPINNs are typically 2 to 10 times slower to train than corresponding PINNs despite being more data-efficient.Sequential subdomain updates and underutilised GPU parallelism account for this observed training-time disadvantage; a multi-threaded implementation is expected to reduce it.
- Future directions: Future work includes irregular domains, adaptive subdomain refinement, inverse problems, alternative architectures, transfer learning, and multi-threaded training.The paper also identifies combining FBPINNs with learned solution families as a possible route to broader efficiency gains.
7 Conclusions
The paper presents FBPINNs as a scalable approach for large and multi-scale differential-equation problems. The reported experiments show accurate solutions, greater data-efficiency than PINNs, and parallel-training potential, while future work targets multi-threading and adaptive refinement.
- Contribution: FBPINNs combine domain decomposition, individual subdomain normalisation, and flexible training schedules to address optimisation complexity and spectral bias.The approach is designed for large domains and multi-scale solutions.
- Results: FBPINNs accurately solve the smaller and larger-scale problems studied, including problems with multi-scale solutions.The conclusion reports this across the evaluated problem set.
- Implications: FBPINNs are more data-efficient than PINNs and can be trained in parallel, potentially improving competitiveness with finite difference and finite element methods.The conclusion frames this as potential enabled by the approach rather than an established parity result.
- Future work: Future work will study multi-threaded FBPINNs and adaptive subdomain refinement to improve accuracy and efficiency.These directions are identified as extensions of the presented framework.
A.1 Forward inference FLOPS calculation
The appendix defines forward-inference FLOPS as cumulative neural-network computation during training and derives a fully connected-network estimate from input size, output size, width, depth, and training-point count. It applies the estimate to PINNs and sums subdomain-network costs for FBPINNs, while excluding gradient and other training computations.
- Definition: Forward-inference FLOPS count cumulative floating-point operations used by neural-network forward passes during training as a measure of data-efficiency.The measure applies to evaluating NN(x; θ) for PINNs or FBPINNs.
- Network estimate: For fully connected tanh networks with linear outputs, Equation 29 estimates forward-inference FLOPS from training points, dimensions, hidden units, and hidden layers.The formula is derived by counting matrix multiplication, bias addition, and activation costs across layers.
- Application: For PINNs, Equation 29 is used directly; for FBPINNs, forward-inference FLOPS are calculated over the subdomain networks used in the training step.The subdomain contribution is represented by a sum over the active subdomain networks.
- Scope: The FLOPS measure excludes gradient computation, backpropagation, and all other parts of the training algorithm.It therefore measures only forward inference, not total training cost.
A.1.1 Scaling with network size / number of subdomains
Under fixed-domain, fixed-density, fixed-subdomain-network assumptions, FBPINN forward-inference FLOPS remain independent of the number of subdomains. Increasing overlap coverage can increase FLOPS because overlapping networks must all be evaluated.
- Scaling with subdomains: FBPINN forward-inference FLOPS depend on subdomain network size, not the number of subdomains, when domain and training-point density are fixed.As subdomains increase, the average training points per subdomain decrease at the same rate, keeping total forward FLOPS constant for fixed network size.
- Overlap effect: Forward-inference FLOPS increase when the proportion of the domain covered by overlapping regions increases.All networks active in an overlapping region must have their outputs computed.
A.2 Finite difference modelling for (2+1)D Wave equation
The (2+1)D wave equation is modeled using finite differences, with the SEISMIC_CPML library providing staggered-grid second-order acoustic-wave modeling and convolutional PML boundaries.
- Finite difference modeling is used as the ground-truth solution for the (2+1)D wave equation.
- The implementation uses the SEISMIC_CPML library, specifically its seismic_CPML_2D_pressure_second_order code.
- The code performs staggered-grid second-order finite difference modeling of the time-dependent 2D acoustic wave equation.
- Convolutional perfectly matched layer boundary conditions are applied at the domain edges.