Source-linked AI summary
Functional Tensor Decompositions for Physics-Informed Neural Networks
Sai Karthikeya Vemuri, Tim Büchner, Julia Niebling, Joachim Denzler
TL;DR
PINNs face exponentially growing collocation demands as PDE dimension rises. This paper combines separate neural networks through CP, TT, and Tucker functional tensor decompositions, reporting improved accuracy and speed on benchmark PDEs, including tests above three dimensions.
Problem
As PDE dimension grows, classical PINNs face an exponentially expanding solution space and collocation demands that make accurate solutions computationally difficult.
Source: §1 · p. 2, §2.2 · p. 6Method
The paper uses functional tensor decompositions—CP, Tensor-Train, and Tucker—with separate neural networks learning components that combine through outer products.
Source: §2.3 · p. 7, Functional tensor decomposition forms… · p. 9Results
The proposed methods significantly outperform existing PINNs on benchmark PDEs, including higher-dimensional cases, even with low collocation counts and ranks, and the authors report faster speed.
Source: §4 · p. 13Takeaways & Limitations
The authors conclude these representations can mitigate the dimensionality challenge and support efficient PINN solutions to complex, higher-dimensional PDEs.
Source: §4 · pp. 12–13No decomposition is best: performance depends on collocation points, rank, and physical knowledge; higher-rank CP can be unstable, while Tucker's core tensor retains dimensionality costs.
Source: §4 · p. 13
Abstract
from arXiv · showhide
Physics-Informed Neural Networks (PINNs) have shown continuous and increasing promise in approximating partial differential equations (PDEs), although they remain constrained by the curse of dimensionality. In this paper, we propose a generalized PINN version of the classical variable separable method. To do this, we first show that, using the universal approximation theorem, a multivariate function can be approximated by the outer product of neural networks, whose inputs are separated variables. We leverage tensor decomposition forms to separate the variables in a PINN setting. By employing Canonic Polyadic (CP), Tensor-Train (TT), and Tucker decomposition forms within the PINN framework, we create robust architectures for learning multivariate functions from separate neural networks connected by outer products. Our methodology significantly enhances the performance of PINNs, as evidenced by improved results on complex high-dimensional PDEs, including the 3d Helmholtz and 5d Poisson equations, among others. This research underscores the potential of tensor decomposition-based variably separated PINNs to surpass the state-of-the-art, offering a compelling solution to the dimensionality challenge in PDE approximation.
1 Introduction
PINNs face exponentially growing solution-space and collocation demands as PDE dimension rises. The paper extends variable separation with neural-network-based tensor decompositions and reports improved accuracy and fewer collocation points on high-dimensional PDEs.
1 Introduction
- Classical PINNs struggle as PDE dimension grows because the solution space scales as n^d, creating computational difficulties and potentially erroneous solutions.The passage describes needing n collocation points for accuracy, with the solution space then growing as n^d.Source: §1 · p. 2
- The proposed PINN alternative generalizes variable separation to arbitrary multidimensional PDEs, using individual neural networks to learn information along each dimension.Unlike classical variable separation, the proposal does not require a separable form.Source: §1 · p. 2
- Earlier work combined multiple networks in PINNs; Separable PINNs assign separate networks per axis, reduce collocation points, and use forward-mode automatic differentiation to reduce computation and memory.Source: §1 · p. 2
- The authors extend the universal approximation theorem to show that outer products of univariate neural networks can approximate multivariate functions, even without a variable-separable form.Source: §1 · p. 2
- The proposed methods outperform previous state-of-the-art PINN architectures for high-dimensional PDEs and require fewer collocation points.The introduction states this result without giving quantitative comparisons.Source: §1 · p. 3
- The proposed functional tensor architectures use Canonic-Polyadic (CP-PINN), Tensor-Train (TT-PINN), and Tucker decompositions.The authors describe these as tensor decomposition forms combined with PINNs.Source: §1 · p. 3
2 Theoretical Background
The background extends variable separation through neural-network tensor decompositions, establishing how outer products of univariate networks can approximate multivariate functions. It frames this approach against PINNs’ dimensionality challenges and introduces CP-, TT-, and Tucker-based architectures.
2.1 Universal Approximation Theorem
- With sufficient rank and controlled component errors and norms, outer products of univariate neural networks can approximate continuous multivariate functions to any desired accuracy.The function is considered on a compact bounded domain.Source: §2.1 · pp. 3, 5
2.2 Physics-Informed Neural Networks
- PINNs combine data and physics losses, but collocation-point counts grow exponentially with dimension, making computation expensive and solution approximation harder.The combined loss weights the physics term with λ.Source: §2.2 · p. 6
2.1 Universal Approximation Theorem
- The classical variable-separable method applies only to limited classes, while the proposed tensor-decomposition approach is not restricted to PDEs with an existing separable form.The paper presents outer products of neural networks as the representational basis for this broader scope.Source: §2 · p. 3, §2.2 · p. 6
2.3 Functional Tensor Decompositions for PINNs
- CP, TT, and Tucker functional decompositions separate variables, with individual neural networks learning each decomposition component.For this paper, Tucker- and TT-PINN component ranks share integer R; Tucker also has a trainable core tensor.Source: §2.3 · p. 7, Functional tensor decomposition forms… · p. 9
3 Experiments
The experiments compare the tensor-decomposition PINNs with existing architectures on three-dimensional PDEs and test their use on higher-dimensional problems. The proposed models show stronger accuracy with fewer collocation points in the reported benchmarks, while results vary by decomposition and task.
3 Experiments
- Each variable is modeled by a four-layer tanh network, and layer feature depth corresponds to decomposition rank unless specified otherwise.The experiments compare against the original PINN and other state-of-the-art architectures on the three-dimensional benchmarks.Source: §3 · p. 9
- Training uses the conventional composite data-and-PDE-residual loss with λ = 1, Adam at learning rate 1e−3 for 50,000 iterations, and L2 error evaluation.Reported speeds are iterations per second on an NVIDIA GeForce GTX 1080 GPU.Source: §3 · p. 9
- A factor of 10 higher accuracy and fewer collocation points are reported for the tensor-decomposition PINNs on the Klein–Gordon and Helmholtz benchmarks.The table caption describes the improvement as a magnitude of 10.Source: Table 2 · p. 11, §3 · p. 11
- For both three-dimensional PDEs, CP-PINN with rank 32 and 64 × 3 collocation points is most accurate, with TT-PINN second best; larger ranks and point counts generally improve accuracy.The table caption says the best and second-best scores are marked separately.Source: §3 · p. 11, Table 2 · p. 11
- In these experiments, for (2+1)d flow mixing, with collocation-point amount fixed while decomposition rank varied, Tucker-PINN outperformed CP-PINN and TT-PINN; proposed architectures solved the problem only at rank 64.Source: §3 · p. 12
4 Discussion and Conclusions
The paper presents functional tensor-decomposition PINNs as a controlled-rank approach to multivariate PDEs, using separate neural networks without requiring a unique separable form. It reports improved performance on benchmark and higher-dimensional PDEs, while noting that no decomposition form is best in all settings.
4 Discussion and Conclusions
- The method represents a multivariate PDE solution with outer products of smaller, controlled-rank tensors learned by individual neural networks, then optimizes a PINN loss using PDE residuals and boundary conditions.It investigates three tensor-decomposition forms to reduce computational complexity.Source: §4 · p. 12
- A unique decomposition need not exist because the decomposition is estimated in a data-driven manner.Source: §4 · p. 13
- The experiments report outperforming existing methods by a factor of ten, with results spanning benchmark PDEs and higher-dimensional tests including 5d Poisson and (2+1)d flow mixing.The reported benchmarks include the Klein-Gordon and 3d Helmholtz equations.Source: §4 · p. 13
- No tensor-decomposition form is best across settings; performance depends on collocation points, rank, and available physical knowledge.Source: §4 · p. 13
- CP is straightforward at a given rank but becomes unstable at higher ranks; Tucker's core retains a milder dimensionality curse, while TT is proposed as a lower-dimensional, more stable candidate.The paper describes TT as a candidate rather than establishing it as universally preferable.Source: §4 · p. 13