Physics Informed Random Feature Neural Networks for Solving PDEs

1.1. Related Work.

• Random features approximate kernels with finite-dimensional maps and fixed hidden weights, avoiding dense kernel-matrix scaling while making randomized neural-network theory a motivation for this work.The related-work discussion says randomized neural networks offer simple implementation and efficient training, while their theoretical guarantees remain largely unexplored.Source: §1 · p. 2, §1.1 · p. 3

1. Introduction.

• PIRFNs replace trainable PINN hidden layers with kernel-induced random features and optimize only output coefficients, simplifying optimization while retaining RKHS-based approximation guarantees.The training redesign uses PINN-style automatic differentiation with Adam and L-BFGS, extending approximation analysis to the Sobolev H1 norm.Source: §1 · p. 2
• Appropriately designed random feature distributions improve approximation accuracy on representative PDEs, including Helmholtz, high-wave-speed linear advection, and wave equations.The paper identifies kernel design as central to performance and notes that automatic kernel selection remains future work.Source: §1 · p. 2
• Product random features let spatial and temporal variables use different kernels and sampling distributions, accommodating their distinct characteristic scales in time-dependent PDEs.The construction is motivated by product kernels.Source: §1 · p. 2
• The paper reports convergence studies and comparisons with PINNs, self-adaptive PINNs, and Extreme Learning Machines across representative PDE benchmarks.The introduction describes these studies as demonstrating the framework's effectiveness across different kinds of PDEs.Source: §1 · p. 3

1.2. Notation.

• The notation section defines real and complex numbers, vector and matrix conventions, L2 norms, and Sobolev spaces, including Hs(X) as W s,2(X).It also names Hs_mix(X) for the Sobolev space of dominating mixed smoothness.Source: §1.1 · p. 3, §1.2 · pp. 3–4

2.1. Reproducing Kernel Hilbert Space and Kernel Method.

• An RKHS makes point evaluation a bounded linear functional, and its reproducing kernel represents evaluations through inner products in the function space.The canonical feature map identifies the kernel value with an inner product of feature vectors.Source: §2.1 · p. 4
• Kernel methods model nonlinear relationships through linear operations in feature spaces, but large kernel matrices require O(m^3) training time and O(m^2) storage.Random features approximate the kernel matrix with a low-rank representation to reduce these costs.Source: §2.1 · p. 5, §2.2 · p. 6

2.2. Random Feature.

• Random Fourier features approximate translation-invariant kernels by sampling from their spectral distribution and forming a finite-dimensional feature map.The construction follows by applying Monte Carlo sampling to the kernel’s integral representation.Source: §2.2 · pp. 6–7
• Random feature networks fix randomly sampled hidden-layer weights and biases while learning only output coefficients, reducing trainable parameters and yielding a convex optimization problem.The paper describes this optimization as often reducible to linear regression.Source: §2.2 · p. 8

2.3. Physics-Informed Neural Networks (PINNs).

• PINNs approximate PDE solutions with neural networks trained using losses for the PDE residual and initial and boundary conditions at collocation points.The collocation points are randomly sampled in the respective PDE, initial-condition, and boundary-condition domains.Source: §2.2 · p. 8, §2.3 · pp. 8–9

3.1. Physics-informed Random Feature Network.

• The PDE solution is modeled as a finite random-feature sum, with the paper assuming a pointwise well-defined PDE and a unique strong solution.For linear PDEs, the constraints reduce to a constrained min-norm problem with solution c = A†y; nonlinear PDE and boundary conditions enter as soft constraints.Source: §3.1 · p. 9
• Product random features separately sample spatial and temporal components, allowing different feature counts and distributions when variables have distinct scales.The construction is motivated by product kernels and can also assign different features to spatial directions with different scales.Source: §3.1 · p. 10

3.2. Universal Approximation.

• The universal-approximation result gives high-probability random-feature approximation for functions in the stated class, with uniform approximation when the feature has a Lipschitz form.The theorem states the approximation for every function in the specified compact family; the supplied passage does not include its quantitative bound.Source: §3.2 · p. 11

3.3. Approximation Error.

• The approximation analysis extends random-feature error bounds to H1 for functions in F(ρ) ∩ eF(ρ), assuming bounded features and a high-probability event.The derivation combines an L2 error bound with a bound on the gradient error.Source: §3.3 · pp. 12, 14
• For Gaussian random weights, the paper derives an explicit BN(δ) tail bound because Gaussian features are used in its numerical experiments.The general theorem’s good event and BN(δ) depend on the random-feature choice.Source: §3.3 · p. 14

4.1. Measure of Accuracy.

• The experiments compare methods using relative L2 and L∞ errors estimated on withheld test points; decay-rate tests also use L2 and H1 errors.The norms are estimated by Monte Carlo evaluation.Source: §4.1 · p. 15

4.2. Training Strategy.

• PIRFN coefficients use Adam followed by L-BFGS, while comparison networks use the same two-stage procedure and a comparable feature count.Results average 10 trials with different randomly sampled training points and fixed test points.Source: §4.2 · pp. 15–16

4.3. Error Decay Rate.

• Across dimensions d = 2, 4, and 8, estimated L2 and H1 error decay rates are consistent with those established in Theorems 3.3 and 3.4.Figure 1 plots both errors against the number of features.Source: §4.3 · p. 16, Figure 1 · p. 17

4.4. Linear Advection-Diffusion Equation.

• On the advection-diffusion equation, product random features achieve substantially lower RMSE and L∞ errors than uniform random features.The authors attribute the difference to accommodating distinct spatial and temporal scales through different sampling distributions.Source: Table 1 · p. 17, §4.4 · p. 17

4.5. Helmholtz Equation.

• PIRFN achieves substantially lower relative L2 and L∞ errors than the comparison methods on the 2D Helmholtz equation.The reported results are averaged over 10 independent trials, and the approximation captures the oscillatory solution.Source: §4.5 · p. 18, Figure 2 · p. 18, Table 3 · p. 19

4.6. Linear Transport.

• For the linear transport test at c = 12, the random-feature method captures wave propagation and achieves lower errors than PINN and its variant.The authors describe this as a challenging regime for learning methods.Source: Table 5 · p. 19, §4.6 · p. 19, Figure 3 · p. 20

4.7. Wave Equation.

• PIRFN’s relative L2 error is on the order of 10^-2 versus 10^-1 for other methods, and it has the smallest L∞ error; Figure 4 shows accurate space-time evolution.The wave-equation test includes multiple spatial and temporal frequencies.Source: §4.7 · pp. 20–21, Table 7 · p. 21, Figure 4 · p. 21

• On Helmholtz, linear transport, and wave benchmarks, PIRFN achieved lower errors than tested PINN, self-adaptive PINN, and ELM configurations.Source: §5 · p. 21
• For advection–diffusion at the same total feature count, product representation improved accuracy over uniform random features.Source: §5 · p. 21
• Nonlinear Poisson tests in dimensions 2, 4, and 8 showed error decay consistent with theoretical estimates.Source: §5 · p. 21
• The findings highlight matching random-feature distributions to solution scales and support further study of oscillatory and multiscale PDEs.Source: §5 · p. 21
• A fuller accuracy analysis would account for finite collocation sampling, optimization error, and PDE stability; kernel and sampling choices remain problem-dependent.Source: §5 · pp. 21–22

• Theorem A.1 states a vector-valued Bernstein inequality for random variables in a separable Hilbert space, with positive parameters b and σ.Source: Appendix A · p. 22
• The appendix also presents a frequently used bounded-random-variable setting for Bernstein’s inequality.Source: Appendix A · p. 22
• Under the stated moment condition, Lemma A.2 gives b = 2c and σ = v, while zero mean permits b = c.Source: Appendix A · p. 22