Spectral convergence of random feature method in one dimension

1. Introduction

• Deep learning PDE solvers are meshfree and flexible for complex geometries and high-dimensional problems, but face high computational costs and inconsistent accuracy improvement.Source: §1 · p. 1
• RFM fixes inner-layer weights and reduces training to linear least squares for outer-layer weights, while PUM is used to enhance local accuracy and resolve intricate solution structures.These approaches are associated with exponential or near-exponential convergence as feature count increases and patch size is refined.Source: §1 · pp. 1–2
• Although empirical spectral convergence has been observed, a rigorous proof is lacking, and the singular-value decay underlying RFM matrix ill-conditioning remains unexplained.The paper frames its analysis around these unresolved spectral-accuracy and conditioning questions.Source: §1 · p. 2

1.1. Our contributions.

• The paper establishes RFM approximation bounds in W^σ,p and derives PDE convergence rates ranging from super-exponential to algebraic according to solution regularity.The bounds clarify dependence on weight distribution, solution regularity, and domain size.Source: §1.1 · p. 2

• PUM gives error decay O(r^α), with α scaling as Nloc^-σ for Gevrey targets and min(Nloc^-σ, s^-σ) for Sobolev targets.Source: §1 · p. 2

3. Singular-value decay of RFMtx.

• For bounded hidden weights and collocation points, the m-th singular value decays as nN(SR)^(m−3)/(m−3)!, while the condition number grows exponentially with min(n, 2N).The analysis attributes ill-conditioning to spectral approximation among features.Source: §3 · p. 2
• The global matrix's m-th singular value is closely approximated by the ⌈m/(number of patches)⌉-th singular value of a sub-block, explaining how PUM mitigates excessive decay.Source: §3 · p. 2

1.2. Related work.

• Prior work often focuses on high-dimensional settings with rates rarely exceeding 1/N, whereas this paper gives explicit log-term-free spectral convergence rates for Gevrey and Sobolev targets under general prescribed hidden-weight distributions.The passage contrasts this analysis with approaches that constructively select parameters or tune sampling for unbiased kernel estimation.Source: §1.2 · p. 3
• Earlier conditioning analyses report exponential ill-conditioning for uniform-grid sampling and good conditioning in some Gaussian-sampling regimes, while bounded-domain sampling can produce exponential or super-exponential singular-value decay.The cited comparison includes deterioration at the interpolation threshold for Gaussian sampling.Source: §1.2 · pp. 3–4

1.3. Notation and preliminaries.

• The Gevrey class G^s comprises smooth functions whose derivative growth is bounded by (n!)^s; G^1 is the real-analytic function space.Source: §1.3 · p. 4

2.1. Spectral convergence with respect to the number of features.

• RFM uses random Fourier features with hidden weights sampled from distributions satisfying Assumption 2.1.The feature count scale depends on max(C_u, μ), which reflects approximation difficulty; choosing μ too large or too small can respectively add excessive high-frequency features or fail to capture high-frequency solution components.Source: §2.1 · pp. 4–5
• Smoother target functions admit RFM convergence rates ranging from algebraic to super-exponential, under the stated Gevrey-class assumptions.The section derives these bounds for Sobolev norms and relates them to convergence rates for PDE approximation.Source: §2.1 · p. 6

2.2. Convergence rates with respect to the patch size.

• With PUM, local random feature approximants are assembled using a partition of unity, and the Gevrey error bound depends on patch size r_j.Under Theorem 2.9's assumptions, the stated rate is O(r^(N_loc+1/p−m−1)|ln r|^(N_loc−1)); bounded hidden weights improve it by one order.Source: §2.2 · pp. 6–7
• The PUM bound's local factor A_j motivates smaller patches where the solution's local bound is large, to equilibrate and reduce error.A_j depends on the local solution bound, sampling parameter, and patch size.Source: §2.2 · p. 7
• For quasi-uniform patches, the Sobolev-space corollary gives a rate approaching s−σ as N_loc grows and N_loc−σ−1+1/p as s grows.The corollary permits t=0, corresponding to a fixed sampling law for local hidden weights during patch refinement.Source: §2.2 · p. 7

3.1. Abstract error estimates for RFM solvers.

• The abstract RF error estimates depend on PDE stability and trial-space approximation capacity, not on the particular random-feature realization.Taking expectations and applying the paper’s approximation results yields the final error estimates.Source: §3.1 · p. 7

3.1.1. Strong-form RFM.

• Under the stated assumptions, the strong-form RF solution’s H2 error is bounded by its trial-space best-approximation error when the solution is unique.The result also accommodates general boundary conditions and problems with a non-trivial null space, including the one-dimensional Neumann Laplace equation.Source: §3.1.1 · p. 8

3.1.2. Weak-form RFM.

• For a continuous, coercive bilinear form, the weak-form RF solution satisfies a Céa-type bound by the best approximation in the RF trial space.The weak problem has a unique solution by Lax–Milgram.Source: §3.1.2 · p. 8

3.1.3. RFM for Eigenvalue Problems.

• For sufficiently rich RF trial spaces, exactly q discrete eigenvalues converge to an eigenvalue of multiplicity q, with eigenvalue error bounded by the square of the eigenspace approximation error.The eigenspace convergence rate matches the best-approximation error, while eigenvalues converge at twice that rate.Source: §3.1.3 · p. 9

3.2. Fast singular-value decay and exponential ill-conditioning of RFMtx.

• For smooth activations, singular-value decay depends on their approximation by Taylor polynomials; trigonometric features can decay exponentially or super-exponentially, and tanh decays exponentially when SR < π/2.The analysis also establishes fast decay for matrices involving derivatives of random features.Source: §3.2 · p. 10
• PUM relates global-matrix singular values to submatrix singular values, with the m-th value approximated by the ⌈m/J⌉-th submatrix value, reducing log-scale decay by slightly less than the patch count.The practical overlapping-patch bounds have a lower bound that captures the decay trend but is loose when overlap-region derivatives make adjacent-block entries dominate.Source: §3.2 · p. 11

4.1. Proofs of spectral convergence with respect to feature count.

• The feature-count proof matches a Taylor expansion through degree 2N −1 using Vandermonde systems, then controls the coefficients and approximation error on a high-probability event.The proof combines coefficient bounds, event estimates, and feature polynomial-approximation bounds.Source: §4.1 · pp. 11–12
• For Gevrey and Sobolev targets, the proof smooths or truncates the function, approximates that intermediate function with RFM, and uses interpolation and scaling to extend the estimates.The construction separates smoothing error from feature-approximation error.Source: §4.1 · pp. 14–16

4.2. Proofs of convergence rates with respect to patch size.

• PUM controls global approximation error through local patch errors, to which the feature-count approximation bound is applied.The proof uses a global error decomposition and estimates the local error on each patch.Source: §4.2 · p. 16, §4.1 · p. 16
• For patch-size estimates in Sobolev spaces, the proof smooths the target, selects a smoothing scale from the patch size, and combines local estimates with interpolation.The smoothing construction controls its Gevrey constant and the resulting local scaling factors.Source: §4.2 · p. 17

5.1. Proofs of abstract error estimates.

• The abstract error-estimate proof uses a Fredholm operator argument and a bounded inverse to control the solution error.It applies the resulting estimate to the difference between the approximate and exact solutions.Source: §5.1 · p. 18

5.2. Proofs of ill-conditioning and PUM mitigation.

• The singular-value proof constructs linearly independent coefficient vectors whose feature combinations have small responses, then applies the minimax principle.The construction approximates cosine features by Maclaurin polynomials and matches their coefficients.Source: §5.2 · pp. 18–19
• The condition-number lower-bound argument reduces to bounding a random variable ρ and derives a lower bound through separate cases.The proof distinguishes B ≤ 0 from B > 0 when establishing the bound for ρ.Source: §5.2 · pp. 19–20

• Under the stated uniform sampling setup, the proof obtains P(ρ ≤ 1 −1/δ) ≤0.95N using a choice of λ that controls the expectation term.The proof sets λ = 1/[8(δ2 −1)].Source: §5 · p. 20
• The PUM singular-value argument partitions patches by parity and uses Weyl’s inequality to bound singular values through the odd- and even-index blocks.Within each parity class, the patches do not overlap, so the corresponding matrices share singular values with their diagonal blocks.Source: §5 · p. 20

6. Conclusion and outlook

• The paper’s approximation and conditioning arguments extend to analytic activation functions when their Taylor series and derivatives satisfy stated conditions.The spectral analysis treats one feature as the target approximated by other features.Source: §6 · p. 20
• Higher-dimensional convergence rates remain conjectural; the paper notes numerical observations in dimensions 2 and 3.The proposed rates scale with N^(1/d), rather than N.Source: §6 · pp. 20–21

Appendix A. Bounds for the Gamma function and sampling constants

• For ν > 2, the appendix states that f(ν) is strictly decreasing and g(ν) is strictly increasing, alongside a two-sided Gamma-function bound.The monotonicity proof uses properties of the digamma function.Source: Appendix A · p. 21

Appendix B. Proof of Lemma 4.2 and Lemma 4.3

• The Lemma 4.2 proof uses the moment method, splitting E|Y|^λ into three parts and treating k ≥ 1/η and k < 1/η separately.The proof notes that Y becomes large as X approaches k.Source: Appendix B · p. 22
• For k ≥ 1/η and k ≤ 1/η, the proof derives separate bounds and combines them to establish the desired estimate.The estimates use convexity, monotonicity, Jensen’s inequality, and a split of the interval when k ≤ 1/η.Source: Appendix B · pp. 22–23
• The Lemma 4.3 proof bounds two k regimes—k > 4/3 and k ≤ 4/3—and combines them to obtain the desired estimate.For the smaller regime, it uses monotonicity of four expressions in k.Source: Appendix B · p. 24

Appendix C. Properties, Cutoffs, and Fourier Decay of Gevrey Functions

• The Gevrey-function estimates are developed in d dimensions because the authors consider them independently interesting.Source: Appendix C · p. 24

C.1. Basic calculus and examples.

• Gevrey classes are closed under linear combinations, multiplication, and differentiation, with bounds on the associated constants.Examples in G0 include trigonometric and exponential functions, monomials, and bandlimited functions.Source: §C.1 · p. 24

C.2. Gevrey cutoff construction via Hörmander mollifiers.

• Hörmander-mollifier cutoffs equal one on [−R, R]^d, are supported in [−R−ρ, R+ρ]^d, and have explicit derivative bounds.The bound is sup_x |D^αφ(x)| ≤ (2ζ_s/ρ)^|α| (α!)^s.Source: §C.2 · p. 25

C.3. Fourier decay of compactly supported Gevrey functions.

• Compactly supported Gevrey functions have subexponential Fourier decay, and cutoff extension gives the same form of bound for functions on a domain.The extension bound uses constants depending on the Gevrey parameters, cutoff width, and domain size.Source: §C.3 · pp. 25–26

Appendix D. Gevrey approximation of Sobolev functions

• For u in W^s,p, Theorem D.1 constructs a G0 approximant, and the appendix describes explicit error estimates involving Gevrey constants, bandwidth, and approximant norms.The construction convolves with a bandlimited mollifier; bounded Lipschitz domains are handled through a bounded extension operator.Source: Appendix D · pp. 26, 29
• For non-integer s, the proof controls both remainder terms and establishes the estimate in (D.8).The bound for the second remainder has the right-hand side of (D.9), with ε^s replaced by ε^(s+d).Source: Appendix D · p. 28

Appendix E. Auxiliary estimates for RFMtx conditioning

• Numerical integration gives J(α) an upper bound of 0.72, implying P{ρ ≤ 1/11} ≤ 0.72^N in the proof of Theorem 3.4.Source: Appendix E · p. 30