PaperlayerView exact revision on arXiv

Source-linked AI summary

Spectral Convergence of Random Feature Method in Multiple Dimensions

Pingbing Ming, Hao Yu

arXiv:2609.03401v1math.NAcs.AIcs.LGmath.ST

TL;DR

Multidimensional RFM convergence theory must address multiple questions together, while high-accuracy RFM matrix computations face severe ill-conditioning. The paper develops high-probability approximation estimates and proves rates from super-exponential to algebraic, depending on target regularity.

  • Problem

    Multidimensional RFM convergence theory must address multiple questions together, while high-accuracy RFM matrix computations face severe ill-conditioning.

    Source: §1 · p. 1

  • Method

    The paper combines empirical-operator concentration, an interpolation-scale estimate, and effective-dimension bounds to obtain target-independent high-probability approximation across admissible error norms.

    Source: §1 · p. 2

  • Results

    Rates range from super-exponential to algebraic with target regularity under both sampling schemes, on one event approximating an entire source ball with one norm-independent coefficient vector per target.

    Source: §6 · p. 17

  • Takeaways & Limitations

    The approximation theory supplies a low-dimensional mechanism for singular-value decay by approximating feature columns in a common space.

    Source: §6 · p. 18

    The Fourier-kernel analysis requires the support assumption on τ_s, since without it injectivity of Σ may fail.

    Source: §3.1 · p. 8

Abstract

from arXiv · show

We first prove spectral convergence of the random feature method (RFM) for multidimensional targets in Sobolev, Gevrey, ultra-analytic, and bandlimited classes. The analysis establishes general high-probability approximation estimates in the interpolation scale generated by a kernel integral operator. On a single event determined only by the sampled features, one random space approximates every target in a prescribed source ball; moreover, for each target, a single coefficient vector defines an approximant that attains spectral accuracy simultaneously in all admissible error norms. For both regularity-adapted frequency distributions and uniform distributions on growing frequency windows, the resulting rates range from super-exponential to algebraic, depending on the regularity of the target. Second, we establish abstract error estimates for strong- and weak-form RFM discretizations, thereby converting the preceding approximation bounds into convergence estimates for multidimensional second-order elliptic boundary value and eigenvalue problems. Finally, for random feature matrices (RFMtxs), we prove super-exponential singular-value decay with Fourier features and exponential decay with $\tanh$ features, together with corresponding condition-number lower bounds. The analysis identifies a common mechanism: the same spectral approximation that yields high accuracy also drives severe ill-conditioning.

1. Introduction

The paper develops a multidimensional, operator-theoretic theory of random feature approximation with target-uniform, high-probability control and one reconstruction that works across admissible error norms. It applies this framework to spectral rates, elliptic PDE and eigenvalue solvers, and the singular-value decay underlying ill-conditioning.

1. Introduction.

  • Prior RFM work reports spectral accuracy on complex geometries, but learning-based PDE solvers still face high computational cost and severe matrix ill-conditioning.The paper’s analysis addresses the approximation, stability, and conditioning mechanisms behind these linear least-squares discretizations.Source: §1 · p. 1, §1.2 · p. 3

1.1. Our contributions.

  • A single feature-dependent high-probability event gives every target in a source class a spectrally accurate approximant, while one coefficient vector per target works across admissible error norms.The interpolation-scale estimate and effective-dimension bounds preserve both forms of uniformity; stronger error norms can require stronger sampling conditions.Source: §1.1 · pp. 2–3, §1 · p. 2
  • Sobolev targets yield algebraic rates, Gevrey targets stretched-exponential rates, and ultra-analytic or bandlimited targets super-exponential rates under regularity-adapted or growing-window frequency sampling.The estimates also hold simultaneously in Sobolev norms W^t,p(Ω), 1 ≤ p ≤ ∞, not only in L2.Source: §1.1 · pp. 2–3
  • The approximation estimates yield error bounds for strong- and weak-form RFM discretizations of elliptic boundary value problems and for their eigenspaces and eigenvalues.The analysis combines elliptic regularity, strong-form stability, Céa’s lemma, and compact-operator spectral approximation.Source: §1.1 · p. 3
  • Fourier-feature RFM matrices have super-exponential high-index singular-value decay, while tanh-feature matrices have exponential decay, with corresponding condition-number lower bounds.The paper links these conditioning results to the low-dimensional approximation of bounded smooth features.Source: §1.1 · p. 3, §1.2 · p. 4

1.2. Related work.

  • Unlike earlier randomized approximation results with fixed norms, algebraic rates, or one-dimensional expectation bounds, this work combines target-uniform high-probability control, norm-independent reconstruction, and spectral convergence across an admissible norm scale.Its multidimensional operator-theoretic framework also covers elliptic problems and singular-value decay.Source: §1.1 · p. 3, §1.2 · p. 4

2. Approximation in kernel interpolation spaces

The section builds a kernel-operator interpolation scale and derives high-probability random-feature ridge approximation and coefficient bounds, with error behavior governed by source, regression, and error regularities.

2.1. RKHS, interpolation spaces and feature representation.

  • Kernel-operator spectral powers define spaces Ha, with H0=L2, H1=Hk, continuous embeddings Ha→Hb for a≥b, and Ha=(L2,Hk)a,2 for 0<a<1.The feature operator T satisfies Σ=TT*, linking the random-feature representation to the operator that generates the scale.Source: §2.1 · pp. 5–6

2.2. Ridge approximation in Hθ.

  • The ridge problem samples N features and fits targets u∈Hs in Hθ while measuring error in Hp; normalized leverage-score sampling attains the defined effective-dimension bound.The admissibility threshold specifies how many sampled features are needed for a given penalty, confidence level, and sampling density.Source: §2.2 · pp. 6–7

2.3. Main abstract estimate and its interpretation.

  • With probability at least 1−δ, Theorem 2.3 bounds ridge approximation error and coefficient norm on a feature-only event uniform over the unit ball of Hs.For p≤θ, one event controls all admissible p; error rates saturate below 2θ−1, while p≥θ requires a penalty threshold depending on 1−p.Source: §2.3 · p. 7

3. Uniform approximation of regularity classes by random Fourier features

The section establishes high-probability spectral approximation for regularity classes using either regularity-adapted or growing-bandwidth uniform frequency measures. It relates these rate regimes to spatial smoothness and identifies when the kernel operator is injective.

3.1. Two equivalent forms of random Fourier features.

  • Symmetric Fourier features and randomly shifted cosine features induce the same kernel, whose integral operator is injective when the symmetrized frequency support has nonempty interior.The support condition is essential: finite discrete frequency measures can yield finite-rank kernels and a nontrivial null space.Source: §3.1 · p. 8

3.2. Approximation with regularity-adapted reference measures.

  • With probability at least 1 − δ, one sampled feature space approximates every target in its prescribed class, and one coefficient vector per target attains all admissible error-norm bounds.Regularity-adapted reference measures yield algebraic Sobolev and stretched-exponential Fourier rates; super-exponential Fourier and bandlimited classes attain super-exponential rates.Source: §3.2 · pp. 9–10
  • Super-exponential Fourier and bandlimited target classes achieve super-exponential approximation rates with regularity-adapted measures.The theorem gives these rates alongside the algebraic and stretched-exponential regimes for less rapidly decaying Fourier profiles.Source: §3.2 · pp. 9–10

3.3. Approximation with growing-bandwidth uniform reference measures.

  • A uniform frequency cube with leverage-score sampling matches the regularity-dependent rates while using a target-uniform sampling event and a single reconstruction for all admissible W^t,p norms.For Sobolev and stretched-exponential Fourier classes, bandwidth growth is regularity-independent; the faster super-exponential regime uses slower, narrower-window growth.Source: §3.3 · pp. 11–12

3.4. Spatial interpretation of the regularity conditions.

  • Spatial Gevrey and analytic regularity imply stretched-exponential and exponential Fourier weights, while super-exponential decay corresponds to a Gelfand–Shilov characterization and the approximation rates follow the hierarchy of Fourier decay.The Gelfand–Shilov equivalence also requires matching physical-space decay; converting pointwise bounds to weighted L2 may require a strictly smaller exponent.Source: §3.4 · pp. 12–13

4. RFM discretizations of PDE and eigenvalue problems

The section gives abstract error bounds for RFM discretizations in terms of PDE stability and trial-space approximation, then applies them to strong-form, weak-form, and eigenvalue problems. The bounds do not depend on the particular random-feature realization.

4.1. Strong-form RFM.

  • The strong-form loss evaluates the boundary residual in Hs(∂Ω), a stronger trace-space control that may enhance PDE approximation stability and accuracy [36] [80].The formulation includes a boundary penalty parameter γ > 0 and assumes g ∈ Hs(∂Ω).Source: §4.1 · p. 14
  • For a unique strong-form solution, ∥u − uRF∥H^{s+l+1/2}(Ω) ≤ C inf_w∈VRF ∥u − w∥H2(Ω), for 0 ≤ s ≤ 3/2−l.The theorem assumes a smooth domain, a properly elliptic operator with smooth coefficients, and VRF ⊂ H2(Ω). It also accommodates general boundary conditions and nontrivial null spaces, including the Neumann Laplace problem.Source: §4.1 · p. 14

4.2. Weak-form RFM.

  • Under Assumption (4.4), the weak-form RFM error in V is bounded by M/α times the best VRF approximation error.The result follows for a continuous, coercive bilinear form; the Neumann example uses V = H1(Ω).Source: §4.2 · p. 15

4.3. RFM for Eigenvalue Problems.

  • For sufficiently rich VRF, exactly q discrete eigenvalues converge to an eigenvalue of multiplicity q, with eigenvalue error bounded by Cη_RF(E)^2.The corresponding eigenspace convergence matches the best approximation error, while eigenvalues converge at twice that rate.Source: §4.3 · p. 16

5. Singular values and exponential ill-conditioning of RFMtxs

The section establishes fast singular-value decay for Fourier- and tanh-feature RFMtxs and lower bounds on their condition numbers. It identifies the decay as a consequence of spectral approximation that also makes the least-squares systems severely ill-conditioned.

5. Singular values and exponential ill-conditioning of RFMtxs

  • The RFMtx has one feature per column and one collocation point per row, and its trainable coefficients are computed by solving a least-squares system.The analysis sets γ = 1, assumes n > 2N, and uses a scaling argument when the domain is not contained in (−1, 1)^d.Source: §5 · p. 16
  • Fourier-feature RFMtxs have super-exponential singular-value decay, while tanh-feature RFMtxs have exponential decay.The bounds apply to frequencies in [−S, S]^d and indices satisfying the theorem’s stated threshold.Source: §5 · pp. 16–17
  • Theorem 5.2 states condition-number lower bounds assuming −ξ⊤A(x)ξ ≥ a|ξ|² and c(x) ≥ 0, with M sufficiently large in terms of d and S and 2N−1 ≥ 3M ln(15M).Source: §5 · p. 16
  • Simultaneous approximation of feature columns in a common low-dimensional trial space, followed by the singular-value min–max principle, yields the decay bounds and links spectral accuracy to severe ill-conditioning.The resulting residuals are exponentially or super-exponentially small, creating an obstacle to stable high-precision least-squares solutions.Source: §5 · p. 17
  • Numerical experiments suggest slower singular-value decay as dimension d increases, with rank deficiency most pronounced in one dimension and partially mitigated in higher dimensions [10].This trend is qualitatively consistent with bounds whose decay exponents scale as M^(1/d).Source: §5 · p. 17

6. Conclusion

The conclusion presents a multidimensional, operator-theoretic theory of spectral approximation for random feature methods and applies it to elliptic PDEs and eigenvalue problems. It also links spectral approximation to ill-conditioning and identifies what is needed to extend the analysis beyond trigonometric features.

6. Conclusion

  • High-probability RFM approximation rates range from super-exponential to algebraic, with one target-independent sampled space covering a source ball and one coefficient vector attaining spectral accuracy across admissible norms.The rates depend on target regularity and hold under regularity-adapted sampling and uniform sampling on growing frequency windows.Source: §6 · p. 17
  • Abstract solver estimates transfer the approximation bounds to convergence estimates for strong- and weak-form RFM discretizations of second-order elliptic PDEs and eigenvalue problems.Source: §6 · p. 17
  • RFMtxs exhibit super-exponential singular-value decay with Fourier features and exponential decay with tanh features, with corresponding condition-number lower bounds.The conclusion identifies spectral approximation as the mechanism behind both high accuracy and severe ill-conditioning.Source: §6 · p. 17
  • Beyond trigonometric features, spectral convergence requires control of kernel interpolation spaces, source conditions, and effective dimensions; bounded tanh features provide an analytic example.The framework extends the one-dimensional Fourier analysis of [47] to a multidimensional, operator-theoretic setting.Source: §6 · pp. 17–18

Appendix A. Proofs for approximation in interpolation spaces

The appendix proves the interpolation-space approximation estimates by reducing uniform error and coefficient-norm bounds to estimates for preconditioned empirical operators. It then uses high-probability concentration and resolvent comparisons to establish the abstract theorem.

Appendix A. Proofs for approximation in interpolation spaces

  • The resolvent reduction expresses the uniform approximation error and solution-vector norm through operators built from the preconditioned empirical integral operator.The appendix defines the empirical operator from sampled features and its preconditioned version using the kernel integral operator.Source: Appendix A · pp. 20–21
  • Under the admissibility condition, preconditioned empirical concentration and resolvent comparison hold with probability at least 1 − δ, uniformly for a in [0, 1/2].The theorem proof applies these bounds at admissible penalty levels and passes to the critical level ςN by continuity.Source: Appendix A · pp. 21, 23
  • The concentration proof combines Cordes’ inequality for fractional resolvent comparison with Minsker’s Bernstein inequality for an intrinsic-dimension tail bound.The tail bound uses the effective rank of the variance operator rather than the ambient dimension.Source: Appendix A · p. 21
  • For the coefficient-norm estimate when θ ≤ p < 1, the proof applies concentration with γ = 1 − p and uses the resolvent bounds.The theorem proof also derives the error and coefficient-norm estimates using admissible penalty levels approaching ςN.Source: Appendix A · p. 23

Appendix B. Proofs for uniform approximation by random Fourier features · B.1. Effective-dimension estimates for translation-invariant kernels. · B.2. Fourier descriptions and interpolation-space embeddings.

Appendix B develops Fourier-specific ingredients for uniform approximation by random Fourier features, establishing injectivity, effective-dimension bounds, and interpolation-space identifications. It also relates Gevrey and Paley–Wiener-type regularity to finite Fourier-source norms.

Appendix B. Proofs for uniform approximation by random Fourier features

  • Injectivity holds for the considered kernel operators, so their positive spectral subspace is all of L2(Ω).The proof uses compact support of the zero extension and analytic continuation of its Fourier transform from an open set.Source: Appendix B · p. 24

B.1. Effective-dimension estimates for translation-invariant kernels.

  • There are constants c1 > 0 and M0 > 0, depending on d, s, θ, γ, Ω, cτ, and Cτ, such that d(λ) ≤ M when λ = c1(|Ω|/M)^(2s(1−θ)/d), for M ≥ M0.Source: §B.1 · p. 25

B.2. Fourier descriptions and interpolation-space embeddings.

  • Fourier descriptions identify translation-invariant RKHSs through weighted Fourier spaces and bandlimited RKHSs through Fourier support on the frequency window.These characterizations underpin the embeddings of weighted Fourier target classes into the kernel interpolation scale.Source: §B.2 · pp. 29–30
  • Kernel interpolation spaces embed into Sobolev error norms, with exponential-decay and bandlimited kernels yielding smooth functions.The Sobolev-to-W^{t,p} embeddings hold under the stated regularity conditions on the interpolation parameter and Sobolev index.Source: §B.2 · pp. 30–31
  • For Assumption 3.5(b), a Gevrey extension is localized and extended by zero to a compactly supported function agreeing with u on Ω [62, Theorem 1.6.1].The proof uses the Fourier characterization of compactly supported Gevrey functions.Source: §B.2 · p. 31
  • For every 0 < κ < c, the resulting weighted Fourier transform is L2, so ∥u∥κ,1/s is finite.Source: §B.2 · p. 31
  • Under Assumption 3.5(c), a Paley–Wiener-type result [66, Theorem IX.13] gives ∥u∥κ,1 < ∞.The proof takes U = u in the definition of this norm.Source: §B.2 · p. 31

B.3. Proof of the uniform approximation theorem. · B.4. Effective dimension for growing uniform supports.

The B.3 proof establishes uniform approximation on a feature-sampling event, with one coefficient vector supporting estimates across admissible error norms. B.4 bounds the effective dimension for growing uniform frequency supports by a quantity proportional to J^d.

B.3. Proof of the uniform approximation theorem.

  • Both feature representations induce the same kernel integral operator and effective dimension; the optimal density makes its maximum effective dimension equal to d(λ; θ, γ).Under the stated bound d(λ; θ, γ) ≤ M, this yields the sufficient sampling condition used in the proof.Source: §B.3 · p. 32
  • With probability at least 1 − δ, one event depending only on the sampled features gives approximation estimates uniformly over the unit ball of Hσ, for θ ≤ σ ≤ 1.The event is supplied by Theorem 2.3 and is uniform over targets in that source ball.Source: §B.3 · p. 32
  • The proof treats four cases, including Sobolev-kernel, exponential-kernel, and bandlimited settings, and establishes the corresponding estimates by taking suprema over the stated source balls.For exponential kernels, the proof also uses whole-space extension and restriction estimates.Source: §B.3 · pp. 32–34
  • The sampling event, coefficients, and approximant are independent of t and p, so the estimates hold simultaneously for every t ≥ 0 and 1 ≤ p ≤ ∞.The proof obtains the error-norm estimates through deterministic restriction and embedding bounds.Source: §B.3 · p. 33

B.4. Effective dimension for growing uniform supports.

  • For growing uniform supports, d(λJ,S; 1 − a, a) ≤ 2J^d whenever J ≥ max{2c, 2}, with c = SR and a constant independent of S, R, and J.The bound applies to the integral operator generated by the kernel associated with the uniform distribution on the support window.Source: §B.4 · p. 34

B.5. Proof of the growing-bandwidth leverage estimate.

The proof derives a Sobolev restriction bound and combines it with the regularization estimate to establish (3.6), with one sampling event supporting all stated error norms.

B.5. Proof of the growing-bandwidth leverage estimate.

  • The restriction argument gives ∥f|Ω∥W^t,p(Ω) ≤ C_{d,t,p,a,R}(π/S)^{d(1−a)/2}(1+dS^2)^{(2t+d+1)/4}∥f∥_{H^{1−a}}.This follows by applying Proposition B.8(3) on Q_R with Sobolev order t + (d + 1)/2, then restricting to Ω.Source: §B.5 · p. 35
  • Combining the regularization estimate with (B.20) cancels the powers of π/S and S/π, proving (3.6).The argument uses a + (1−a) = 1.Source: §B.5 · pp. 35–36
  • For each fixed ϱ, the sampling event and u_ϱ N,S are independent of t and p, so the estimate holds simultaneously for all stated error norms.The coefficient bound follows from the theorem's coefficient estimate; its λ_J,S power is zero because the source index is s = 1.Source: §B.5 · p. 36

B.6. Proof of the growing uniform reference measure theorem. · Appendix C. Technical Lemmas

The proof establishes feature-only sampling events and fixed approximants that yield simultaneous rates for growing uniform reference measures. Appendix C supplies technical summation lemmas used to control the estimates.

B.6. Proof of the growing uniform reference measure theorem.

  • A single event determined only by sampled features yields (3.6) for every target supported in the prescribed frequency window.The event is uniform over targets in the bandlimited class, and the proof constructs its approximants within that window.Source: §B.6 · p. 36
  • For u in W^{s,p}(Ω), the proof obtains an algebraic error bound proportional to J^{-(s-t)} across admissible p and t.It uses one approximant independent of t and p; the displayed estimate applies when 1 ≤ p ≤ ∞ and 0 ≤ t ≤ s.Source: §B.6 · p. 36
  • For Fourier-regular targets, the bounds are super-exponential in J: exp(-cJ^{1/s}) for Gevrey regularity and exp(-cJ log J) for ultra-analytic regularity.The same extension, truncations, sampling event, and approximant are independent of t and p in both cases.Source: §B.6 · pp. 36–37
  • Choosing J proportional to (N/log(N/δ))^{1/d} converts the three J-dependent rates into the stated sample-size rates (3.11).The proof verifies the required sample-size condition (3.5) for sufficiently large N.Source: §B.6 · p. 37

Appendix C. Technical Lemmas

  • Lemma C.1 establishes a summation-series bound for arbitrary C, λ > 0 and s > 1.The appendix introduces the lemma specifically to control the summation series.Source: Appendix C · p. 37
  • Lemma C.2 controls a summation series for s ≥ 1 and 0 < λ ≤ e^{-max(κ̃,1)}, splitting it at r = κ̃^{-s} ln^s(1/λ).Its proof bounds the resulting sums using the function h(T) = T^{d-1}e^{-κ̃T^{1/s}} and its integral.Source: Appendix C · pp. 37–38
  • Lemma C.3 bounds a summation under 0 < κ̃ < 2, 0 < λ ≤ 1, and λ = (M/d)^{-κ̃M} for M ≥ d.The proof splits the sum into two parts and uses factorial bounds to control them.Source: Appendix C · p. 40

Appendix D. Proofs for RFM solvers · D.1. Proof of the strong-form error estimate.

The proof of the strong-form error estimate combines boundary-operator stability and continuity with auxiliary estimates, then uses the minimality of the random-feature solution to compare it with admissible approximants.

D.1. Proof of the strong-form error estimate.

  • Proposition D.1 supplies stability and continuity estimates with a constant independent of the exact and random-feature solutions.The proof uses these estimates under Theorem 4.2’s assumptions.Source: §D.1 · p. 41
  • For every 0 < r ≤ 2, the boundary value operator w ↦ (Lw, Bw) satisfies the cited estimate with a constant independent of w.The proof invokes [41, Chapter 2, Theorems 5.4 and 7.4].Source: §D.1 · p. 41
  • The proof embeds L2(Ω) into Ξr−2(Ω), then sets r = s + l + 1/2 and w = u − uN to obtain Proposition D.1.The embedding is stated for 0 < r ≤ 2.Source: §D.1 · p. 41
  • Lemma D.2(1) gives an auxiliary estimate under Theorem 4.2’s assumptions, derived from the definitions of L, L, and B and the trace theorem.This estimate is used in the proof of Theorem 4.2.Source: §D.1 · p. 42
  • Lemma D.2(2) gives an auxiliary estimate under Assumption 4.1, which follows directly from its coefficient bounds.Source: §D.1 · p. 42
  • Theorem 4.2 follows by applying Proposition D.1 and Lemma D.2(1) to the minimality inequality, then taking the infimum over w ∈ VRF.For any w ∈ VRF, the minimality of uRF gives L(uRF) ≤ L(w).Source: §D.1 · p. 42

Appendix E. Proofs for singular-value estimates and condition-number lower bounds · Auxiliary lemmas.

Appendix E provides auxiliary estimates and proofs for Theorems 5.1 and 5.2, including lemmas on Sobolev norms, weighted tanh features, and low-frequency Taylor approximations.

Appendix E. Proofs for singular-value estimates and condition-number lower bounds

  • Appendix E collects auxiliary estimates and the proofs of Theorems 5.1 and 5.2.Source: Appendix E · p. 42

Auxiliary lemmas.

  • Multiplication by either specified weight preserves Sobolev norms up to a dimension- and smoothness-dependent constant.The lemma states this for all v in W^s,p(Ω), 1 ≤ p ≤ ∞, and j = 1, 2.Source: Auxiliary lemmas · p. 42
  • For k in [−S,S]^d and v in R, the Gaussian-weighted tanh feature has κ,1-norm bounded by a constant depending on d times 1 + S^d.Here κ = min(1, 11π/(96dS)).Source: Auxiliary lemmas · p. 42
  • The tanh-feature bound is obtained by analytically continuing the weighted feature and shifting the Fourier-transform integration region.The continuation is valid when |kᵀ Im ζ| < π/2; the proof chooses a frequency-dependent shift.Source: Auxiliary lemmas · pp. 42–43
  • Lemma E.3 gives a low-frequency Taylor approximation for Fourier features, with the same estimate applying to their real and imaginary parts.Its proof differentiates the polynomial approximation through order two and uses the exponential-series remainder.Source: Auxiliary lemmas · p. 43
  • For Fourier-feature Taylor approximation, taking real or imaginary parts does not increase the norm.Source: Auxiliary lemmas · p. 43
  • Lemma E.4 gives a low-frequency Taylor approximation for tanh features when |v| ≤ dS, using a polynomial and a frequency cutoff ρ_T.The proof uses analyticity of tanh and Cauchy estimates to control derivatives through order two.Source: Auxiliary lemmas · pp. 43–44

Proof of Theorem 5.1. · Proof of Theorem 5.2.

The proofs bound RFMtx singular values by approximating sampled features in a common low-dimensional space, then use those bounds and frequency-specific arguments to establish the stated condition-number estimates. The arguments cover Fourier and tanh features under the hypotheses stated in the theorems.

Proof of Theorem 5.1.

  • Feature approximation reduces the singular-value bound to finding a subspace on which the sampled feature matrix is small, with dependence on collocation points, domain, and equation minimized.The proof approximates each feature in W^{2,∞}(Ω) by a function in a space of dimension at most m−1.Source: Proof of Theorem 5.1 · pp. 44–45
  • A single sampled Fourier-feature space supplies approximants for all 2N target functions on an event uniform over the target ball, yielding (5.1).The event has probability at least 1−δ0>0; taking real parts preserves the error and coefficient-norm bounds.Source: Proof of Theorem 5.1 · p. 45
  • For tanh features, uniform feature-norm bounds and a common sampled trial space give approximants whose substitution proves (5.2).The proof uses κT with σ=1 and θ=1/2, with a constant aT depending on d and S.Source: Proof of Theorem 5.1 · p. 45

Proof of Theorem 5.2.

  • Assuming σ2N>0, Theorem 5.1 applies with m=2N, and the proof treats Fourier and tanh cases separately to establish the condition-number bounds.The proof uses the stated convention if σ2N=0.Source: Proof of Theorem 5.2 · p. 45
  • For Fourier features, the proof splits high- and low-frequency cases; the low-frequency case constructs a polynomial-based vector to bound σ2N from above.In the high-frequency case, an estimate on feature-column norms combines with (5.1); the low-frequency argument uses polynomial approximation and factorial decay.Source: Proof of Theorem 5.2 · pp. 46–47
  • For tanh features, lower bounds on individual feature columns lead to a frequency-threshold split, and the high- and low-frequency cases establish (5.4).The column estimate uses bounded preactivations and the sign agreement of terms involving tanh′′; the low-frequency case uses polynomial approximation.Source: Proof of Theorem 5.2 · pp. 47–48

Related papers

Loading 2609.03401v1…