Fully Deterministic Shallow Neural Networks with Localized Activations. Part II: Condition Number Estimates

Motivation

• The study addresses how small Gram eigenvalues and near-cancellation affect recovery of arbitrary coefficients in the prescribed feature family.The square root of the Gram condition number measures synthesis-map sensitivity from Euclidean coefficients to H^m.Source: §1 · p. 4

Scope

• The recovery argument applies to arbitrary coefficient vectors and includes feature families used for optimal approximation; the upper bounds cover midpoint and Gauss-Legendre offsets.The Gauss-Legendre extension uses a polynomial lower bound on offset spacing.Source: §1 · p. 5

Main results

• For fixed integer m and sufficiently large feature resolution, the raw and column-normalized Sobolev Gram matrices are positive definite and have quantitative condition-number upper bounds.The exponent has order N^2/d / log(N + 2), with N ≍ M^d.Source: §1 · p. 5

Proof strategy

• Homogeneous direction selectors separate ridge-direction groups, while one-dimensional coefficient-recovery estimates control synthesis from observations on an interior ball.The selectors have degree O_d(M) and coefficient norm exp(O_d(M log M)).Source: §1 · p. 5

Main results

• Matching lower bounds on fixed ellipsoids establish the upper-bound exponential order for raw Gaussian and raw and normalized tanh-difference midpoint matrices.This sharpness applies over the stated class of domains and admissible angular degree sequences, not necessarily on every fixed domain.Source: §1 · p. 6

2.1. Midpoint ridge families

• Midpoint features use localized Gaussian or tanh-difference ridge activations on deterministic directions and offsets; their feature columns are linearly independent for sufficiently large resolution.The domain radius sets the base offset range, and the features use a common scale for each activation.Source: §2.1 · pp. 6–8

2.2. Gauss-Legendre offsets

• Gauss-Legendre offsets replace midpoint nodes while retaining the direction set and scale, with O(M) offsets per representative direction and at least C^-1 M^-2 separation.The resulting feature count is comparable to M^d.Source: §2.2 · p. 11

2.3. Sobolev Gram matrices

• For fixed m, the full H^m inner product defines raw and column-normalized continuous Gram matrices; both are positive definite and satisfy condition-number upper bounds for sufficiently large M.The result applies to each activation in the midpoint construction, and the same upper bound extends to Gauss-Legendre features.Source: §2.3 · p. 12

2.4. Proof outline

• The proof isolates direction groups with homogeneous differential selectors, applies one-dimensional recovery estimates, and transfers recovery from an interior ball to the full ridge sum.Feature-norm bounds then control raw and normalized Gram spectra; offset-spacing bounds extend the result to Gauss-Legendre nodes.Source: §2.4 · p. 13

3. Homogeneous direction selectors

• Selectors of common degree O_d(M) encode cardinal values at the antipodal representatives, so the corresponding differential operator isolates one direction group.Their construction depends only on the prescribed directions, not on the activation, offsets, target, or output coefficients.Source: §3 · p. 14, §3.2 · pp. 19–20

3.1. Antipodal completion and Jacobi nodes

• Jacobi latitude nodes and antipodal completion produce distinct direction nodes, while their spacing bounds control the inverse factors used in recursive interpolation.The latitude count k and the signed-offset count K refer to different node sets.Source: §3.1 · pp. 14–16

3.2. Cardinal polynomials and homogenization

• Cardinal polynomials on the recursive sets are symmetrized across antipodes and homogenized to a common even degree, preserving their values on the unit sphere.The resulting selectors have degree O_d(M) and coefficient bound B_sel ≤ exp(C_d M log M).Source: §3.2 · pp. 19–20

3.3. Planar selectors

• For the actual planar direction set, selectors can be chosen with common even degree r_pl ≤ J and coefficient bound B_sel ≤ e^(CM), while satisfying P_ν(±ω_μ) = δ_νμ.A direct product gives this alternative planar construction.Source: §3.3 · pp. 20–21

3.4. Rotation and local coordinates

• Rotating coordinates so the target direction becomes the first axis makes its direction group depend only on the first local coordinate, enabling recovery from the ridge sum’s local L2 norm.The rotated coefficient norm increases the logarithmic bound by at most O_d(M).Source: §3.4 · pp. 22–23

4.1. Exponential sums on a fixed band

• Separated real offsets yield a fixed-band lower bound for exponential sums through distinct sampled phase nodes and an inverse Vandermonde estimate.The argument covers K = 1 as well as larger groups.Source: §4.1 · pp. 24–25

4.2. Recovery on the real line and a fixed interval

• Plancherel and the Gaussian transform give a whole-line lower bound, while derivative-tail estimates yield a lower bound on a fixed large interval.The interval bound is uniform for the selector derivative orders considered, once M is sufficiently large.Source: §4.2 · pp. 26–28

4.3. Recovery on an interior interval

• Taylor approximation and polynomial propagation transfer the large-interval lower bound to a fixed interior interval for the entire Gaussian extension.The propagation estimate controls polynomial growth from the interior interval to a fixed bounded complex set.Source: §4.3 · pp. 28–29
• The interior-interval recovery exponent has order M^2/TM, since ε^-2 = M^2/TM and MTM = o(M^2/TM).The estimate applies under (4.1) for 0 ≤ r ≤ CrM and sufficiently large M.Source: §4.3 · pp. 29, 31

4.4. Coefficient recovery on the domain

• The group-factor contribution to the exponent, using the stated bound on J relative to J_M, yields the second inequality in (4.23).The passage gives the bound as “J ≤CJM d−1”; its mathematical formatting is unclear.Source: §4.4 · p. 35

5.1. Distance from constants

• Lemma 5.1 asserts that a positive constant CA, depending only on A, exists under (5.3), for every v ∈ C_K and γ ∈ C; the excerpt does not specify its role.Condition (5.3) requires 0 < ε ≤ 1, |aj| ≤ A/ε, and |ai − aj| ≥ h/ε for i ≠ j.Source: §5.1 · p. 36

5.2. Entire-function propagation

• Lemma 5.2 propagates bounds for an entire function from a fixed interval to a disk, with dependence on the Taylor truncation degree.The result supports propagation of polynomial approximations to the interval shifted by iπ.Source: §5.2 · p. 39

5.3. Polynomial differences under an imaginary shift

• Lemma 5.3 bounds a polynomial's nonconstant part using its difference under the imaginary shift iπ, with explicit dependence on its degree.A polynomial with period iπ is constant.Source: §5.3 · p. 40

5.4. Local derivative lower bounds

• Under the offset-separation assumptions, Theorem 5.4 bounds every derivative norm of a tanh-difference sum from below in terms of its coefficient norm.The proof combines the distance-from-constants estimate with Taylor approximation, entire-function propagation, and the imaginary-period argument.Source: §5.4 · p. 41

5.5. Coefficient recovery on the domain

• Theorem 5.6 gives coefficient recovery for the prescribed tanh-difference midpoint dictionary and for sums with a common width and bounded, separated centered offsets.The domain estimate uses direction selectors to isolate a group, then a compactly supported test function to connect that group's derivative norm to the full sum.Source: §5.5 · pp. 44–45
• Compactly supported Sobolev test functions are justified by weak derivative bounds and mollification, enabling integration by parts and rescaling on the domain.The test-function estimates use derivatives through order 2r, and the rescaling is x = x0 + εy.Source: §5.5 · pp. 46–47
• Selecting a direction group with maximal coefficient norm gives a group norm of at least J^-1/2 times the total coefficient norm, allowing single-group recovery to control all coefficients.The argument then squares this bound to obtain the stated global recovery estimate.Source: §5.5 · p. 47
• For the midpoint dictionary, the logarithmic terms in the recovery estimate are absorbed into C M^2/T_M for sufficiently large M, uniformly with respect to rounding in h_M.The constants and threshold depend only on the fixed construction and geometric parameters.Source: §5.5 · pp. 47–48

6.2. Column norms and Gram eigenvalues

• Upper bounds on column norms control the largest eigenvalue and the effect of column normalization.For normalized matrices, each diagonal entry is one, so λmax ≤ N.Source: §6 · p. 48, §6.2 · p. 51

6.1. Recovery exponents

• Gaussian recovery has exponent C(M^2/T_M + MT_M), with a common threshold across direction groups and coefficient vectors.The estimates are uniform over the direction groups.Source: §6.1 · p. 49

6.2. Column norms and Gram eigenvalues

• Because the full H^m norm contains the L2 term, coefficient recovery bounds the smallest Gram eigenvalue and proves positive definiteness and linear independence.The lower bound applies for each fixed m.Source: §6 · p. 48, §6.2 · p. 50
• For normalized midpoint matrices, λmin ≥ e^-C_mM^2/T_M and λmax ≤ N, giving a condition-number upper bound.The stated feature-count form uses constants depending on m and the fixed construction and geometric parameters.Source: §6.2 · p. 51

6.3. Gauss-Legendre condition-number upper bounds

• The raw and column-normalized full H^m Gram matrices also satisfy the upper bounds with Gauss-Legendre offsets, for either activation.The constants depend on m and the fixed construction and geometric parameters.Source: §6.3 · p. 52

• The minimum distance between signed Gauss-Legendre offsets is at least C^-1M^-2, while the signed offset count satisfies K ≤ CM.The centered offsets are bounded by B0.Source: Gram matrices are positive definite… · p. 52
• For Gaussian features, the new node separation keeps the Vandermonde inverse bound at most exp(CK T_M), yielding the same recovery exponent.The resulting exponent is C(M^2/T_M + MT_M).Source: Gram matrices are positive definite… · p. 52
• For tanh differences, applying the recovery estimate with the Gauss-Legendre gap and prescribed scale satisfies the width conditions for sufficiently large M.Together, the estimates give the corresponding upper bound for both activations.Source: Gram matrices are positive definite… · p. 52

• Both activations satisfy the recovery estimate with Gauss-Legendre columns, and the spectral bounds apply to raw and column-normalized matrices.The column-norm bound is uniform in the offsets.Source: M 2/TM + M log(2 + MTM) + M · p. 53
• Since N_GL ≍ M^d, the Gauss-Legendre estimates take the stated feature-count form.Source: M 2/TM + M log(2 + MTM) + M · p. 53
• The matching lower bounds in the following section concern midpoint feature families.Source: M 2/TM + M log(2 + MTM) + M · p. 53

7. Condition-number lower bounds

• The bounds use midpoint-grid columns from a signed direction group, with interlacing extending subset estimates to the full matrix; the ellipsoidal constructions use the direction e1.Column normalization preserves the interlacing argument, while Gaussian normalized bounds instead construct small-norm linear combinations.Source: §7 · pp. 53–54

7.1. Gaussian lower bounds

• The raw Gaussian Gram matrix has a lower bound of the same exponential order as the upper bound; the normalized Gaussian matrix has polynomial lower bounds on every fixed domain and an exp(cM) lower bound on a fixed ellipsoid.Source: §7 · p. 53

7.2. Tanh-difference lower bounds

• Raw and column-normalized tanh-difference matrices both have matching exponential-order lower bounds on fixed ellipsoids.The proof cancels successive terms in exterior-column expansions; the resulting vanishing moments yield the small Rayleigh quotient.Source: §7.2 · pp. 60, 63, 64

7.3. Optimality over bounded Lipschitz domains

• On fixed ellipsoids, raw Gaussian and both raw and column-normalized tanh-difference matrices attain lower bounds matching the upper bound’s exponential order.These matching orders hold for every fixed d ≥ 2 and integer m ≥ 0.Source: §7 · p. 53, §7.3 · p. 64
• On fixed ellipsoids, matching lower bounds rule out improving the upper exponent for raw Gaussian and raw and normalized tanh-difference matrices.The exponent M^2/T_M is equivalent to N^(2/d)/log(N+2).Source: §7.3 · pp. 64–65
• The ellipsoidal results do not prescribe growth on each domain, and the bounds leave growth on the centered ball undetermined.On the ball, positive offsets beyond the projection interval lie only in a shrinking margin; the polynomial bound for normalized Gaussian columns still applies.Source: §7.3 · p. 65
• For normalized Gaussian columns, no matching lower bound is established; the available exp(cmM) lower bound does not show slower growth after normalization.The table identifies the normalized Gaussian case as not established, and Proposition 7.2 supplies the weaker lower bound.Source: §7.3 · pp. 64–65
• For raw Gaussian columns on the short-axis ellipsoid, exterior-column Gaussian decay creates an exponential norm disparity from interior columns, yielding the matching lower bound.Column normalization removes this norm disparity.Source: §7.3 · p. 65

8. Numerical observations on a square

• The experiments use 2M equally spaced directions and M positive midpoint offsets for each activation.They compare resolutions M ∈ {3, 5, 7, 9, 13} on the square (−1, 1)^2.Source: §8 · p. 65
• All four raw and column-normalized condition-number sequences increase across the sampled resolutions.The figures plot log10 κ2 against M^2/log(M + 2).Source: §8 · p. 65, Figure 2 · p. 66, Figure 3 · p. 66
• Column normalization substantially reduces Gaussian condition numbers and gives a flatter curve over the plotted range.The comparison uses raw and normalized Gaussian L2 Gram matrices.Source: §8 · p. 65, Figure 2 · p. 66
• Rapid tanh-difference condition-number growth remains after column normalization.The raw and normalized tanh-difference matrices use the same square and node counts as the Gaussian comparison.Source: §8 · p. 65, Figure 3 · p. 66

9. Conclusion

• κ2 ≤ exp(CmM^2/log(M + 2)) for raw and column-normalized matrices with either activation and either offset grid on fixed bounded Lipschitz domains.For fixed integer m ≥ 0 and sufficiently large M, N ≍ M^d makes the exponent order N^(2/d)/log(N + 2).Source: §9 · p. 67
• Homogeneous direction selectors, one-dimensional coefficient recovery, and interior-ball observations control arbitrary coefficient vectors for the prescribed nodes.Under Part I's additional geometric and parameter hypotheses, this quantifies coefficient sensitivity for feature families attaining optimal Sobolev approximation order.Source: §9 · p. 67
• On fixed ellipsoids, midpoint offsets yield matching lower bounds for raw Gaussian and raw and normalized tanh-difference matrices, establishing optimality over the domain class in those cases.The Gaussian lower bound uses column-norm disparities; the tanh-difference bound uses cancellation of successive tail terms. Normalized Gaussian matrices instead have a polynomial lower bound on fixed domains and an exp(cmM) lower bound on a fixed ellipsoid.Source: §9 · p. 67
• Square-domain L2 experiments illustrate condition-number growth and the effect of normalization at finite resolutions.The passage reports the experiments' scope but no specific numerical outcomes.Source: §9 · p. 67
• The normalized Gaussian lower-to-upper-bound gap remains unresolved, as do sharp growth rates on origin-centered balls and matching lower bounds for Gauss-Legendre offsets.Origin-centered balls have projection range (−R, R), excluding the fixed exterior offset intervals used in the ellipsoidal proofs.Source: §9 · p. 67