The generalization error of random features regression: Precise asymptotics and double descent curve

Motivation and contribution

• The model’s test error follows double descent: it decreases, peaks at interpolation, and decreases again in the overparametrized regime, often reaching its minimum at very large complexity.The paper’s main contribution is an analytically tractable model with precise formulas for this curve [BMM18].Source: §1 · pp. 3–4

What the framework adds

• Compared with simpler random-covariate models, this framework captures optimal vanishing regularization, optimal large overparametrization, and the disappearance of double descent under optimal regularization.Earlier models can show divergence at interpolation but may favor underparametrization, confound parameter count with ambient dimension, and always favor positive ridge regularization [AS17, HMRT19, BHX19].Source: §1 · pp. 4–5

Model and setting

• Random features are modeled as ridge regression on σ(θ_a^T x/√d), equivalently a two-layer neural network with random first-layer weights, for data on the d-dimensional sphere.The features are also interpretable as a randomized approximation to kernel ridge regression, with first-layer weights held fixed.Source: §1 · p. 4

Asymptotic regime

• The analysis takes N, n, d → ∞ with N/d → ψ1 and n/d → ψ2, and allows linear or nonlinear regression functions with independent noise.The resulting prediction-error asymptotics depend explicitly on ψ1, ψ2, noise level τ^2, activation σ, regularization λ, and the linear and nonlinear signal powers.Source: §1 · p. 5

Main asymptotic results

• Theorem 1 establishes convergence in probability of the prediction risk for linear truth at every fixed λ > 0, with an explicit formula, while Section 5.1 gives an analogous nonlinear result.The ridgeless limit is obtained by taking λ → 0 after the proportional asymptotic, and exchanging this limit with d → ∞ remains unproved.Source: §1 · p. 5

Validation and proxy model

• For ReLU random features, numerical results agree excellently with the asymptotic prediction and display the key double-descent features; the model also admits a Gaussian-covariate proxy with the same asymptotic prediction error.The Gaussian equivalence applies despite the original random features being highly non-Gaussian and motivates further analyses [MRSY19].Source: Figure 1 · p. 3, §1 · p. 6

Double descent

• Both bias and variance peak and diverge at the interpolation threshold ψ1 = ψ2 as λ → 0, so double descent persists even when the error is entirely due to bias.This contrasts with linear regression models, where the threshold peak is attributed entirely to diverging variance.Source: §2 · p. 6

Optimal overparameterization

• The minimum test error is achieved as ψ1 = N/d → ∞, and in the ridgeless overparameterized regime the generalization curve decreases monotonically once ψ1 > ψ2.The model is presented as a natural analytically tractable setting where large overparameterization is necessary for optimal prediction without postulating special misspecification.Source: §2 · pp. 6–7, Figure 2 · p. 7

Regularization

• Optimal regularization removes the interpolation peak: the lower envelope of test-error curves becomes monotone decreasing with the number of parameters.Regularization compensates for overparameterization rather than making the interpolation threshold intrinsically optimal.Source: §2 · pp. 7–8, Figure 3 · p. 7

SNR phase transition

• The optimal λ depends on SNR: positive regularization helps at low SNR, whereas λ → 0 is optimal at high SNR, with the regimes separated by a critical value ρ⋆.Above ρ⋆, highly overparameterized near-interpolators are statistically optimal, although exact interpolation is not proved under the order of limits used.Source: §2 · p. 8, Figure 4 · p. 8

Self-induced regularization

• The ridgeless optimum is explained by self-induced regularization: concentrated kernel diagonals act like an effective penalty, with the intuition becoming valid in the wide limit N/d → ∞.Random off-diagonal kernel fluctuations complicate the picture but do not invalidate this wide-limit explanation.Source: §2 · p. 8

3.1 Learning via interpolation

• Interpolation can generalize in deep networks and kernel methods, while prior analyses established double descent and identified interpolation-threshold peaks.These results include kernel regression and ridgeless regression under certain conditions, with prior work also studying correlated, Gaussian, over-specified linear, and Fourier models.Source: §3.1 · p. 9
• Prior calculations captured interpolation-threshold peaks but did not explain large-overparametrization optimality, high-SNR interpolators, self-induced regularization, or double-descent disappearance.The cited random-features variance calculation applied only in the ridgeless limit λ →0.Source: §3.1 · p. 9

3.2 Random features and kernels

• Random-features theory links finite-rank kernel approximations to RKHSs, while this work studies N,n = Θ_d(d) and derives precise test-error asymptotics for finite n/d and N/d.It confirms the nonlinear-target lower bound and achieves it as N/d,n/d →∞; related work also connects random features with neural-network and neural-tangent-kernel dynamics.Source: §3.2 · pp. 9–10

3.3 Technical contribution

• The analysis expresses full prediction error through derivatives of a block-matrix log-determinant, derives Stieltjes-transform fixed-point equations by leave-one-out arguments, and integrates them.Unlike prior work addressing only variance in the λ →0 limit, the method handles both bias and variance.Source: §3.3 · p. 10
• Unlike polynomial-scaling analyses proving near-isometry, this work studies a non-isometric kernel matrix with a nonparametric target and characterizes spectral and eigenvector properties determining test error.The setting is y_i = f_d(x_i) + ε_i, and the asymmetric kernel matrix’s singular-value distribution alone is insufficient because eigenvectors also affect prediction error.Source: §3.3 · p. 10

Basic notation

• Scalars, complex numbers, natural numbers, positive-imaginary complex numbers, imaginary units, spheres, and integer index sets follow explicit definitions.The notation includes R, C, N, C+, i, S^{d−1}(r), and [k].Source: §4 · p. 10

Asymptotic notation

• The paper uses O_d, o_d, Ω_d, O_{d,P}, o_{d,P}, and O_d(Poly(log d)) for asymptotic and probabilistic bounds.The subscript d identifies the asymptotic variable, while O_{d,P} and o_{d,P} denote bounds and convergence in probability.Source: §4 · p. 11

Vector and matrix notation

• Bold lowercase and uppercase symbols denote vectors and matrices, with I_n, 1_{n×m}, and 0_{n×m} representing identity, all-ones, and all-zero matrices.These conventions establish the basic notation for matrix-valued quantities used throughout the paper.Source: §4 · p. 11

Matrix operations

• Matrix operations include Frobenius, nuclear, operator, and maximum norms, Moore–Penrose inverses, entrywise functions, traces, partial traces, and Hadamard products.The notation also defines Tr(A), Tr_[a,b](A), and A ⊙ B for element-wise multiplication.Source: §4 · p. 11

Probability distributions

• Probability notation includes the standard Gaussian measure, the uniform distribution on the sphere, and Gaussian- and sphere-based inner-product distributions.These are denoted by μ_G, γ_d, τ_d, and ˜τ_d.Source: §4 · p. 11

5.1 Statement of main result

• Theorem 2 gives the asymptotic prediction error for random-features ridge regression when N/d and n/d converge proportionally, under the stated activation, data, and noise assumptions.The formula applies for every λ > 0 and reduces to Theorem 1 when the nonlinear component vanishes.Source: §5.1 · p. 13
• For nonlinear truths, random-features regression estimates the linear component while the nonlinear component acts like additional noise, increasing irreducible prediction error by F 2⋆.The resulting formula differs from the linear case by adding F 2⋆ and replacing τ 2 with τ 2 + F 2⋆, consistent with the lower bound RRF(fd, X, Θ, λ) ≥ F 2⋆ + o_d,P(1) in the stated regime.Source: §5.1 · p. 13

5.2.1 Ridgeless limit

• In the ridgeless limit λ → 0+, the asymptotic prediction error decomposes into bias, variance weighted by τ 2 + F 2⋆, and the nonlinear irreducible term F 2⋆.Both bias and variance diverge at the interpolation threshold, while for ψ1 ≥ ψ2 both strictly decrease as the neuron ratio ψ1 increases, enabling highly overparameterized models to outperform smaller ones.Source: §5.2.1 · pp. 14–15
• The ridgeless interpolation peak arises from simultaneous divergence of bias and variance, unlike random-covariate linear regression models where the peak is entirely variance-driven [AS17] [HMRT19] [BHX19].The accompanying analysis reports that both terms diverge at ψ1 = ψ2 and decrease for ψ1 > ψ2.Source: §5.2.1 · p. 15

5.2.2 Highly overparametrized regime

• In the highly overparameterized limit, prediction error remains lower bounded by F 2⋆ because random-features methods only learn linear functions for N = O_d(d^(2−δ)) [GMMM19].The minimum prediction error is often achieved as ψ1 → ∞, but this limit does not recover a nonlinear-fitting kernel method in the stated regime.Source: §5.2.2 · p. 16
• Regularization in highly overparameterized networks is not uniformly beneficial: vanishing regularization is optimal above a critical signal-to-noise ratio ρ⋆, but not below it.The limiting risk is either strictly increasing in λ or first decreasing and then increasing, with an explicit phase-transition optimizer λ⋆.Source: §5.2.2 · p. 16

5.2.3 Large sample limit

• The large-sample limit provides an asymptotic prediction-error formula when n/d → ∞, describing the approximation error of the random-features class as sample size dominates dimension.This regime is defined by n being larger than any constant multiple of d, rather than literally infinite sample size.Source: §5.2 · p. 13, §5.2.3 · pp. 16–17

6 Asymptotics of the training error

• The asymptotic characterization is built from uniquely defined analytic functions ν1 and ν2 satisfying coupled equations and large-imaginary-part bounds.These functions appear in Definition 2, which specifies the analytic and uniqueness conditions used for the training-error formulas.Source: §6 · p. 17
• Theorem 6 gives asymptotic expressions for the regularized training error and squared norm of the minimizer for any λ > 0 under the stated proportional-limit assumptions.The formulas use functions L and A defined through Definition 2 and expectations over covariates, features, noise, and the nonlinear regression component.Source: §6 · p. 18

6.1 Numerical illustrations

• At fixed small λ, training error decreases monotonically with N/n and is nearly zero for N/n > 1, while the test-error peak coincides with the interpolation threshold.The training error is not exactly zero because λ remains positive.Source: §6.1 · pp. 18–19
• The complexity proxy ψ1∥â(λ)∥2 is non-monotone: it rises to the interpolation threshold, then declines for N/n > 1 and converges as ψ1 →∞.This behavior is presented as intuition for why generalization error descends after the interpolation threshold.Source: §6.1 · p. 19
• With optimally chosen ridge regularization, test error becomes strictly decreasing as ψ1 = N/d increases, reducing or eliminating double descent.The authors attribute this improvement to a smaller model-complexity proxy, ψ1∥â(λ)∥2.Source: §6.1 · p. 19

• The construction uses Gaussian x, noise, and auxiliary weights, with feature covariates u_j = μ_0 + μ_1⟨θ_j, x⟩/√d conditional on random directions Θ.Source: §7 · p. 19
• The resulting covariate vector is Gaussian with covariance Σ = μ_1^2ΘΘ^T/d + μ_*^2I_N, and y is jointly Gaussian with u and therefore linear in u plus independent noise.Source: §7 · p. 20
• The Gaussian-covariates model has the same asymptotic prediction error as nonlinear random features when N/d → ψ_1 and n/d → ψ_2.Its prediction error is given by the same formula R as in Definition 1.Source: §7 · p. 20
• Theorem 7 gives the Gaussian-model prediction error as (1 + τ^2)R(ρ, ζ, ψ_1, ψ_2, λ/μ_*^2) + o_d,P(1) for λ > 0.Source: §7 · p. 20
• Figure 9 shows excellent agreement between simulated and predicted Gaussian-model test errors across N/n, using the same theoretical prediction as Figure 5.The simulations use n = 300, d = 100 and ReLU-matched parameters μ_1 = 0.5 and μ_* = (π − 2)/(4π).Source: §7 · p. 20
• A heuristic explanation is that transformed random-feature covariates match Gaussian covariates in their first two moments and have cross-feature correlations of order O_P(1/d).Source: §7 · p. 20

• Under the stated assumptions, samples and random feature weights are independent uniform sphere vectors, with fixed ridge parameter λ > 0.The proof works in proportional asymptotics with N/d and n/d held fixed.Source: §8 · pp. 22, 25
• The proof first decomposes the exact test error into traces involving a resolvent and four random matrices, using rotational invariance, kernel replacement, and concentration.Rotational invariance randomizes the signal direction; replacing the kernel matrix by µ2 incurs negligible error, and the risk concentrates around its expectation.Source: §8 · p. 22
• A block-structured matrix A converts the required traces into Stieltjes-transform and log-determinant quantities, whose asymptotics are obtained by leave-one-out analysis.The argument handles dependence by decomposing vectors into components parallel and orthogonal to a feature direction, replacing spherical projections with Gaussian variables, and isolating a low-rank correction.Source: §8 · pp. 23–24
• The Stieltjes transforms converge to the unique analytic solution of two nonlinear fixed-point equations, which determines the limiting transform and enables an explicit log-determinant formula.The solution of the fixed-point system gives the limits of the finite-dimensional transforms, while integrating the limiting transform yields the asymptotic log determinant.Source: §8 · pp. 24–25
• Because the fixed-point solution is stationary for the log-determinant functional, its derivatives simplify and, when substituted into the risk decomposition, recover the theorem’s bias and variance expressions.The resulting quantities coincide with the definitions of B and V, completing the proof of Theorem 2.Source: §8 · pp. 25–26

9 Proof of Proposition 8.1

• The Gaussian process is represented using independent spherical-harmonic coefficient vectors, with the degree-one coefficient replaceable by a uniform vector on its sphere by symmetry.Lemma 9.1 supplies the Gaussian coefficient representation, while Lemma 9.2 justifies replacing a fixed degree-one vector by a uniformly distributed one.Source: §9 · p. 26

9.1 Proof of Lemma 9.3

• Lemma 9.3 decomposes the expected risk by expanding the feature map and target in spherical harmonics and separately evaluating the resulting terms.The proof introduces harmonic evaluation matrices at the sample and feature points, then combines the evaluations of terms T1, T2, and T3.Source: §9.1 · p. 28

9.2 Proof of Lemma 9.4

• Lemma 9.4 follows from resolvent norm bounds, Wishart estimates, and Gegenbauer expansions that approximate the covariance matrix by a linear covariance term plus an identity term.These estimates yield E[|1 − 2S10 + S20|] = od(1), E[supk≥2 |S2k − S3|] = od(1), and E|S21 − Ψ2| = od(1), with the final equation handled analogously.Source: §9.2 · pp. 29–31

9.3 Proof of Lemma 9.5

• Lemma 9.5 controls the remaining risk terms by rewriting them as quadratic forms and bounding their variances using operator-norm and harmonic-dimension estimates.The proof uses Gaussian-to-spherical concentration for the degree-one coefficient and symmetry between the sample and feature matrices; the remaining variance bounds are analogous.Source: §9.3 · pp. 31–32

10 Proof of Proposition 8.3

• The section’s final step, stated in its proof plan, transfers the Gaussian Stieltjes-transform asymptotics to the spherical version by showing that changing the vector distributions has negligible effect.This distributional comparison is deferred to Section 10.4, while the present part establishes the Gaussian fixed-point analysis.Source: §10 · p. 33, §10.1 · p. 33

10.1 The key lemma: partial Stieltjes transforms are approximate fixed point

• Lemma 10.1 shows that Gaussian partial Stieltjes transforms m1,d and m2,d approximately satisfy the fixed-point equations under the stated polynomial-activation and linear-regime assumptions.For Im(ξ) > ξ0, the deviations from F1 and F2 are bounded by C·err(d), with err(d) → 0.Source: §10.1 · pp. 33–34

10.2 Preliminaries of the proof of Lemma 10.1: Stieltjes transforms and the fixed point equation

• For sufficiently large Im(ξ), the fixed-point map preserves a bounded complex domain and is 1/2-Lipschitz, yielding a unique solution by Banach’s theorem.The normalized partial Stieltjes transforms are also Stieltjes transforms of probability measures, so they are analytic and map C+ into C+; concentration makes them converge to their means almost surely and in L1.Source: §10.2 · pp. 34–35

10.3 Proof of Lemma 10.1: Leave-one-out argument

• The proof of Lemma 10.1 uses a leave-one-out decomposition, Schur complements, perturbation bounds, and Sherman–Morrison–Woodbury simplifications to derive the fixed-point equations.The argument obtains ψ1,dω3 = F1(m1,d,m2,d; ξ; …), with the second equation following symmetrically.Source: §10.3 · pp. 36, 37, 39
• The leave-one-out argument establishes the needed resolvent identities by combining variance bounds, operator-norm controls, and stability and concentration of partial Stieltjes transforms.Partial transforms remain stable under deleting a row and column and under small dimension changes, yielding convergence to the deterministic transform and proving Eqs. (161), (176), and (177).Source: §10.3 · pp. 40, 41, 43

10.4 Equivalence between Gaussian and sphere version of Stieltjes transforms

• If ∥A − A∥⋆/d = o_d,P(1), then E[supℑξ≥ξ0 |∆(A, A, ξ)|] = o_d(1).Source: §10.4 · p. 44

10.5 Proof of Proposition 8.3

• The proof extends Proposition 8.3 from polynomial to general activations by Hermite-polynomial approximation, preserving the relevant coefficients and proving uniform convergence on compact subsets of C+.For every ε > 0, a polynomial approximation satisfies ∥σ − ˜σ∥L2(τd) ≤ ε for all sufficiently large d; letting ε → 0 establishes the stated convergence.Source: §10.5 · pp. 46–47

11.1 Properties of the Stieltjes transforms and the log determinant

• As ℑξ → ∞, the partial Stieltjes transforms satisfy m1(ξ; q)ξ → −ψ1 and m2(ξ; q)ξ → −ψ2.The argument uses the fixed-point map’s 1/2-Lipschitz property for sufficiently large imaginary part.Source: §11.1 · p. 48
• The auxiliary log-determinant analysis controls Gd and g at large imaginary arguments and establishes their required derivative bounds uniformly over fixed parameter regions.The analysis obtains the asymptotic relation for Ξ2 and bounds resolvents and derivatives using Wishart operator-norm control.Source: §11.1 · pp. 49–50

11.2 Proof of Proposition 8.4

• The proof of Proposition 8.4 concludes by sending ε → 0 to obtain Eq. (76) and, by a similar argument, Eq. (77).Source: §11.2 · p. 51

12.1 Proof of Theorem 3

• 12.1 Proof of Theorem 3: The limiting product m_1m_2 equals −[(ψζ^2 − ζ^2 − 1)^2 + 4ζ^2ψ]^(1/2) + (ψζ^2 − ζ^2 − 1).This identifies the λ → 0 formula for χ with its alternative representation through the relationship between χ and m_1m_2.Source: §12.1 · pp. 51–52
• 12.1 Proof of Theorem 3: Eigenvalue multiplicity and positivity imply that one of m_1 and m_2 scales as 1/u while the other scales as u when u → 0 and n > N.Consequently, m_1m_2 is non-positive and converges to the unique non-positive root of the derived quadratic equation.Source: §12.1 · p. 52

12.2 Proof of Theorem 4

• 12.2 Proof of Theorem 4: Under the same lemma conditions, simple calculus reduces the theorem to an explicit expression involving ψ_2, ζ, and λ.The expression is −[(ψ_2ζ^2 − ζ^2 − (λψ_2 + 1))^2 + 4ζ^2ψ_2(λψ_2 + 1)]^(1/2) + (ψ_2ζ^2 − ζ^2 − (λψ_2 + 1)).Source: §12.2 · p. 53

12.3 Proof of Theorem 5

• 12.3 Proof of Theorem 5: The proof uses simple calculus and a lemma symmetric to Lemma 12.2, yielding the corresponding formula with ψ_1.The expression is −[(ψ_1ζ^2 − ζ^2 − (λψ_1 + 1))^2 + 4ζ^2ψ_1(λψ_1 + 1)]^(1/2) + (ψ_1ζ^2 − ζ^2 − (λψ_1 + 1)).Source: §12.3 · p. 53

13.1 Proof of Proposition 5.1

• Brless and Vrless are strictly decreasing in ψ1 when ψ1 > ψ2, while at ψ1 = ψ2 only E0,rless vanishes and E1,rless, E2,rless remain nonzero.As ψ1 approaches zero, E1,rless has the stated O(ψ1) expansion, and E0,rless/ψ1 has the supplied finite limit as ψ1 approaches infinity.Source: §13.1 · pp. 53–54

13.2 Proof of Proposition 5.2

• Rwide(ρ, ζ, ψ2, λ) is either strictly increasing in λ or strictly decreasing initially and then strictly increasing.The proof uses that ω(λ, ψ2, ζ) is always negative and increasing in λ, together with the derivative structure of Rwide.Source: §13.2 · pp. 54–55

A Technical background

• The technical background introduces decompositions in hyperspherical harmonics on S^(d−1) and orthogonal polynomials on the real line.These classical tools are connected to prior harmonic-analysis and statistics literature cited in the appendix.Source: A Technical background · p. 60

A.1 Functional spaces over the sphere

• Functions on the sphere are decomposed into orthogonal harmonic subspaces V_d,ℓ, with projections P_k defined through spherical harmonics.The appendix also specifies the sphere, L2 structure, and dimensions of the harmonic subspaces.Source: §A.1 · p. 60

A.2 Gegenbauer polynomials

• Gegenbauer polynomials form the rotationally invariant basis linked to degree-ℓ spherical harmonics, and their kernels represent projectors onto degree-k harmonic components.Their normalization is chosen so Q_ℓ^(d)(d) = 1.Source: §A.2 · p. 61

A.3 Hermite polynomials

• Hermite polynomials provide the Gaussian-limit analogue of Gegenbauer polynomials, with fixed-degree coefficients converging as the sphere dimension grows.They form an orthogonal basis of L2(R, μG), and the appendix relates their coefficients to the spherical coefficients λ_d,k(σ).Source: §A.3 · pp. 61–62

B Proof of Proposition 8.2

• The proof of Proposition 8.2 establishes the determinant identity using a continuous logarithm branch, eigenvalue continuity, linearity of A(q), and block-matrix inversion.The integer branch index remains locally constant under sufficiently small perturbations of q.Source: B Proof of Proposition 8.2 · p. 63

C.1 Proof of Lemma 9.1 and Lemma 9.2

• The proofs of Lemmas 9.1 and 9.2 use Gegenbauer expansions whose zeroth- and first-order coefficients vanish under Assumption 3, then establish the result by orthogonal rotational invariance.For any orthogonal matrix O, a corresponding transformation preserves the relevant distributional structure and immediately yields the lemma.Source: §C.1 · p. 64

C.2 Proof of Lemma 9.6 and Lemma 9.7

• The proofs of Lemmas 9.6 and 9.7 reduce their terms through Sherman–Morrison–Woodbury identities and obtain A = (K12 + 1)^2/(K11(1 − K22) + (K12 + 1)^2)^2 ≥ 0, with bounded quantities yielding expectation bounds.Lemma 9.7 is proved first for a singleton index set A = {α}, with extension to arbitrary sets stated as direct.Source: §C.2 · pp. 64, 65, 67

C.3 Some auxiliary lemmas

• Under exponential-growth assumptions on σ and σ′ with M/d bounded above and below, the auxiliary lemmas control random kernel matrices and related operator norms by Od,P(exp{C(log d)1/2}).The bounds use Gaussian-to-spherical comparisons, truncation, concentration, and Frobenius-to-operator-norm control; one intermediate bound is Od,P((log d)1/2).Source: §C.3 · pp. 69–70

C.4 The decomposition of kernel inner product matrices

• For independent uniform-sphere rows Θ and X with n/d and N/d bounded, Lemmas C.6 and C.7 provide decompositions and operator-norm estimates for kernel inner-product matrices, extending results previously proved in the Gaussian case [GMMM19].The estimate for Qk(ΘX^T) follows by embedding Θ and X into W = [Θ^T|X^T]^T and bounding it through Qk(WW^T) − I_N+n.Source: §C.4 · p. 72
• The kernel inner-product matrix is rewritten using Q = ΘΘT/d, with E[∥∆∥2 op] = o_d(1).Source: §C.4 · p. 72

C.5 A lemma on the variance of the quadratic form

• Lemma C.8 assumes independent centered unit-variance iid vectors g and h, and its quadratic-form calculation yields terms involving E[g4] − 3, Tr(B^TB), and Tr(B2).Source: §C.5 · pp. 72–73

Proof of Lemma 8.1

• The fixed-point equation for m1 and m2 shows that (m1(ξ; q), m2(ξ; q)) is a stationary point of Ξ(ξ, ·, ·; q).This holds for fixed ξ ∈ C+ and q ∈ R5.Source: D Proof of Lemma 8.1 · p. 74
• Implicit differentiation is applied to the stationary-point characterization.The proof introduces u = (s1, s2, t1, t2, z1, z2)^T in this calculation.Source: D Proof of Lemma 8.1 · p. 74
• Basic algebra completes the proof of Lemma 8.1.Source: D Proof of Lemma 8.1 · p. 74

Setup

• Theorem 6 is analyzed asymptotically with ψ1,d=N/d and ψ2,d=n/d held constant as d grows, using the resolvent matrices Ξ and Π.This establishes the scaling regime and matrix objects used throughout the proof.Source: E Proof sketch for Theorem 6 · p. 75

Step 1. The expectation of regularized training error

• Step 1 computes the expectation of the regularized training error for random-features regression and shows the coefficients before F_0^2 asymptotically vanish.The calculation also invokes Lemma C.6 to control the relevant resolvent expression.Source: E Proof sketch for Theorem 6 · p. 75

Step 2. The norm square of minimizers

• Step 2 evaluates the squared norm of the minimizers as a component of the theorem’s asymptotic calculation.The supplied proof sketch identifies this norm calculation as a separate step after the training-error expectation.Source: E Proof sketch for Theorem 6 · p. 75

Step 3. The derivatives of the log determinant

• Step 3 derives log-determinant derivatives by introducing a block matrix A, differentiating g, applying Daskin’s theorem, and completing the argument with elementary calculus.The block matrix has dimension M=N+n and depends on q=(s1,s2,t1,t2,p).Source: E Proof sketch for Theorem 6 · p. 76