Benign overfitting in ridge regression

1.1 Motivation and our contribution

• Overparameterized models can interpolate noisy data while generalizing with little, zero, or even negative regularization, challenging classical bias-variance intuition.The paper studies these phenomena in ridge regression with p-dimensional data and n < p i.i.d. noisy observations.Source: §1.1 · p. 1
• The analysis replaces the independence assumption in Bartlett et al. (2020) with a weaker condition on the tail Gram-matrix condition number and gives tight bias and variance bounds through eigenvalue separation.For ridgeless regression, the relevant regime has k* ≪ n leading eigenvalues removed and a tail covariance sequence with effective rank large relative to n.Source: §1.1 · p. 2
• The ridge-regression extension covers a larger range of regularization parameters than Hsu et al. (2014) and gives conditions where negative regularization is optimal, with arbitrarily high multiplicative excess-risk gains.The extension follows algebraically from the same eigendirection separation and enables comparison with classical results.Source: §1.1 · p. 2

1.2 Related work

• Related work divides into asymptotic spectral-limit analyses, distribution-specific bounds, and non-asymptotic bounds for arbitrary covariance structure, the category containing this paper.The introduction places the paper alongside Bartlett et al. (2020), Kobak et al. (2020), Chinot and Lerasle (2021), and Dereziński et al. (2019, 2020) in the third category.Source: §1.2 · pp. 2–3

2. Ridge regression setup

• Ridge regression learns an unknown real-valued function from noisy observations at n points in the overparameterized regime p > n.Source: §2 · p. 3

2.1 Covariate model

• The covariates are n i.i.d. zero-mean vectors in R^p, with results governed by the eigenvalue spectrum of their covariance matrix Σ.The whitened data have isotropic centered i.i.d. sub-Gaussian rows.Source: §2.1 · p. 3

2.2 Response model

• Responses follow a linear true function with coefficient vector θ* plus noise whose components are i.i.d. centered random variables.Source: §2.2 · p. 4

2.3 Learning procedure

• Ridge regression estimates θ* from X and y using regularization parameter λ, while λ = 0 gives the ridgeless Gram-matrix case and λ shifts its eigenvalues.The matrix λI_n + XX^⊤ is central to the analysis.Source: §2.3 · p. 4

2.4 Excess risk and its bias-variance decomposition

• Excess risk is the population-average squared prediction error and decomposes into noiseless bias from incomplete spanning or regularization and variance from learning the zero function from noise.The paper seeks sharp non-asymptotic bounds for these two terms, whose noise contribution scales linearly with the noise variance.Source: §2.4 · pp. 4–5

3.1 Essentially high-dimensional linear regression vs. essentially low-dimensional

• In essentially low-dimensional regression, uniform sample-covariance approximation yields the classical k/n rate because noise is projected onto a low-dimensional subspace.Increasing model size improves signal approximation but increases sensitivity to noise, producing the classical bias-variance trade-off.Source: §3.1 · pp. 5–6
• In isotropic p ≫ n regression, the minimum-norm interpolator learns almost none of the signal while damping noise by p/n, effectively behaving like a zero out-of-sample estimator.Down-weighting higher-frequency features can instead make the minimum-norm solution learn low-frequency signal while interpolating noise with high-frequency components.Source: §3.1 · pp. 6–7

3.2 The ground provided by the previous work

• Bartlett et al. (2020) showed that ridgeless interpolation damps noise beyond a constant factor only when, after removing k components, the covariance tail has effective rank large relative to n.Their result combines classical low-dimensional variance in the first k components with essentially high-dimensional variance in the remaining components, but sharply bounds only variance.Source: §3.2 · p. 7

3.3 Our contribution

• The authors complete Bartlett et al. (2020) by sharply bounding bias, extending the analysis to nonzero ridge regularization, and replacing component independence with a broader sufficient condition.Their eigendirection split shows that the high-dimensional tail contributes implicit regularization to the leading components while most tail signal energy remains in the error.Source: §3.3 · pp. 9–10
• The analysis uses covariance eigenbasis blocks for the first k and remaining directions, with results depending on the eigenvalue sequence through effective dimensions rather than ambient p.The central tail Gram matrix is shifted by λ, and the framework remains valid when dimensions are padded with zero eigenvalues.Source: §3.3 · p. 9
• Instead of independent components and high effective rank, the authors assume oracle control keeping all eigenvalues of the tail matrix A_k within a constant factor.This condition implies the relevant tail effective-rank bound; small-ball conditions support it, with stronger sufficient conditions for sub-Gaussian data.Source: §3.3 · p. 9
• Negative ridge regularization can be optimal when tail noise and signal energy are small, but the tail’s effective rank abruptly becomes much larger than n.In this regime, the essentially high-dimensional tail supplies too much implicit regularization, which negative λ compensates for.Source: §3.3 · p. 10

3.4 Additional notation

• The section’s additional notation defines eigenvalue ordering, matrix entries, data columns, leave-one-component analogues, and tail effective-rank ratios used throughout the paper.A consolidated notation appendix is provided for reference.Source: §3.4 · p. 10

• Outside the noncritical regime, eigenvalues of A_k may be smaller and previously dominated error terms can matter; Section 8 only gives an upper bound there, with negative regularization potentially improving rates.Source: §4 · p. 11
• The upper-bound tightness argument requires NoncritReg(k, γ) and CondNum(k, δ, L), with the latter explicitly controlling the condition number of A_k.The authors use this direct condition-number assumption instead of distribution-specific sufficient conditions because they seek minimal assumptions and do not require sub-Gaussianity.Source: §4 · pp. 10–11
• Theorem 1 shows that, under NoncritReg and CondNum, choosing k = min(k̄, k*) preserves the assumptions and yields a constant lower bound on ρ_k with high probability.The condition-number assumption requires A_k to be positive-definite with bounded condition number; shifting k to k* changes constants but retains the result.Source: §4 · p. 11
• Because the lower bounds use different assumptions from the upper bounds, they do not prove that the upper bound is always tight.The results instead show that stronger bounds require specific knowledge of the data and signal distributions.Source: §4 · p. 12
• Theorem 2 provides lower bounds for variance under IndepCoord and for bias under PriorSigns and ExchCoord, when ρ_k exceeds a constant threshold and k ≤ k*.The variance argument assumes independent coordinates, while the bias argument assumes exchangeable whitened coordinates and independently randomized signs of the signal.Source: §4 · pp. 12–13

5. Effective ranks and control of the spectrum of A_k

• Controlling the condition number of A_k requires identifying when Assumption CondNum(k, δ, L) holds, with effective rank and norm concentration emerging as central conditions.The section combines necessary and sufficient conditions across sub-Gaussian and bounded-4+ε-moment settings.Source: §5 · p. 13, §5.1 · p. 14

5.1 Effect of λ on the condition number

• Three regularization regimes control A_k: sufficiently large positive λ, λ above a lower spectral bound, or λ = −µ + ♦ with ♦ proportional to the spectral gap.The third regime can use a smaller-order ♦ when the spectrum is very well concentrated, but it lies near the critical negative value and the bounds may not always be sharp.Source: §5.1 · pp. 13–14

5.2 Informal necessary conditions

• For λ = 0, a constant-factor spectral condition requires the high effective rank condition ρ_k > c and concentration of tail norms across data points.The tail norm must lie within a constant factor of a fixed value with probability (1 −δ)^(1/n).Source: §5.2 · pp. 14–15

5.3 Controlling condition number under sub-Gaussianity

• Under sub-Gaussianity, constant conditioning requires effective rank bounded below by a constant plus a small-ball condition lower-bounding regularized tail norms.Sub-Gaussianity controls the largest eigenvalue but does not prevent degeneracy of the smallest; the necessary and sufficient constants have a gap.Source: §5.3 · p. 15

5.4 Heavy-tailed case

• Under Theorem 4’s two tail assumptions, with probability at least 1 −cn^(1−h/4) −nδ, the norm concentrates within a constant factor, while heavy-tailed effective rank should exceed n.Source: §5.4 · p. 16

6.1 Upper bound

• The upper-bound proof separates spiked and tail bias and variance terms, then combines algebraic inequalities with concentration of sample covariances and i.i.d. sums.Sub-Gaussianity is used through these concentration inputs, while the algebraic component requires A_k to be positive definite; the resulting bounds apply when A_k is positive semidefinite under nonnegative regularization.Source: §6.1 · pp. 17–18
• The proof strategy explicitly separates essentially low-dimensional and high-dimensional parts, enabling bounds that remove coordinate independence and control the bias term.The key device is to isolate the first k coordinates and use uniform convergence there, with an algebraic identity giving access to the spiked-part error.Source: §6.1 · pp. 17, 19

6.2 Lower bounds

• The lower bounds match the upper bounds under the effective-rank condition, but the variance lower bound requires independent components and the bias lower bound requires PriorSigns and StableLowerEig.Independence can leave a gap in particular upper-bound cases, yet does not generally improve the upper bounds; bias lower bounds cannot improve without information about how signal signs interact with the data distribution.Source: §6.2 · pp. 19–20

6.3 Connecting upper and lower bounds

• The bounds identify the correct split index as k ≤ k∗, while sub-Gaussian data allow any controlled k < n to be shifted to k∗ with tight bounds but worse constants.This resolves the problem that condition-number control may hold for an incorrect k, as in isotropic settings where many coordinates can be excluded; Lemma 11 transfers the assumptions from k to k∗.Source: §6.3 · p. 21

6.4 The role of sub-Gaussianity

• Sub-Gaussianity is not essential for Corollary 6: weak moment assumptions can replace its concentration inputs, but tightness then requires selecting the appropriate k and controlling the operator norm at k∗.Heavy-tailed analogues can provide the needed concentration, while additional work is required to bound ||x_k∗:∞|| and relate r_k to r_h,k.Source: §6.4 · p. 22

7.1 Alternative form of the bound and its relation to classical in-sample analysis

• Theorem 10 yields alternative bias and variance bounds, up to constant factors, when ρ_k is bounded or k = k∗.These expressions closely resemble classical in-sample ridge-regression bias and variance formulas.Source: §7.1 · p. 22
• Equations (6)–(7) arise from classical in-sample risk formulas by replacing empirical eigenvalues with population eigenvalues and increasing λ by the tail energy P_i>k λ_i.The alternative form therefore provides a population-spectrum interpretation of the bounds.Source: §7.1 · p. 23
• The alternative form also interprets the bias term as population ridge-regression bias with infinite data, using an effective regularization level λ + P_i>k λ_i.The population-ridge bias equals the alternative bias when the population regularization is set to nλ_k+1ρ_k = λ + P_i>k λ_i.Source: §7.1 · p. 23

7.2 Dependence on λ

• For k < n/c and k∗ < k, Lemma 12 implies that every k in [k∗, n/c] gives the same Equations (6)–(7) bounds up to a constant factor.This lets larger λ values be analyzed without decreasing k from its initial k∗, yielding sharp bounds with explicit λ dependence.Source: §7.2 · p. 23
• When λ > 2 P_i>k λ_i and all eigenvalues of A_k equal λ up to a constant multiplier, Corollary 13 gives the corresponding result without requiring k∗.Under λ > 2 P_i>k λ_i, λ/n is equal to ρ_kλ_k+1 up to a multiplicative constant.Source: §7.2 · pp. 23–24

7.3 Comparison with other results

• Compared with Hsu et al. (2014), Corollary 13 has the same form up to constants but applies over a wider λ range when n is sufficiently large.The comparison uses effective dimension d(λ/n): the earlier result restricts to d(λ/n) = O(n/log n), whereas Corollary 13 applies when d(λ/n) = O(n).Source: §7.3 · p. 25
• In the interpolating regime λ = 0, the bounds of Hastie et al. (2020) match Equations (6)–(7) up to constants when ˜V ≤ 1 − 1/c and ρ_kλ_k+1 is comparable to ˜λ.The paper notes that this agreement is surprising because the regimes considered by the two analyses are significantly different.Source: §7.3 · pp. 25–26

• Negative regularization cannot improve noise damping because V decreases with λ, so noise must be sufficiently small for it to be beneficial.Source: §8 · p. 27
• Tail signal must also be sufficiently small: negative λ can amplify tail-induced noise, making estimation error for leading components dominate the error from leaving tail components unestimated.Source: §8 · pp. 26–27
• Negative regularization can reduce excess risk by more than a constant factor only at the critical choice λ = −P_i>k λ_i, where the leading-component weighting changes materially.The comparison is otherwise indistinguishable up to a constant multiplier in the section’s risk bound.Source: §8 · p. 27
• The proof uses independent components to obtain tight eigenvalue concentration, then combines a uniform lower bound for all λ ≥ 0 with an upper bound achieved by some λ < 0.These bounds are stated in Lemmas 17 and 18 and yield the sufficient condition in Theorem 19.Source: §8 · pp. 27–28
• Theorem 19 shows that the risk-minimizing λ is negative with high probability when effective rank has a sharp jump, variance is small, and tail signal is sufficiently weak.These conditions are sufficient but not known to be necessary outside the critical regime covered by Assumption NoncritReg(k, γ).Source: §8 · p. 28

First category

• Unlike asymptotic random-matrix studies, this approach needs no limiting spectral distribution, eigenvalue separation, or coordinate independence, and covers p = ∞ and fixed-kernel regression.Those studies distinguish constant error rates, whereas this work investigates when error can be below a constant.Source: §9 · p. 29

Second category

• Featurized and kernel-regression works assume regular data, near-spherical kernels, and n large relative to d^α, whereas this paper directly assumes feature-vector properties.In their shared regime, kernel regression behaves like ridge regression on polynomial features through degree α; the proof separates leading eigendirections from regularizing remainder directions.Source: §9 · pp. 29–30

Third category

• The paper generalizes Bartlett et al. (2020), contains Kobak et al. (2020)’s one-spike negative-ridge result as k = 1, and supplies broader sufficient conditions for negative regularization.Section 8 establishes optimal negative regularization for a rich set of covariance structures.Source: §9 · p. 30
• Compared with Chinot and Lerasle (2021), the paper’s tight bias bound scales with ∥θ∗∥Σ rather than ∥θ∗∥, while their variance bound is larger but holds with smaller probability.Thus their bias bound can diverge while this paper’s remains finite, and this paper’s bias bound is no worse than theirs up to a constant.Source: §9 · p. 30
• The results are not directly comparable to Derezi´nski et al. (2019, 2020) because they study different quantities or norms, despite overlap through projected bias.Derezi´nski et al. (2019) analyze approximation error, while Derezi´nski et al. (2020) use Loewner-order projector bounds and this paper measures bias in the Σ-norm.Source: §9 · p. 30

10. Conclusions

• Ridge regression’s excess risk is governed by data geometry, which determines both the learned signal component and the degree of noise damping.Source: §10 · p. 30
• For a range of regularization values, learning combines classical ridge regression on the first k components with the zero estimator on the remaining components.The two regions are termed the “essentially low-dimensional part” and “essentially high-dimensional part.”Source: §10 · p. 30
• Negative regularization is optimal under sufficient conditions of small noise, small low-dimensional signal energy, and an abrupt effective-rank jump.Source: §10 · p. 30
• The proof separates an algebraic component, valid with probability 1 for non-negative regularization, from a probabilistic component using standard high-dimensional concentration results.This decomposition makes the estimator’s terms easier to trace and supports the paper’s geometric interpretation.Source: §10 · p. 31

A.1 Sub-Gaussianity

• The paper defines sub-Gaussian random variables and sub-Gaussian norms for random vectors as background for its probabilistic analysis.Source: §A.1 · p. 32

A.2 Standard mathematical objects

• Standard notation includes matrix entries, Euclidean, operator, and Frobenius norms, traces, unit spheres, identity matrices, PSD quadratic forms, and ordered eigenvalues.Source: §A.2 · p. 32

A.3 Data and the learning procedure

• The data model uses responses y = Xθ∗ + ε, an independent test draw x, diagonal covariance Σ, and sub-Gaussian design rows.These assumptions define the signal-plus-noise regression setting and the distribution used for test evaluation.Source: §A.3 · p. 32

A.4 Splitting the coordinates

• The analysis splits coordinates at k < n, with matrix, vector, and covariance notation distinguishing the first k components from the remainder.This partition supports separate treatment of the leading and trailing coordinate groups.Source: §A.4 · p. 33

B. Ridge regression

• For overparameterized regression, the estimator is extended continuously through λ = 0 using Xᵀ(λI_n + XXᵀ)^−1y while λI_n + XXᵀ remains positive definite.At λ = 0, this expression equals the minimum-norm interpolating solution; the original formulation is singular at zero and lacks a solution for λ < 0.Source: B. Ridge regression · pp. 33–34
• The ridge estimator’s mean-squared error separates into a noiseless bias term and a pure-noise variance term, and the paper seeks sharp non-asymptotic bounds for both.Linearity in y enables the decomposition into signal and noise contributions.Source: B. Ridge regression · p. 34
• With sub-Gaussian noise, the variance quadratic form is controlled by its expectation with high probability, and that expectation scales linearly with the noise variance.This motivates studying a purified variance term that decouples the noise magnitude from the remaining analysis.Source: B. Ridge regression · p. 34

C. Concentration inequalities

• The concentration results bound non-standard quadratic norms of sub-Gaussian vectors by splitting a decreasing eigenvalue sequence into blocks whose values decrease geometrically.The blockwise argument yields sub-exponential control, with the summed norm bounded by 2Cσ^2λ1.Source: C. Concentration inequalities · pp. 35–36
• Additional lemmas provide high-probability concentration for sums of squared norms of independent isotropic sub-Gaussian rows and for PSD quadratic forms in independent centered sub-Gaussian noise.The latter is a weakened Hanson-Wright inequality with probability at least 1 − 2e^−t/c for t > 1.Source: C. Concentration inequalities · p. 36

D. Controlling the singular values

• Under sub-Gaussianity and independence, Lemmas 23–24 control the off-diagonal Gram-matrix norm and the full matrix norm with exponentially high probability.The proof combines decoupling, net arguments, and concentration bounds for independent sub-Gaussian vectors.Source: D. Controlling the singular values · pp. 37–39
• Lemma 3 gives sufficient and necessary conditions, under NoncritReg(k, γ), for controlling the condition number µ1(Ak)/µn(Ak) by L.The sufficient condition is expressed through spectral and row-norm bounds, while the necessary direction assumes a lower-probability bound on µn(Ak).Source: D. Controlling the singular values · pp. 39, 41, 42
• The condition-number criterion can be reduced to a bound involving ρk, L, λk+1, row norms, and a free parameter ζ.This reduction yields a probabilistic sufficient condition when ρk exceeds L^2, with constants depending on the model parameters.Source: D. Controlling the singular values · pp. 41–42
• Lemma 25 establishes high-probability bounds under NoncritReg(k, γ) and CondNum(k, δ, L), combining eigenvalue-sum and spectral estimates.Its conclusion holds with probability at least 1 − δ − 2e^(-ct) for t ∈ (0, n).Source: D. Controlling the singular values · pp. 43–44
• Lemma 11 shows that the index k can be replaced by k∗: assumptions at any k ≥ k∗ imply the corresponding assumptions at k∗ with adjusted constants and failure probability.The transformed condition is NoncritReg(k∗, γ) with CondNum(k∗, δ + ce^(-n/c), L′).Source: D. Controlling the singular values · pp. 44–46

E. Lower bounds

• Because bias and variance can be written as sums of nonnegative series, Lemma 26 enables separate lower bounds for each term and their combined total.This provides a convenient mechanism for proving high-probability lower bounds on both contributions.Source: E. Lower bounds · p. 46
• For lower bounds, Lemma 26 converts coordinatewise high-probability lower bounds on nonnegative random variables into a high-probability lower bound for their weighted sum.If each ηi exceeds ti with probability at least 1 − δ, the aggregate bound holds with probability at least 1 − 2δ.Source: E. Lower bounds · p. 46

E.1 Variance term

• The variance term admits a high-probability lower bound under NoncritReg(k, γ) and IndepCoord, based on controlling eigenvalues of leave-one-out matrices.At least n − k eigenvalues of A−i have bounded magnitude, enabling the lower-bound argument.Source: §E.1 · pp. 46, 48

E.2 Bias term; Proof

• The bias term has a high-probability lower bound under PriorSigns(θ̄) and StableLowerEig(k, δ, L), with expectations taken over the prior on θ∗.The proof uses Sherman–Morrison identities, diagonalization after prior averaging, and nonnegative-term bounds.Source: §E.2 · p. 49, Proof · pp. 49–50

F. Deriving a useful identity; F.1 Derivation in the ridgeless case; F.2 Checking for the case of non-vanishing regularization

• The analysis splits covariance principal directions into a small-dimensional subspace and a high-dimensional remainder, optimizing the remainder first to derive a useful identity.This identity is checked both for the ridgeless minimum-norm interpolant and for nonzero regularization.Source: F. Deriving a useful identity · p. 52, §F.1 · p. 52, §F.2 · p. 52

G. Variance

• For k < n with A_k positive definite, Lemma 27 bounds the variance term through the spectrum of A_k and concentration of the corresponding sample covariance matrix.The trace estimates rely on isotropic-vector concentration when k is small compared with n.Source: G. Variance · p. 53

G.1 First k components

• The first k variance components are bounded by extracting the norm from the derived identity, applying Cauchy–Schwarz, and averaging over independent noise.Noise averaging leaves a trace in the numerator, followed by a positive-definite matrix trace inequality.Source: §G.1 · pp. 53–54

G.2 Components starting from k + 1-st

• The variance components from the (k + 1)-st onward are bounded similarly, using noise independence and A ⪰ A_k to control the inverse spectrum.The argument again applies tr(MM′M) ≤ µ1(M)^2 tr(M′) for positive-definite matrices.Source: §G.2 · p. 54

H. Bias; H.1 First k components

• For k < n with A_k positive definite, Lemma 28 provides an absolute-constant bound for the bias term by separating the first k coordinates from the remainder.The first-k contribution is rewritten using the error vector and bounded through singular-value comparisons.Source: H. Bias · p. 55, §H.1 · p. 55

H.2 The rest of the components Σ

• The remaining bias components are controlled by decomposing A into A_k plus the first-k contribution and applying Sherman–Morrison–Woodbury and positive-semidefinite norm bounds.For λ > 0, the relevant PSD matrix has norm at most 1, and combining the bounds yields the result.Source: §H.2 · pp. 56–57

I.1 Upper bound on the prediction MSE

• Theorem 5 gives a high-probability prediction-MSE upper bound for k < n/c whenever A_k is positive definite.The constant c depends only on σ_x, and the guarantee holds with probability at least 1 − c e^(−n/c).Source: §I.1 · p. 57

Proof

• The proof of the upper bound controls its bias and variance terms through concentration of sub-Gaussian sample covariances, squared norms, and signal-related quantities.These concentration steps yield simultaneous inequalities that are substituted into the earlier bias-and-variance bounds.Source: §I.1 · pp. 57–60

I.1 Upper bound on the prediction MSE

• Under NoncritReg(k, γ) and CondNum(k, δ, L), Corollary 6 ensures ρ_k > c^−1 and gives the resulting high-probability bound.The guarantee applies for k < n/c and has probability at least 1 − δ − c e^(−n/c), with c depending on σ_x, γ, and L.Source: §I.1 · p. 60

I.2 Upper bound matches the lower bound

• The upper bound from Theorem 5 matches the lower bounds from Lemmas 7 and 8 for suitable k, under the theorem’s alternatives involving ρ_k.The result applies when either ρ_k lies in (a, b) or k is the first index with ρ_k > b; controlling the condition number at a smaller index permits choosing such k.Source: §I.2 · p. 62

I.3 Alternative form of the main bound

• Lemma 12 supplies an alternative form of the main bound when k < n/c and k* < k, using the condition ρ_k* > b.Its proof converts the bound through the relation k − k* < n/c and the corresponding spectral-threshold condition.Source: §I.3 · p. 65, Proof · p. 65

Proof

• Corollary 13 derives the alternative-form result by selecting γ = 0 and suitable constants, then applying Theorem 1, Theorem 10, and Lemma 12.The argument transfers bounds from an index k* to an initial index k̃ while retaining a high-probability event.Source: Proof · pp. 65–66
• Lemma 14 establishes the auxiliary lower bound d(λ/n) ≥ n^2 / max(2, (c + 1)^2) when n ≥ c^2 + c.The proof follows from the elementary inequality n/c − 1 ≥ n/(c + 1).Source: Proof · p. 67

Non-negative regularization

• Under IndepCoord and PriorSigns(¯θ), any λ ≥0 incurs the lemma’s high-probability lower bound on bias when k = min{κ : ρκ(0) > b} is positive.The proof reduces the uniform λ ≥0 case to λ = 0 because the relevant lower-bound expression is non-decreasing in λ.Source: J. Negative regularization · pp. 68–69

Negative regularization

• Some λ < 0 satisfies the cited result with probability at least 1 − ce−n/c; for k < n/c2, the probability is at least 1 − c1e−n/c1 − c2e−n/c2.Source: J. Negative regularization · pp. 69–70
• The chosen negative regularization balances bias in the first k components against tail bias and variance, with separate constructions for small- and large-noise regimes.The proof selects the balancing parameter differently depending on whether noise is small or large, while preserving the required high-probability risk bound.Source: J. Negative regularization · pp. 71, 73