PaperlayerView exact revision on arXiv

Source-linked AI summary

Copula Active Subspaces II: Error Decomposition, A Posteriori Estimation, and Sharpness of the Bounds

Joshua Chen, Peter Jan van Leeuwen

arXiv:2610.14498v1stat.MEmath.STstat.COstat.MLstat.TH

TL;DR

CAS approximates a non-Gaussian observation-noise density from residuals, and the resulting noise-density KL divergence bounds posterior KL divergence averaged over the true predictive distribution. This paper decomposes that error and develops a priori and residual-sample estimates; across 11 of 12 example–sample-size combinations, the estimated sum exceeds the reference divergence.

  • Problem

    When the noise law is unknown, the CAS density estimate cannot be compared with a reference, motivating residual-sample estimates of its error contributions.

    Source: §1 · p. 1

  • Method

    The paper decomposes the noise-density KL error into marginal, projection, and reduced-density terms, then bounds or estimates them using a priori analysis and residual samples.

    Source: §8 · p. 15

  • Results

    In 11 of 12 combinations of example and sample size, the estimated marginal and reduced-density terms plus the trace bound exceed the reference divergence after averaging over estimated models.

    Source: §8 · p. 15

  • Takeaways & Limitations

    The error decomposition enables assessment of CAS density-estimation error from residual samples when the true noise law is unavailable.

    Source: §1 · p. 1, §8 · p. 15

    The score-error correction is the dominant contribution to projection-bound looseness, and the saturation assumption for its Hermite estimate cannot be verified from two fitted scores alone.

    Source: §4.2 · p. 9, §7 · p. 14

Abstract

from arXiv · show

The Copula Active Subspaces (CAS) method of Part I estimates a non-Gaussian observation-noise density $\pi_{\boldsymbol{n}}$ from residual samples. Its KL divergence $\mathcal{D}_{\mathrm{KL}}(\pi_{\boldsymbol{n}}\,\|\,\widehat\pi_{\boldsymbol{n}})$ admits an exact decomposition into marginal, projection and reduced-density terms and bounds the posterior KL divergence averaged over the true predictive distribution. We derive a priori bounds for these terms, together with estimates or bounds from residual samples. Weighted nearest-neighbour entropy estimates are used for the marginal and reduced-density terms, with asymptotic standard errors in exact rank coordinates. For the projection error, the score is fitted on one subsample, and its mean squared magnitude outside the retained subspace is estimated on the other. Corrections for score and rank-transform errors then give a theoretical upper bound with probability at least $1-\delta$. Tests on three known noise laws in dimension $d=20$ use the estimated rank transform. In $11$ of $12$ combinations of example and sample size, the sum of the two estimated terms and the trace bound exceeds the reference divergence after averaging over the estimated models. In these tests, score-estimation error is the largest contribution to looseness. The overestimate from the Gaussian log-Sobolev inequality alone is at most a factor of $1.98$.

1. Introduction.

Part II decomposes the CAS noise-density error into marginal, projection, and reduced-density contributions, then develops bounds and residual-sample estimates for those terms. Its examples show that projection and score-estimation errors are important sources of looseness.

1. Introduction.

  • The exact decomposition assigns marginal, projection, and reduced-density errors to the corresponding CAS estimation steps.The marginal term comprises d one-dimensional divergences and two corrections; the reduced-density term compares r-dimensional laws, while the projection term is bounded using copula-score variance outside the retained subspace.Source: §1 · p. 1
  • A direct entropy estimate of the full noise-density KL divergence faces a dimension-dependent minimax rate, while the decomposition limits entropy calculations to dimensions 1 and r.For densities on [0, 1]^d in Lipschitz balls of smoothness s = 2, the cited minimax root-mean-square error has order (N log N)^−2/(d+2) + N^−1/2.Source: §1 · pp. 1–2
  • For N ≥ 12,500, Eproj is the largest term in the examples, and the Gaussian log-Sobolev bound overestimates it by a factor of 1.64 to 1.98.The tighter bound's gradient-descent subspace differs from CAS's top-r eigenbasis by ∥sin Θ∥F ≤ 0.0004; Hermite score truncation accounts for most remaining a priori-bound looseness.Source: §1 · p. 2
  • Weighted nearest-neighbour estimators address the marginal and reduced-density terms, while a split-sample score estimate supports a probabilistic projection-error bound with score and rank-transform corrections.Under the stated assumptions, the projection bound holds with probability at least 1 − δ; nested multi-index-set increments estimate Hermite truncation error only under a saturation assumption, not sampling or regularization error.Source: §1 · p. 2

2. The estimator and its exact error decomposition.

CAS transforms the noise to Gaussian-marginal rank coordinates, fits a copula score to select a retained subspace, and fits a reduced density there. The resulting KL error decomposes into marginal, projection, and reduced-density terms, with corrections for rank transformation and normalization.

2. The estimator and its exact error decomposition.

  • Componentwise CDF-to-Gaussian transforms give the noise standard Gaussian marginals, and CAS replaces the unknown CDFs with rank-based estimates.The marginal densities are estimated separately with Gaussian-kernel estimates.Source: §2 · p. 2
  • Stage 1 fits the copula score by Hermite score matching and selects a top-r eigenbasis of its empirical second-moment matrix.The retained coordinates U and complementary coordinates W separate the reduced subspace from its orthogonal complement.Source: §2 · p. 2
  • Stage 2 fits a regularized, constrained score in r variables and normalizes the reduced factor before composing it with the estimated rank transform and marginal densities.Because empirical rank transforms and kernel marginal estimates do not generally match, the composed density requires an additional normalizer.Source: §2 · p. 3
  • The CAS noise-density KL error decomposes into Emarg, Eproj, and Edens, corresponding to marginal, subspace, and reduced-density errors.Emarg includes the coordinatewise marginal divergences, a rank-transform correction, and the composition-normalization correction; Eproj is a conditional divergence from a Gaussian reference.Source: §2 · p. 3

3. A priori error bounds.

The a priori analysis bounds marginal, subspace, and reduced-density errors, then combines them in an overall bound. The bounds retain terms tied to approximation, regularization, constraints, and the estimator’s sampling setup.

3.1. The marginals.

  • The marginal bound includes a normalization correction, log bZcomp, which is computed for each estimate rather than bounded a priori.Apart from this correction, the bound tends to zero as N increases under the stated assumptions.Source: §3.1 · p. 4, §3.4 · p. 6

3.2. The subspace.

  • The Stage-1 bound separates score approximation, rank-r truncation, and subspace estimation errors; the last is O_p(N^-1/2) + O(τ_N), with a nonvanishing fixed-ridge term.A nonzero Hermite truncation error alone does not prevent subspace recovery when C and C_Λ have rank r and the interaction order is unrestricted.Source: §3.2 · p. 5

3.3. The reduced density.

  • The Stage-2 KL bound includes truncation, estimation, regularization-bias, and constraint contributions, and requires a positive log-Sobolev constant for the fitted density.The analysis does not bound the constraint distance or the log-Sobolev constant; the truncation bound tends to zero only if the excluded linear Hermite coefficients vanish.Source: §3.3 · pp. 5–6

3.4. The overall a priori bound.

  • The overall bound combines the marginal, subspace, and reduced-density bounds, with the reduced-density contribution evaluated for the chosen basis.The a priori result assumes independent Stage-2 observations after conditioning on the Stage-1 sample.Source: §3.4 · p. 6
  • The overall expected-error bound relies on an independent Stage-2 sample and exact rank transforms; it has not been proved for the implementation using the same sample and an estimated transform.The implementation’s fitted densities retain their stated individual bounds, but the overall bound (3.10) is unproved for that setting.Source: §3.4 · p. 6

4. A posteriori error estimation.

The a posteriori analysis estimates the marginal and reduced-density errors from held-out residuals and bounds the projection error using a split-sample score estimate. These components form an overall estimate, but omitted corrections mean it is not guaranteed to upper-bound the KL divergence.

4.1. The marginals and the reduced density.

  • Weighted nearest-neighbour entropy estimates target the reduced-density and one-dimensional marginal terms, while fitted log-density contributions use negative sample means.The normalizer is computed by Sobol’ quadrature; its convergence rate alone gives no numerical error bound for 8192 points.Source: §4.1 · p. 7
  • Under the stated conditions, the reduced-density and marginal estimates converge in probability and have asymptotic standard-error estimates in exact rank-Gaussianized coordinates.The standard-error estimate sums square roots of consistent variance estimates and divides by √M.Source: §4.1 · pp. 7–8
  • The convergence result does not directly cover the implemented rank transform, whose held-out image law has atoms and lacks a density.The transform is constant beyond the estimation-sample range, so the corollary’s hypotheses do not hold for that image law.Source: §4.1 · p. 8

4.2. The subspace.

  • A split-sample score procedure bounds the projection error with probability at least 1−δ, adding corrections for score-estimation and rank-transform errors.The fitted score and rank transform use one half of the sample; the other half estimates the score magnitude outside the retained subspace.Source: §4.2 · pp. 8–9
  • The projection-error method avoids estimating a dimension-16 entropy by bounding the score outside the retained subspace with a Gaussian-reference inequality.The bound adds corrections for score estimation and the rank transform.Source: §4.2 · p. 8
  • The hierarchical projection-error bound assumes saturation: enlarging the approximation space reduces score error by a factor β<1, which two fitted scores cannot verify.The score-error correction also depends on the increment η and β; its penalty-dependent O(τN) term need not vanish at fixed penalty strength.Source: §4.2 · pp. 9–10

4.3. The overall estimate.

  • The overall a posteriori estimate combines estimated marginal and reduced-density terms with a score-trace bound, but omitted corrections can make it fall below the KL divergence.The score-trace probability statement does not apply to the total estimate.Source: §4.3 · p. 10

5. Sharpness of the bounds.

The Gaussian-reference bound is close to the divergence in the tested examples, though its looseness depends on the example and the chosen subspace. Tightening the bound or optimizing it changes the selected basis little relative to Monte Carlo variation.

5. Sharpness of the bounds.

  • The top-r eigenspace minimizes the Gaussian-reference bound, but the divergence’s minimizing subspace generally differs from it.The experiments separate looseness of the Gaussian-reference bound from error due to approximating the score with Hermite polynomials.Source: §5 · p. 10
  • 1.91, 1.64, and 1.98: the Gaussian-reference bound exceeds the divergence at its minimizing subspace by these factors on Examples 1, 2, and 3.At the estimated subspace in Example 1, the ratio is between 1.38 and 1.45 for every N≥2,500.Source: §5 · p. 10
  • Replacing the Gaussian-reference bound with the tighter bound reduces it by less than 0.5% on each problem.Gradient descent moves the basis by at most ∥sin Θ∥F≤0.0004, less than the separation between the two Monte Carlo eigenbasis estimates.Source: §5 · p. 10
  • The two bounds closely agree in these examples because twelve independent standard-Gaussian coordinates contribute unit eigenvalues to the complement.The estimated eigenvalues for those coordinates lie in [0.995, 1.024].Source: §5 · p. 11

6. Numerical results.

Across three d=20 examples, the numerical study compares estimated error terms and bounds with reference values. Mean estimates exceed the reference total in 11 of 12 example/sample-size combinations.

6.1. The three example problems.

  • The three d = 20 examples have four coupled latent coordinates, and r = 4 matches them; rank-transforming mixed coordinates makes the rank-four reference projection error nonzero.The examples use smooth noise densities and copula scores, with the remaining coordinates independent standard Gaussians.Source: §6.1 · p. 11

6.2. Reference values.

  • Reference projection and reduced-density values are computed using quadrature over discarded coordinates and Monte Carlo averaging over retained coordinates.The reference basis uses a top-r eigenspace estimated from Nref = 10^6 draws; the reduced-density terms use the Stage-1 reference basis.Source: §6.2 · p. 11

6.3. The three terms of the decomposition.

  • Across all three examples, the marginal term decreases with N, while the subspace term is largest for N ≥12,500 and changes by less than 4% from N = 12,500 to 200,000.At these sample sizes, accuracy is limited by the rank and multi-index set.Source: §6.3 · p. 11

6.4. A priori terms and the a posteriori bound.

  • At N = 5 × 10^4, the Stage-1 projection bound is 16.9–27.0 times reference Eproj, with the Hermite term contributing more than half the bound.The Hermite term is largest on every example; only the statistical term depends on N and it decreases as N grows.Source: §6.4 · p. 12
  • The Corollary 4.2 bound exceeds its target trace for all 96 models, and its finite-sample term decreases as N grows.The target trace is estimated using an independent set of 2 × 10^5 residuals mapped through the first-half rank transform.Source: §6.4 · p. 12, Figure 2 · p. 13
  • At N = 5 × 10^4, corrected-bound ratios to reference Eproj range from 21.8 to 34.4; interaction-order enlargement gives tighter Hermite-error bounds than degree enlargement (ratio ≤6 versus >29).For N ≥12,500, score error eN is the dominant term in the corrected bound.Source: §6.4 · p. 12

6.5. The overall estimate against the reference values.

  • In all 150 models, bEmarg lies within twice the combined standard errors of reference Emarg, while bEdens is biased upward by about one combined standard error for N ≥2,500.At N = 500, the reduced-density estimate's mean excess is at most 2.4 combined standard errors.Source: §6.5 · p. 13
  • Mean estimates exceed the reference total in 11 of 12 example/sample-size combinations; the exception is Example 3 at the largest sample size shown.There, the trace bound is below reference Eproj because the estimated score captures little complement-direction score variance and the estimate omits score error eN.Source: §6.5 · p. 13

7. Limitations.

The analysis has important limits from unverifiable assumptions, finite-sample score error, and the fixed linear-subspace representation. Its experiments also use known noise laws and do not test operational assimilation residuals.

7. Limitations.

  • The estimated-transform image law has atoms, so the exact-coordinate smoothness result and the identity requiring a density do not apply.Residual samples also cannot verify the smoothness assumptions or transform condition required for the marginal rate.Source: §7 · pp. 13–14
  • The figure’s effectivity index is the estimate-to-reference ratio, and its diamond–bar gap comes from the Chebyshev term.The vertical axis is logarithmic.Source: §7 · p. 14, Figure 3 · p. 14
  • Score-estimation error is the dominant term in the projection bound, and the Hermite estimate assumes (SAT), which two multi-index sets alone cannot verify.On Example 3, increasing interaction order by one multiplies Tr(CΛ) by 3.7, while increasing degree raises it by only 0.5%.Source: §7 · p. 14
  • At N = 500, the bounded contribution exceeds the reference total by more than 100-fold on every example, although the two estimated-term errors are much smaller.Only 250 samples are available to estimate the Stage-1 score at this sample size.Source: §7 · p. 14
  • The reduction represents dependence through a fixed linear subspace, so dependence directions varying along a curved set are represented only through variation along that subspace.The analysis also fixes the truncation and regularizer in advance rather than covering choices that change with N.Source: §7 · p. 14
  • The reference comparisons require closed-form noise laws, so the experiments do not assess the error estimate on residuals from an operational assimilation system.Transport-based reductions [11] are mentioned as alternatives for laws with curved dependence directions.Source: §7 · pp. 14–15

8. Conclusion.

The paper decomposes the density-estimation error and develops sample-based estimates and bounds for its components. In tests with known noise laws, the overall estimate exceeded the reference divergence in 11 of 12 example–sample-size combinations, while score truncation and score-estimation error remained important sources of looseness.

8. Conclusion.

  • The exact decomposition separates marginal, projection, and reduced-density contributions, with sample-based estimates or bounds for each.The projection bound combines a fitted-score trace bound outside the retained subspace with corrections for score-estimation and rank-transform errors.Source: §8 · p. 15
  • 11 of 12 example–sample-size combinations had an averaged overall estimate exceeding the reference divergence.Across 150 models, the marginal estimate agreed with its reference value and the reduced-density estimate had a small upward bias.Source: §8 · p. 15
  • The Gaussian log-Sobolev bound was 1.64 to 1.98 times the divergence on the three examples, while score truncation contributed more than half of every a priori bound.The tighter bound’s numerically descended basis was within ∥sin Θ∥F ≤ 0.0004 of CAS’s top-r eigenbasis in each example.Source: §8 · p. 15
  • For N ≥ 12,500, the subspace term was largest on every example and changed little with further sampling; rank and multi-index truncation limited accuracy.The trailing-eigenvalue sum and, under saturation, increments between nested multi-index sets help assess whether to increase rank or enlarge the index set.Source: §8 · p. 15
  • Code, saved numerical results, and reproduction instructions are available in the repository, with release v1.0-part2 used for the article.The reproduction script generates figures from saved results and can recompute them; Supplement SM4 records settings and random states.Source: Reproducibility · p. 15

Appendix A. Proofs of the decomposition and the Stage-1 bounds.

The appendix establishes the decomposition and supporting bounds by connecting density-factorization identities, projection inequalities, and sample-based score estimates. It also gives the finite-sample concentration argument underlying the projection bound.

Appendix A. Proofs of the decomposition and the Stage-1 bounds.

  • For any score field S′, the L2 norm of the score outside the retained projector is bounded using score approximation error plus the projected second-moment trace.Choosing the fitted score gives the projection bound used in Corollary 4.2.Source: Appendix A · p. 15
  • The leading rank-r eigenspace minimizes the retained-tail trace, and a positive eigengap makes the maximizing subspace unique.The suboptimality gap is controlled by the basis error when the eigengap condition holds.Source: Appendix A · p. 16
  • The truncated-score projection bound follows by separating score truncation from basis-estimation error; the latter is bounded at order Op(N^-1/2) + O(τN) without an eigengap assumption.The eigengap is needed only for the sin Θ rate.Source: Appendix A · p. 16
  • Conditioning on the first sample half makes the fitted transform and score fixed, so the held-out score second moments yield the projection trace bound with probability at least 1 − δ.Replacing the population fourth moment with its estimate adds a moment-ratio argument; the proof treats the zero-moment case separately.Source: Appendix A · p. 16
  • Under the saturation assumption, nested multi-index score estimates recover the Hermite truncation contribution up to the stated sampling and regularization errors.The proof expands the bilinear form around its population value and bounds the estimated-score error by the triangle inequality.Source: Appendix A · p. 16
  • The exact identity splits the density KL divergence into marginal KL terms, a reduced copula-density divergence, a rank-transform contribution, and the normalization constant.The transformed density factorization and integrability checks justify taking expectations under the true noise law.Source: Appendix A · p. 17

Appendix B. The Stage-1 score-covariance rate.

The appendix states conditions for the Stage-1 score-covariance rate and derives coefficient and covariance errors from empirical moment control. Empirical-rank coordinates require an assumption for that moment control, while a positive population eigengap supports a basis-rate result.

Appendix B. The Stage-1 score-covariance rate.

  • The score-covariance rate assumes a finite nonconstant dictionary, positive transformed density, empirical moment errors of order Op(N^-1/2), and bounded regularization strength.A positive population eigengap is an additional condition for the estimated-basis rate.Source: Appendix B · p. 18
  • For exact rank coordinates, the empirical moment condition follows from the multivariate central limit theorem; for empirical ranks, it is an assumption.The implementation’s regularization choice contributes an additional Op(N^-1/2) term under that condition.Source: Appendix B · p. 18
  • Positive definiteness of the exact-coordinate Gram matrix follows because a vanishing linear combination of included polynomial gradients would imply a constant polynomial, and the constant mode is excluded.Thus all coefficients in that combination must vanish.Source: Appendix B · p. 18
  • The proof obtains coefficient error Op(N^-1/2) around the regularized solution and regularization bias O(τN), then propagates these errors to the score-covariance matrix.The basis rate uses the population eigengap; no empirical eigengap bound is required.Source: Appendix B · p. 18

Appendix C. Proof of the a posteriori divergence estimate.

These appendices establish estimation and error-control ingredients for the reduced-density estimator, including quadrature, regularization, truncation, and degree-wise variance bounds under stated assumptions.

Appendix C. Proof of the a posteriori divergence estimate.

  • Under the stated conditions, the weighted nearest-neighbour divergence estimate has a central-limit expansion and is consistent.The argument combines an i.i.d. cross-entropy average with an entropy expansion and controls the variance of the log-density difference.Source: Appendix C · pp. 18–19

C.1. Quadrature of the reduced normalizer.

  • For a fixed fitted polynomial, finite Hardy–Krause variation supports a Sobol’ quadrature error bound for the reduced normalizer, but clipping and capping errors are not covered.The polynomial’s even-degree leading term must be strictly negative on the unit sphere; the argument does not bound errors from numerical clipping or capping.Source: §C.1 · p. 19

Appendix D. Stage-2 regularizer: proofs and constant selection.

  • The Stage-2 score-matching problem is a regularized finite-dimensional quadratic whose population normal equations are those of the Fisher projection.The analysis assumes a bounded reduced density ratio and Sobolev regularity; the basis and dictionary are fixed, with exact rank transforms and independent observations.Source: Appendix D · pp. 19–20

D.1. The Stage-2 regularizer.

  • The diagonal Stage-2 penalty combines a sample-size-dependent sampling term with a degree-weighted curvature term, and its constants are selected by validation risk.The degree-squared weighting is fixed rather than adapted to the unknown Sobolev smoothness.Source: §D.1 · p. 20

D.2. Truncation and estimation error.

  • The Stage-2 Fisher error separates into truncation and estimation terms, with Sobolev decay bounding truncation when omitted Hermite coefficients begin at a sufficiently high degree.For the dictionary ΛK2,q2, omitted linear or interaction coefficients can prevent that degree threshold from exceeding K2.Source: §D.2 · p. 21

D.3. Per-class noise bound for the variance regularizer.

  • The per-degree variance bound controls each Hermite mode using the class trace and the regularizer’s sample variance statistic.Under the bounded-density and class-trace assumptions, the canonical-basis variance is bounded by C⋆k²(Cb + CA∥θr,⋆∥²)/N.Source: §D.3 · p. 22

D.4. Error bound for the Stage-2 estimator.

The Stage-2 theorem bounds expected Fisher error for a data-dependent regularizer, retaining truncation, sampling, regularization, and constraint effects under its assumptions.

D.4. Error bound for the Stage-2 estimator.

  • The expected Stage-2 Fisher error is bounded under assumptions of an exact-transform basis, independent observations, a positive-definite population Gram matrix, and a bounded fourth moment.The theorem allows a measurable, nonempty closed convex constraint set, potentially based on the same observations.Source: §D.4 · pp. 22–23
  • The proof handles constraints through projection in the regularized metric and controls the contribution from the event where the empirical Gram matrix is poorly behaved.The stated expectation bound includes an O(N^-1) contribution from that event.Source: §D.4 · pp. 23–24
  • The bound retains truncation and regularization-bias contributions rather than requiring them to vanish as sample size grows.Its sampling scales include 1/Csamp and 1/(N Ccurv) for the two regularizer components.Source: §D.4 · pp. 23–24

Appendix E. Log-Sobolev property of the Stage-2 estimate.

The appendix establishes a tight logarithmic Sobolev inequality for polynomial-potential densities and applies it to the Stage-2 estimate.

Appendix E. Log-Sobolev property of the Stage-2 estimate.

  • A density proportional to exp(−V) satisfies a tight logarithmic Sobolev inequality when V has even degree at least four and a positive-definite leading homogeneous part.The proof establishes tail growth and combines a defective logarithmic Sobolev inequality with a Poincaré inequality.Source: Appendix E · p. 24
  • For the Stage-2 estimate, the polynomial-potential result applies when its effective degree is at least four; the quadratic case is covered by the Gaussian logarithmic Sobolev inequality.The degree-at-least-four condition follows from the imposed negative bound on the leading part of the fitted log density.Source: Appendix E · p. 25

SHARPNESS OF THE BOUNDS

The supplementary results check the subspace-recovery rate, examine reference divergence calculations, and compare a sharper KL bound with the trace bound. On the three examples, the sharper bound removes little of the trace bound’s overestimate.

Stage-1 rate check

  • A fitted mean-error slope of −0.502 matches the N^-1/2 subspace-recovery scaling on the Gaussian-copula check.The check uses a law with zero truncation error; the figure caption specifies d = 20, r = 3, and within-block correlation ρ = 0.7.Source: SHARPNESS OF THE BOUNDS · pp. 27–28, Figure SM2 · p. 28

Reference divergence calculations

  • After rank Gaussianization, the reference calculations show positive projection divergence at the rank-four CAS subspace in each example.Before the transform, the four-dimensional subspace generated by the coupled latent coordinates has an independent standard-Gaussian complement, so retaining it is exact.Source: SHARPNESS OF THE BOUNDS · p. 29

Sharper-bound comparison

  • For centered nonsingular Gaussian noise, the sharper upper bound is exact for every retained subspace.The bound gap vanishes when the orthogonal-complement Fisher-information matrix is the identity.Source: SHARPNESS OF THE BOUNDS · p. 30
  • 1.90 versus 1.91, 1.63 versus 1.64, and 1.97 versus 1.98 are the Jhi and trace-bound factors over reference divergence on Examples 1–3.The sharper bound therefore removes little of the overestimate on these examples.Source: SHARPNESS OF THE BOUNDS · p. 30, Table SM1 · p. 31
  • Gradient descent moves each example’s basis less than the separation between eigenspaces estimated from 10^6 and 2·10^6 samples.The movement is at most ∥sin Θ∥F = 0.0004, while the estimated-eigenspace separations are 0.0014, 0.0019, and 0.0013.Source: SHARPNESS OF THE BOUNDS · p. 31

Error decomposition

  • The plotted component sum is not an exact decomposition comparison: it omits marginal corrections and combines terms evaluated at different bases, alongside reference numerical errors.These differences contribute to the gap between the plotted sum and the reference total.Source: SHARPNESS OF THE BOUNDS · p. 32
  • For Example 2, the marginal term dominates at N ≤ 2500, while the projection term is largest at N ≥ 12,500; their plotted sum nearly matches the reference total at larger sample sizes.The reduced-density term levels off slightly below the projection term.Source: Figure SM3 · p. 33
  • For Example 3, the projection term is largest at every sample size, and the plotted sum nearly matches the reference total for N ≥ 12,500.The marginal term falls below the reduced-density term between N = 2500 and N = 12,500.Source: Figure SM4 · p. 33
Loading 2610.14498v1…