PaperlayerView exact revision on arXiv

Source-linked AI summary

Copula Active Subspaces I: A Score-Covariance Method for Reduced-Order Non-Gaussian Density Estimation

Joshua Chen, Peter Jan van Leeuwen

arXiv:2609.36142v2stat.MEstat.COstat.ML

TL;DR

Non-Gaussian observation noise challenges Bayesian inference because common simplified models omit dependence structure. CAS rank-transforms the noise, reduces its copula using score covariance, and estimates the reduced density. It reports lower noise and posterior KL divergences in its examples.

  • Problem

    Common noise simplifications omit dependence structure, while the copula score covariance can identify directions that ordinary covariance does not distinguish.

    Source: §1 · p. 2, §5.1 · p. 14

  • Method

    CAS rank-Gaussianizes the noise marginals and uses the copula score covariance to identify a reduced subspace for modeling dependence.

    Source: §2 · pp. 3–4, §7 · p. 18

  • Results

    At the largest sample size, posterior KL divergence falls more than sevenfold, and noise KL divergence is lower across the reported examples.

    Source: §7 · p. 18

  • Takeaways & Limitations

    The score-covariance eigenspace gives a rank-r reduction that targets dependence, including when ordinary data covariance does not identify its directions.

    Source: §5.1 · p. 14, §7 · p. 18

    The guarantees concern population quantities; the paper does not bound the error in the estimated score covariance or marginal estimates.

    Source: §6 · p. 18

Abstract

from arXiv · show

In Bayesian inference problems with non-Gaussian observation noise, the posterior is only as accurate as the noise density, and gradient-based samplers need that density and its gradient evaluable pointwise, whether from an explicit expression or from code, and without an inner solve. We propose Copula Active Subspaces (CAS) to represent this noise density. A componentwise rank transform isolates the noise law's dependence in its copula, and a rank-$r$ reduction keeps only the directions along which that dependence varies. These directions are the leading eigenvectors of the copula score covariance $\boldsymbol{C} := \mathrm{Cov}_{\pi_{\boldsymbol{Z}}}(\nabla\log c^{Z})$, which is what makes the reduction a copula active subspace. Because $\boldsymbol{C}$ vanishes when the coordinates are independent, these are directions of dependence, which the covariance of the data need not identify. From this construction follow a Gaussian-reference KL divergence bound with the explicit constant $\tfrac{1}{2}$, minimized over all rank-$r$ reductions by exactly this eigenspace; a diagnostic for the error the reduction leaves behind, computable from the samples alone; and, from Hermite score matching, a reduced log-density and gradient in closed form, with the truncation orders and the Stage-2 regularization constants chosen on validation samples. The reduction replaces a $d$-dimensional density estimation problem by an $r$-dimensional one. On a $d=20$ noise law and a Bayesian inference problem with that noise, CAS lowers noise KL divergence more than fivefold and posterior KL divergence more than sevenfold against Gaussian-copula, product-of-marginals, and PCA-subspace baselines, and lowers noise KL divergence by factors of about $3.5$ and $2.7$ on two further $d=20$ examples.

1. Introduction.

Non-Gaussian observation noise makes Gaussian likelihoods inaccurate, while gradient-based posterior samplers require the noise log-density and its gradient. CAS separates marginals from dependence, reduces the copula to active directions, and estimates the reduced density for posterior use.

1. Introduction.

  • The average posterior divergence under a candidate noise density is bounded by the divergence between the true and estimated noise densities.The KL chain rule establishes this relation for the likelihood setting described in the introduction.Source: §1 · p. 2
  • The paper estimates the noise density itself, using N i.i.d. noise samples or residual-based approximate samples when noise is not directly observed.The samples are taken as given, and their construction is discussed later in the paper.Source: §1 · p. 2
  • A Gaussian likelihood, independent-coordinate model, or Gaussian copula can omit dependence structure and increase noise divergence.The componentwise rank transform gives standard-Gaussian marginals, leaving the copula to represent dependence.Source: §1 · p. 2
  • The reduction uses residual samples rather than forward-map gradients to identify dependence directions in the copula.The related-work discussion distinguishes this from likelihood-informed subspaces, which use forward-map gradients.Source: §1 · p. 2
  • CAS applies a componentwise rank transform, reduces copula dependence to a low-dimensional subspace, and estimates it by Hermite copula score matching.The method returns a reduced log-density and gradient in closed form for gradient-based posterior samplers.Source: §1 · pp. 2–3

2. Bound-optimal dimension reduction with a closed-form diagnostic.

CAS uses the copula score covariance to identify a rank-r reduction and quantify its residual error. Its experiments report substantial noise- and posterior-KL improvements over several baselines.

2. Bound-optimal dimension reduction with a closed-form diagnostic.

  • The Gaussian-reference KL bound has explicit constant 1/2, and its minimum over rank-r subspaces is attained by the leading eigenspace of the copula score covariance.This makes the score-covariance eigenspace the bound-optimal reduction.Source: §2 · p. 3
  • A closed-form diagnostic estimates rank-r truncation error from samples.The diagnostic is sample-computable.Source: §2 · p. 3
  • More than fivefold lower noise KL divergence and more than sevenfold lower posterior KL divergence are reported against Gaussian-copula, product-of-marginals, and PCA-subspace baselines.The comparison uses a d = 20 non-Gaussian copula example and a Bayesian inference problem with the same noise; gains persist across three structurally distinct noise laws.Source: §3 · p. 3

3. Empirical validation.

The paper frames its experiments around three example problems and states population-level properties of the copula score covariance and its bound-optimal subspace.

2. Bound-optimal dimension reduction with a closed-form diagnostic.

  • The experiments use three example problems.The supplied passage does not report their individual outcomes.Source: §3 · p. 3
  • For the true noise law, the copula score covariance is zero exactly under coordinate independence.Its top-r eigenspace minimizes a KL divergence bound among rank-r reductions.Source: §2 · p. 3
  • These statements concern population quantities rather than estimated quantities.The passage explicitly distinguishes this section’s true-law analysis from estimation.Source: §2 · p. 3

2. Copula dimension reduction.

CAS separates marginal estimation from dependence estimation by rank-Gaussianizing the coordinates, then represents copula variation through a reduced subspace. The leading score-covariance eigenspace minimizes a Gaussian-reference KL bound.

2. Copula dimension reduction.

  • The componentwise rank transform preserves the copula while mapping the coordinates to standard-Gaussian marginals.The transformation separates marginal estimation from dependence estimation.Source: §2 · p. 3
  • The copula score covariance measures variation in the log-copula, and it is zero exactly when the coordinates are independent.Along a unit direction, its quadratic form is the mean squared directional derivative of the log-copula.Source: §2 · pp. 3–4
  • A rank-r model uses an orthonormal subspace and trades approximation accuracy against the statistical error of estimating its reduced density.At rank zero the model is the product of marginals; at rank d it contains the exact joint law.Source: §2 · p. 4

2.1. Reduced model form.

  • At a fixed subspace, the KL projection keeps the active-coordinate marginal exact and replaces the conditional complement law with a standard Gaussian.The residual is the active-marginal average of the conditional KL divergence; the reduction is exact when the copula does not vary along the complement.Source: §2.1 · pp. 4–5

2.2. KL divergence upper bound and optimal subspace.

  • The Gaussian-reference bound is at most one-half the trace of the copula score covariance projected onto the discarded directions.The Gaussian reference has sharp log-Sobolev constant 1, giving the prefactor 1/2.Source: §2.2 · p. 5
  • The leading r eigenvectors minimize the bound, and the reduction is lossless when the remaining eigenvalues vanish.This top-r eigenspace is the copula active subspace; the guarantee uses the exact score covariance.Source: §2.2 · p. 5

3. The Cas estimator.

CAS combines separately estimated marginals with a reduced copula factor, learned in a score-informed subspace, to form the noise-density estimate.

3. The Cas estimator.

  • CAS combines estimated marginal densities with a reduced copula factor evaluated on rank-transformed data.The marginal densities use Gaussian kernels, while the reduced factor is estimated on projected coordinates.Source: §3 · p. 6

3.1. Stage 1: subspace identification.

  • Stage 1 fits a copula log-density by Hermite score matching and selects the top-r eigenvectors of the empirical score second-moment matrix.The resulting analytical scores yield the matrix in closed form; a finite-sample diagnostic is computable from the score matrix without further eigenvectors.Source: §3 · p. 6, §3.1 · pp. 7–8

3.3.1. The regularizer.

  • The truncation orders and regularization constants are chosen by minimizing the score-matching objective on validation samples.The two regularization constants are global scalars, and both terms are active at every studied sample size.Source: §3.1 · p. 6, Choice of the two constants · p. 9

3.2. Stage 2: score matching for the reduced density.

  • Stage 2 applies Hermite copula score matching to projected samples, producing a reduced log-density and gradient in closed form.The reduced score is fitted on R^r; another estimator of the reduced factor can be used when an adaptive polynomial approximation is unavailable or inaccurate.Source: §3.2 · p. 8

3.3.1. The regularizer.

  • The regularizer combines a sample-size-dependent variance penalty with a curvature penalty concentrated on higher-degree coefficients.The terms address complementary sources of error and keep the solve well-posed across the studied sample sizes, N in [10^2, 10^5].Source: §3.2 · p. 9, §3.3.1 · p. 9

Choice of the two constants.

The reduced-density estimator uses centering and normalizability constraints in a regularized solve; its behavior depends on regularization, truncation, and—in some settings—the chosen basis.

3.3.2. The two constraints.

  • Centering imposes a mean-zero score identity, while level and leading-form bounds ensure the reduced factor is integrable against the Gaussian reference.The bounds make the normalized reduced factor a probability density.Source: §3.3.2 · p. 9, §3.3.3 · p. 10

3.3.3. The estimator and its solve.

  • The constrained estimator is given by a regularized linear-system solve, requiring a Cholesky factorization and an r × r solve rather than d-dimensional solves.The matrix dimensions depend on the reduced dimension r, not d.Source: §3.3.3 · p. 10
  • The centering constraint projects the unconstrained estimate, and the limiting projection does not increase M-weighted variance.The centered limiting estimate has a regularization-bias term contracted by the projection and a second term determined by the expected truncation residual.Source: §3.3.3 · pp. 10–11
  • Under the stated assumptions, the estimator converges to the centered limiting solution at the Monte Carlo rate, O(N^-1/2).Regularization bias includes a sampling-noise part that decreases with N and a curvature part that need not; truncation bias is N-independent and vanishes when the reduced score is representable in the basis.Source: §3.3.3 · p. 11
  • With exact optimization and integration, the estimate is invariant to basis rotations when the interaction-order limit excludes no multi-index.When that limit excludes multi-indices, the estimate can depend on the basis; numerical search and quadrature also limit invariance to their tolerances.Source: §3.3.3 · p. 11
  • At (Csamp, Ccurv) = (3, 10^-5), held-out log-score stayed within 0.18 nats of the best alternative regularizer across three examples and all tested sample sizes.For the inference problem, posterior KL divergence was within 0.04 nats of the best-N Tikhonov regularizer; its best κ shifted from 3 × 10^-2 at N = 10^2 to 10^-5 at N = 5 × 10^4. Held-out tuning of Csamp improved by more than 0.01 nats only for N ≤ 1000.Source: §3.3.3 · p. 11

4. Error at the estimated Stage-1 subspace.

This section separates subspace-recovery error from rank-r truncation error and gives a sample-based diagnostic for the latter. It also states when a coarse Stage-1 Hermite score fit can identify exact rank-r dependence and where that guarantee stops.

  • The tail sum bE_r estimates the minimum truncation error from samples alone, with its estimation error bounded using leading singular values of bC−CΛ.Source: §4 · pp. 11–12
  • Stage 1 projects the copula score onto a Hermite field and uses the leading eigenspace of CΛ as a benchmark, which need not equal the bound-optimal eigenspace of the true score covariance.Source: §4 · p. 12
  • Over rank-r projectors, the leading eigenspace of CΛ minimizes trace truncation error; estimated-subspace excess is bounded above and below quadratically in the recovery angle.Source: §4 · p. 12
  • Under exact rank-r dependence, a coarse Stage-1 expansion introduces no complement directions and identifies the active span once CΛ has rank r; only Stage 2 must fit the score accurately.Source: §4 · pp. 12–13
  • The guarantee concerns known r under exact rank-r dependence: eigenvalue thresholding is not justified, and q1<K1 can add directions outside the active span.Source: §4 · p. 13

5. Experiments.

The experiments test score-covariance subspace recovery, reduced noise-density estimation, and posterior accuracy on three d=20 noise laws. Cas outperforms the supplied baselines on the primary example, while its subspace estimate improves with sample size.

5.1. Example problems.

  • The three d=20 examples have four coupled latent coordinates, but their mixing makes the dependent directions indistinguishable by covariance; Example 1's two-to-one map yields a symmetric bimodal conditional law.Examples 2 and 3 change the input-output map; their copula scores are harder and easier, respectively, for Hermite expansion than Example 1's.Source: §5.1 · pp. 13–14

5.2. Subspace recovery: score covariance versus PCA.

  • At N = 50,000, higher interaction orders reduce mean subspace angle versus N = 10,000 by factors of 16, 6.0, and 11 on Examples 1–3.Cas angle decreases with sample size, whereas PCA and uniformly random subspace angles are N-independent; the Cas angle also depends on the selected multi-index set.Source: §5.2 · p. 15, Figure 2 · p. 15

5.3. Noise KL divergence of the reduced density.

  • At N = 50,000, Cas reaches mean noise KL divergence 0.128 versus 0.70 for each of PCA-HCSM, PoM, and the Gaussian copula; the same ordering holds on Examples 2 and 3.PCA-HCSM and Cas use the same Hermite basis, regularizer, and quasi-Monte Carlo normalizer, so their noise-KL gap is due to subspace choice.Source: §5.3 · p. 15, Figure 3 · p. 16

5.4. Bayesian inference problem with non-Gaussian observation noise.

  • The posterior experiment uses a 2-dimensional parameter and 20-dimensional observation, with rank r = 4; each method is evaluated on the same observation and residual sample within each realization.Posterior KL is computed on an 80 × 80 grid over [−6, 6]², with each candidate posterior normalized to unit mass on the grid.Source: §5.4 · p. 16
  • At N = 50,000, Cas attains mean posterior KL divergence 0.051 versus 0.40 for PCA-HCSM, PoM, and the Gaussian-copula baseline.For the illustrated observation, only Cas places posterior mass on both preimage branches; each baseline produces a single broad mode.Source: §5.4 · p. 16

6. Limitations and scaling.

The paper identifies limits in the reduction's geometry, regularity assumptions, and error accounting, and notes that usable noise samples may themselves be difficult to obtain.

6. Limitations and scaling.

  • Laws whose density or copula score grows faster than the stated bounds fall outside the analysis.The regularity conditions constrain dependence structure but impose nothing on marginal tails; marginal-estimation accuracy still depends on tails through kernel choice.Source: §6 · p. 17
  • The accounting does not separately measure randomized eigendecomposition error or Stage-2 density-estimation error on the estimated subspace.The eigendecomposition contributes a term to Stage-1 subspace error.Source: §6 · p. 18
  • The reduced model does not preserve the rank transform's separation of marginal and dependence estimation.Its i-th marginal includes a factor that equals one for every reduced factor exactly when row i of Vr vanishes.Source: §6 · p. 18
  • The guarantees concern population quantities: they use the exact score covariance and do not bound marginal-estimation error.The paper also does not bound the distance from CΛ to C.Source: §6 · p. 18
  • Residuals formed at a nominal parameter carry both noise and nominal-parameter error, which cannot be separated from residuals alone.Obtaining usable noise samples is a separate problem.Source: §6 · p. 18
  • Cas applies componentwise rank transforms and rank-r score-covariance reduction to copula dependence; experiments recover the subspace unlike PCA, and posterior KL falls more than sevenfold at the largest sample size.Source: §7 · p. 18

7. Conclusion.

The supplied conclusion material documents reproducibility, while the accompanying appendices state regularity consequences and derive the Hermite score-matching system used by both estimator stages.

7. Conclusion.

  • The reported experiments can be reproduced from released code, saved numerical results, and recorded settings, including random states.The estimator is implemented in Python/NumPy without external learned components.Source: Reproducibility · p. 18

Appendix A. Consequences of the regularity assumption.

  • Under the stated assumptions, the projected marginal inherits a pointwise sub-Gaussian bound, the log-density gradient is square-integrable, and the score covariance has finite trace.These are among the regularity consequences used in the later proofs.Source: Appendix A · p. 19

Appendix B. Score matching: the Hermite linear system.

  • The appendix derives a Hermite score-matching identity and linear system used for Stage 1 in d dimensions and Stage 2 in r projected dimensions.The identity removes the unknown true score from the quadratic fitting objective through integration by parts.Source: Appendix B · pp. 19–21

Appendix C. Proof of Proposition 4.1.

The proof establishes the optimal rank-r projector for the trace objective and bounds the excess trace associated with an estimated projector.

Appendix C. Proof of Proposition 4.1.

  • Among rank-r orthogonal projectors, the trace Tr((I − P)CΛ) is minimized by the top-r eigenspace projector Pr,Λ, with minimum Er(CΛ).The excess at an estimated projector is Tr((Pr,Λ − bPr)CΛ).Source: Appendix C · p. 21
  • The proof obtains a lower bound on excess trace using a curvature lemma, with the bound holding trivially when λr = λr+1.It applies the lemma with A = CΛ, E = Pr,Λ, and F = bPr.Source: Appendix C · p. 21
  • The upper bound on excess eigenvalue sum is the sum of the d − r largest singular values of bC − CΛ, each bounded by its operator norm.The argument uses Lidskii's theorem to place the eigenvalue difference sum in the convex hull of permutations.Source: Appendix C · p. 21

Appendix D. Centering constraint: proof of Proposition 3.1.

The centering constraint yields a closed-form constrained estimator and a bias bound, while the Stage-2 solve enforces global boundedness through polynomial constraints. The appendices also describe an iterative solver and its numerical feasibility limits.

Appendix D. Centering constraint: proof of Proposition 3.1.

  • The centered estimator minimizes a strictly convex quadratic subject to Bθr = 0, and eliminating the KKT multiplier gives its closed form when B has full row rank.The constraint is implemented as a cM-orthogonal projection.Source: Appendix D · p. 21
  • The centering bias decomposition is exact, and its first term is bounded in M-norm by ∥M^-1Rθr,⋆∥M.The bound uses that the M-orthogonal projection does not increase the norm.Source: Appendix D · p. 21
  • The estimator is asymptotically normal, with ∥bθr − ¯θ∥ = Op(N^-1/2), under the stated invertibility and finite-second-moment conditions.The argument applies the multivariate central limit theorem and delta method.Source: Appendix D · p. 22

Appendix E. The constrained Stage-2 solve.

  • The Stage-2 constraints impose a negative leading-form margin and a level bound on the data-containing ball, which together bound the reduced log-density above globally.The constraints can conflict for small level bounds, so nonemptiness is assumed for the ideal problem.Source: Appendix E · p. 22
  • The solver iteratively adds violating points from sphere and ball maximizations; the exact algorithm converges under its stated assumptions, and strict convexity gives a unique minimizer.The paper avoids a sum-of-squares restriction because it can exclude admissible coefficient vectors.Source: Appendix E · pp. 22–23
  • Numerical feasibility is checked by search rather than certified: rank-4 fits met both tolerances, while some rank-6 and rank-8 fits reached the round cap with the level bound unmet.The composed estimate remains normalizable in those cases.Source: Appendix E · p. 23

Appendix F. Identification under exact rank-r dependence: proof of Lemma 4.2.

  • Under exact rank-r dependence, the Stage-2 score projection uses active-coordinate polynomials, and its covariance range lies in span(Vr), equaling it when the rank is r.The proof identifies the projected score space using the decomposition into active and complementary coordinates.Source: Appendix E · p. 23

Appendix G. Details of the numerical experiments.

The experiments use reproducible seeded sampling and select model complexity and regularization by validation. The appendix specifies numerical normalization and posterior-KL procedures, including a two-parameter grid evaluation.

Appendix G. Details of the numerical experiments.

  • Each realization draws estimation and evaluation samples in order from a generator seeded at 7000+s, and the observation from a separate generator seeded at 20000+s.The appendix also records seeds for dictionary selection, the sample split, and the posterior illustration.Source: Appendix G · p. 23
  • The posterior comparison keeps parameter dimension at dpar = 2 and evaluates candidates on a uniform grid, dropping true-posterior values below 10^-15 and flooring candidates at 10^-15.This setup is stated to isolate the noise-estimator effect from high-dimensional posterior sampling.Source: Appendix G · p. 24
  • Stage-1 and Stage-2 truncation candidates are selected by cross-validation, choosing the smallest set within one standard error of the minimum criterion; regularization constants are selected on a held-out half of the Stage-2 sample.The selected constants are refit on the full Stage-2 sample.Source: Appendix G · p. 24
  • The normalizer uses scrambled Sobol’ quasi-Monte Carlo with 2^17 points; across three scrambles, log Ẑn ranges by 0.0081 nats and its mean is within 0.002 nats of the 2^19-point estimate.The comparison is reported for one fit per example at N = 50,000.Source: Appendix G · p. 24

A SCORE-COVARIANCE METHOD FOR REDUCED-ORDER NON-GAUSSIAN DENSITY ESTIMATION

This section presents supplementary evaluations and analyses of CAS’s reduced-subspace density estimation, examining its performance against PCA and the robustness of its diagnostics and regularization, alongside centering, polynomial-space selection, and marginal-distribution effects. It also reports a marginal KL bound and limitations on search and exact marginal restoration.

SM1. Stage-1 subspace convergence and PCA subspace baseline

  • The estimated copula-score covariance spectra show a gap at r⋆ = 4 across the three examples, and the Stage-1 subspace angle is smaller where cross-validation chooses a richer multi-index set.The spectra are estimated at N = 50,000 with (K1, q1) = (4, 2), as medians across five realizations.Source: A SCORE-COVARIANCE METHOD FOR… · p. 27, Figure SM1 · p. 28
  • CAS beats PCA in 12 of 15 Example 1 pairs, with a ≈0.35-nat log-likelihood gap, and cuts inference KL by 50% at r = 4 versus at most 3% for PCA.The inference means are 0.23 for CAS and 0.46 for the Gaussian-copula baseline; CAS and PCA share the other estimator components in these comparisons.Source: PCA subspace baseline: per-(r, N)… · pp. 27–28, Table SM1 · p. 28, Table SM2 · p. 29

SM1. Subspace-only KL divergence

SM1. Accuracy of the tail-sum diagnostic

  • The tail-sum diagnostic overestimates the reference error in every realization, and its Ky Fan bound exceeds the realized error by mean factors of 1.9, 1.9, and 1.4 on Examples 1–3.At d − r = 16, the operator-norm bound exceeds the reference tail sum on all three examples and is uninformative, while the Ky Fan bound remains informative.Source: Accuracy of the tail-sum diagnostic · pp. 30–31, Table SM3 · p. 31

SM2.2. Stage-2 coefficient C: stability and margin over RH2

  • For C = 1, 3, and 10, mean paired log-likelihood gains over RH2 are +0.076, +0.101, and +0.095 nats, with C = 3 yielding the largest mean gain and no pair lost beyond 0.05 nats.The constants are selected by validation without hand-tuning or a reference density; the comparisons hold the CAS subspace, rank, and marginals fixed.Source: Accuracy of the tail-sum diagnostic · pp. 31–32, Table SM4 · p. 32

SM2.3. Full RH2 κ-grid and the effect of a fixed κ

SM2.4. Cross-example regularizer comparison

  • Across all three examples, the fixed pair (Csamp, Ccurv) = (3, 10^-5) stays within 0.18 nats of the best alternative, without per-problem hand-tuning.Its gap to the held-out choice closes to within ±0.01 nats for N ≥ 2,500; omitting regularization costs 2.8–3.6 nats at N = 100, while c/N scaling changes mean gaps by at most 0.10 nats.Source: Accuracy of the tail-sum diagnostic · pp. 33, 35

SM2.5. Centering constraint

  • The fixed regularizer pair (Csamp,Ccurv)=(3,10−5) stays within 0.18 nats of the best alternative at every tested N on all three examples.Its distance from the per-realization held-out choice is at most 0.18 nats at N=100 and 0.01 nats for N≥2500; omitting regularization costs 2.8–3.6 nats at N=100 and at most 0.01 nats for N≥2500.Source: Accuracy of the tail-sum diagnostic · p. 35
  • At (K2,q2)=(5,3), centering lowers mean held-out log-score at every N on all three examples; Δctr is −0.0046 nats at N=500 on Examples 1 and 3.At N=50,000, the gap lies between −0.0031 and −0.0001 nats.Source: Table SM8 · p. 36, Accuracy of the tail-sum diagnostic · p. 36
  • At K2=3, centering costs +0.049 and +0.051 nats on Examples 1 and 2; at K2=4, Example 3 instead has Δctr=+0.0250 ± 0.0171 nats.For K2≥4 on Examples 1 and 2, |Δctr|≤0.006 nats; the K2=4 Example 3 result is an exception to the usual small-gap pattern.Source: Accuracy of the tail-sum diagnostic · p. 36, Table SM9 · p. 37

SM3. Greedy choice of the Stage-1 polynomial space

  • A proposed greedy Stage-1 search adapts polynomial degree and interaction order using validation while reusing cached quadratic computations, but it was not used in reported experiments or assessed for performance or runtime.A stop after evaluating all neighboring expansions establishes only local adequacy within the declared search domain; validation reuse is tuning, so predictive performance needs unused data.Source: Accuracy of the tail-sum diagnostic · pp. 37–39

SM4. Marginals of the reduced model

  • A reduced model’s ith marginal is πi multiplied by a factor depending on Zi; it equals πi for every reduced factor if and only if row ai is zero.Selecting r observation coordinates as the subspace leaves the d−r unselected marginals unchanged for every member of the reduced family, but does not guarantee this for selected marginals.Source: Accuracy of the tail-sum diagnostic · pp. 39–40
  • At the KL projection, each marginal’s divergence is bounded by 1/2 Tr((I−VrV⊤)C), equal to 1/2 Σj>r λj(C) for the leading eigenspace and zero when that tail vanishes.The fitted estimator’s marginals can also carry errors from estimating the reduced factor, transform, and marginals; exact restoration conditions are nonlinear in Stage-2 log-density parameters and are left for future work.Source: Accuracy of the tail-sum diagnostic · pp. 40–41
Loading 2609.36142v2…