PaperlayerView exact revision on arXiv

Source-linked AI summary

On the sample complexity of the active subspace method

Fabio Nobile, Matteo Raviola, Raúl Tempone

arXiv:2609.08940v1math.NAcs.NA

TL;DR

High-dimensional parameter-to-output maps make efficient approximation challenging, while covariance-based analyses can yield conservative sampling estimates. The paper derives empirical active-subspace guarantees directly in projection error and uses RKHS smoothness to sharpen sampling and tail bounds. Its theory supports rank and sample-size prescriptions, with numerical experiments supporting the predicted error decay and showing that smoothness-adapted sampling can be conservative in some cases.

  • Problem

    High-dimensional parameter-to-output maps complicate efficient approximation, and covariance-based analyses can give sample-size estimates more conservative than commonly used sampling rules.

    Source: §1 · p. 3

  • Method

    The paper analyzes empirical active-subspace recovery in population projection error and uses RKHS smoothness to derive coherence estimates, importance measures, and tail bounds.

    Source: §7 · p. 45

  • Results

    The theory gives high-probability quasi-optimality bounds governed by regularized inverse Christoffel coherence; experiments support predicted rank-error decay and show smoothness-adapted sampling can improve recovery but be conservative when base sampling already resolves the problem.

    Source: §7 · p. 45

  • Takeaways & Limitations

    Combining finite-sample recovery and smoothness-based tail estimates provides prescriptions for rank, regularization scale, and sample size.

    Source: §7 · p. 45

    For the lognormal model, the uniform-gradient-bound argument under base Gaussian sampling yields no finite coherence bound because parameter-uniform ellipticity constants are unavailable.

    Source: §5.1 · p. 30

Abstract

from arXiv · show

Active subspaces identify low-dimensional linear structure in high-dimensional parameter-to-output maps by estimating the dominant eigenspace of a gradient covariance operator. In practice this covariance is replaced by a Monte Carlo estimator built from a limited number of gradient evaluations. Classical analyses based on controlling the covariance error in operator norm lead to sample-complexity estimates that can be substantially more pessimistic than the sampling rules commonly used in computations. This paper studies the empirical active subspace method directly in the projection-error metric relevant for ridge approximation. We derive non-asymptotic quasi-optimality bounds governed by a regularized inverse Christoffel function associated with the gradient field. Under a bounded-gradient assumption, the resulting estimates already improve the sample-complexity estimates obtained from operator-norm covariance bounds. We then show that additional smoothness of the gradient map, expressed through membership in a reproducing kernel Hilbert space, yields sharper coherence estimates and motivates tractable importance sampling from kernel diagonal measures. Furthermore, the same smoothness assumption yields a priori decay bounds for the population active subspace tail energy, which can be combined with our finite-sample estimate to prescribe rank, regularization scale, and sample size, allowing to fully characterize the a priori sample complexity. The abstract assumptions are verified for lognormal Gaussian and affine uniform parametric elliptic PDEs using weighted summability of Hermite and Legendre series expansions.

1 Introduction

High-dimensional parameter-to-output maps motivate active subspaces, which seek low-dimensional projections capturing dominant variation. This paper analyzes empirical recovery directly in population projection error and combines finite-sample bounds with smoothness-based sampling and rank estimates.

1 Introduction

  • High-dimensional parameter-to-output maps can hinder efficient approximation, motivating active subspaces as a way to capture dominant variation with low-dimensional linear projections.The paper also relates population projection error to ideal ridge-approximation error under suitable conditional Poincaré inequalities.Source: §1 · p. 3

1.1 Active subspace recovery

  • Direct population-error analysis yields quasi-optimality bounds governed by regularized inverse Christoffel coherence, rather than requiring accurate operator-norm covariance estimation.Earlier covariance-based guarantees have sample-size dependence of order r^2 tol^-4 under bounded-gradient assumptions.Source: §1.1 · pp. 3–4
  • For fixed rank, smoothness improves tolerance dependence from tol^-2 to tol^-2θ, while oracle sampling requires order r log r but depends on the unknown gradient covariance.The smoothness result assumes RKHS eigenvalues are ℓθ-summable; the oracle measure is not directly available for sampling.Source: §1.1 · p. 4
  • Smoothness-based tail decay yields overall sample complexity M ≳ tol^-2θ(1+1/q), up to logarithmic factors, and lognormal Gaussian PDEs have twice the ridge-error decay exponent of coefficient-ranked coordinates.The exponent q depends on RKHS spectral summability and additional summability across parameter directions.Source: §1.1 · p. 4

1.2 Other related work

  • Related work covers infinite-dimensional active subspaces, Hilbert-space empirical PCA, ridge-leverage sampling, and weighted Hermite and Legendre estimates for elliptic PDEs.The paper distinguishes its importance-weighted Hilbert-valued-field guarantees from prior PCA results under fixed sampling distributions.Source: §1.2 · p. 4

1.3 Contributions and outline

  • The contributions combine finite-sample recovery, RKHS-based coherence and tail bounds, and PDE regularity verification for lognormal Gaussian and affine uniform models.The RKHS results provide kernel-diagonal importance measures and joint prescriptions for rank, regularization scale, and sample size.Source: §1.3 · p. 5

2 The active subspace method

The active subspace method selects input directions with high average squared gradient sensitivity, using a rank-r projection to reduce the parameter-to-output map. The section defines the population and empirical constructions and explains when projection error controls ideal ridge-approximation error.

The active subspace method

  • The framework allows finite- or countably infinite-dimensional parameter spaces and defines the gradient through a closed Sobolev structure.For standard Gaussian and centered uniform product measures, the paper gives coordinate characterizations of that structure.Source: §2 · pp. 5–6
  • The active subspace method selects directions of high mean-square sensitivity using the dominant eigenspace of the gradient covariance operator.It uses first-order information about f rather than solving the joint nonlinear ridge-approximation problem.Source: §2 · p. 7
  • The squared population projection error is the residual gradient energy outside the rank-r active subspace, also called the active subspace tail energy.A rank-r minimizer is given by the projector onto the eigenspaces associated with the r largest covariance eigenvalues.Source: §2 · pp. 7–8
  • Under a subspace conditional Poincaré inequality, squared gradient projection error bounds the error of the best ridge approximation.For a fixed subspace, that best approximation is the conditional expectation of f given the reduced variables.Source: §2 · p. 8
  • For Gaussian product measures, every finite-rank projector satisfies the subspace conditional Poincaré inequality with a dimension-independent constant.The stated constant is 1.Source: §2 · p. 9
  • For uniform product measures, a suitable uniform conditional Poincaré bound in infinite dimensions is left beyond the scope of this work.Conditioning on general subspaces couples coordinates, so the coordinatewise tensorization argument does not apply.Source: §2 · p. 9
  • The empirical method estimates the gradient covariance from importance-weighted samples and selects a leading rank-r spectral projector.The estimator is unbiased; sampling from the target measure gives the standard Monte Carlo covariance estimator.Source: §2 · p. 9

3 Quasi-optimality of the empirical active subspace

This section establishes high-probability quasi-optimality of the empirical active subspace for population projection error, deriving sample conditions through concentration of a regularized covariance and control of projection errors. With tolerance-dependent regularization, it obtains squared error of order tol² up to a fixed constant, with Monte Carlo-type sampling under bounded gradients and potentially sharper bounds from additional gradient structure.

3.1 The regularized inverse Christoffel function.

  • The regularized inverse Christoffel function acts as a continuous ridge-leverage coherence measure, and finite coherence yields a high-probability regularized covariance concentration event.The coherence and covariance effective dimension describe the sampling stability at regularization scale λ.Source: §3.1 · p. 11

3.2 From regularized concentration to projection-error control

  • A projector-independent tail-trace shift preserves the empirical minimizer, enabling the regularized concentration event to bound projection errors through head, tail, and head–tail terms.The resulting comparison includes residual terms tied to rank r and regularization scale λ.Source: §3.2 · pp. 13, 15
  • Under finite regularized coherence and the theorem’s sample condition, the empirical rank-r projector is quasi-optimal for population projection error with high probability, including positive- and zero-tail cases.For positive tail energy, the theorem also gives a condition at the canonical regularization scale; when the tail is zero and the covariance rank is at most r, it gives exact recovery.Source: §3.2 · p. 15
  • At the canonical regularization scale, the sample requirement is M = O(r log(r)) up to constants and probability factors, but the corresponding sampling measure depends on unknown covariance eigenfunctions.The regularized Christoffel measure minimizes coherence for fixed regularization, but is not directly usable for sampling when those eigenfunctions are unknown.Source: §3.2 · p. 16
  • Canonical regularization replaces sensitivity to the retained eigenvalue σ_r^2 with sensitivity to tail energy per target rank, stabilizing estimates when small covariance eigenvalues lie below the accuracy scale.Unregularized operator-norm covariance bounds do not distinguish energetic from negligible directions and may require additive error to be small relative to the target projection error.Source: §3.2 · p. 16
  • Unlike the covariance-error route, this analysis applies regularized concentration directly at the tolerance-dependent regularization scale.The covariance route first controls operator-norm error on the whole covariance operator before converting it into squared active-subspace error.Source: §3.2 · p. 17
  • A tolerance-dependent regularization scale yields squared population projection error e(Π̂_r)^2 ≲ tol^2, up to a fixed quasi-optimality constant.The additive regularization-scale term remains of order tol^2.Source: §3.2 · pp. 17–18
  • Under boundedness alone, sampling still has Monte Carlo-type tolerance dependence, but additional gradient structure can sharpen the regularized inverse-Christoffel bound.The crude bound is K_{ρ,λ}(∇f) ≤ B_ρ(∇f)^2/λ.Source: §3.2 · p. 18

4 Sample complexity from smoothness

The section uses smoothness of the gradient field to bound regularized coherence and active-subspace tail energy, then combines these bounds to prescribe rank, regularization, and sampling for a tolerance.

4 Sample complexity from smoothness

  • RKHS smoothness controls regularized coherence through bounded RKHS covariance and embedding spectral summability, and controls tail decay when combined with coordinate-wise Schatten summability.Together, these bounds give an a priori sample-complexity estimate as a function of tolerance.Source: §4 · p. 18

4.1 Regularized coherence from a bounded RKHS covariance

  • Under the bounded RKHS covariance assumption, regularized kernel diagonal measures yield coherence bounds; the θ-kernel measure avoids choosing the regularization scale in advance but can have a larger bound.The exact regularized kernel diagonal measure gives a sharper bound, but requires advance estimates of the target rank and RKHS covariance norm.Source: §4.1 · pp. 20–21

4.2 Active subspace tail energy decay from smoothness

  • When q_{θ,β}>0, smoothness assumptions yield an a priori active-subspace tail-energy decay bound, with both parametric regularity and coordinate summability entering the estimate.The exponent θ measures parametric regularity, while β captures additional Schatten summability of the RKHS covariance across coordinates.Source: §4.2 · p. 22

4.3 Tolerance-based sample complexity

  • For a prescribed tolerance, Theorem 4.1 selects rank and regularization and gives a sufficient sample condition for population-error control with probability at least 1−η.The prescription uses either the exact regularized kernel diagonal measure or the θ-kernel diagonal measure.Source: §4.3 · p. 23
  • The leading tolerance dependence combines tol^-2θ from regularized coherence with tol^-2θ/q_{θ,β} from sufficient-rank growth, plus a rank-dependent logarithmic factor.The rank growth is r_tol ≍ tol^-2/q_{θ,β}.Source: §4.3 · p. 24

5 Parametric elliptic PDEs

This section verifies RKHS gradient-regularity assumptions for parametric elliptic PDEs in lognormal and affine-uniform settings. For the affine-uniform model, weighted Legendre smoothness yields bounds on kernel coherence and active-subspace tail energy, along with certified sampling options; the tail bound alone does not imply a ridge-reconstruction error bound.

5 Parametric elliptic PDEs

  • Parametric PDEs provide high-dimensional parameter spaces, and suitable basis assumptions can make their solution maps infinitely differentiable, motivating RKHS verification for the active-subspace results.Source: §5 · p. 25

5.1 The lognormal model

  • Under weighted Hermite assumptions, the lognormal model’s target and coordinate derivatives belong to the RKHS, with jointly summable derivatives and a bounded gradient operator.The resulting smoothness covariance is controlled under the theorem’s assumptions.Source: §5.1 · p. 28
  • For the lognormal model, coherence grows as λ^-θ while active-subspace tail energy decays as r^-2/θ; the tolerance bound has leading tol^-1 power 2θ + θ^2.The explicit sufficient sample bound applies to both the regularized kernel diagonal measure and the θ-kernel diagonal measure.Source: §5.1 · p. 30
  • The lognormal active-subspace projection bound improves the coordinate-truncation decay from r^-1/(2θ) to r^-1/θ, with admissibility requiring θ > 1/r.For fixed r, the smallness condition can be enforced by rescaling the weights, at the cost of worse constants.Source: §5.1 · p. 30
  • Because lognormal coefficients lack parameter-uniform ellipticity, base Gaussian sampling gives no finite coherence bound, whereas kernel diagonal measures yield finite regularized bounds.The regularized measure gives the sharper effective-dimension bound but depends on λ and the smoothness scale; the θ-kernel measure is λ-independent and gives an explicit upper bound.Source: §5.1 · pp. 28, 30

5.2 The uniform model

  • For the affine-uniform model on [-1, 1]^d, weighted ellipticity ensures uniform coercivity, and Lemma 5.3 establishes RKHS admissibility and a bounded Legendre kernel diagonal under its stated conditions.Source: §5.2 · pp. 31–32
  • Under the theorem’s smoothness assumptions, f belongs to Aρ, its coordinate derivatives belong to Aζ, and the derivatives are jointly summable in H.For Legendre expansions, reducing weights to ζ = αρ with α < 1 absorbs differentiation growth and controls the weighted sum of coordinate derivatives.Source: §5.2 · pp. 32, 34
  • The passage states that the active-subspace tail energy satisfies an a priori bound for every r ≥ 1, without specifying the bound’s rate or value.Source: §5.2 · p. 35
  • The active-subspace tail-energy bound cannot be converted into a ridge-reconstruction error bound because the required subspace conditional Poincaré inequality for uniform product measures is not established.Establishing that inequality is described as delicate and beyond the scope of the work.Source: §5.2 · p. 36
  • The base uniform measure is quasi-optimal for the regularized kernel diagonal, while sampling from ρθ retains the sharper kernel effective-dimension bound.The uniform bound on the normalized Legendre basis supports the base-measure option; the kernel diagonal measure provides the sharper effective-dimension bound.Source: §5.2 · p. 36
  • Uniform ellipticity also yields a direct bounded-gradient estimate under the base measure.This estimate supplies the λ^-1 baseline coherence dependence, which the kernel-diagonal estimate improves when θ < 1.Source: §5.2 · p. 36
  • The bounded-gradient estimate gives coherence dependence λ^-1, while the kernel-diagonal estimate improves it to λ^-θ when θ < 1.The base uniform measure is quasi-optimal for the regularized kernel diagonal at every regularization scale, so this rate does not require a change of measure.Source: §5.2 · pp. 35–36

6 Numerical illustrations

The numerical illustrations study how sampling measures affect active-subspace recovery in a rare-activation Hermite model and a lognormal Haar-mode elliptic PDE. They also examine predicted tail decay and ridge-error behavior in the lognormal Haar model.

6.1 A rare-activation Hermite model

  • At odd target ranks, rare activation makes the retained coordinate’s H_2r component concentrated in Gaussian tails, so missing it can reverse its empirical spectral ordering with coordinate r + 1.The tail-concentrated component Qr has unit L2 norm.Source: §6.1 · p. 37
  • Sufficient sample sizes are of order r log(r) for regularized Christoffel sampling, r^(3+ε) for regularized kernel-diagonal sampling, and r^(6θ) for fixed-θ kernel-diagonal sampling.The regularized kernel-diagonal estimate pays for low-degree RKHS modes absent from the gradient.Source: §6.1 · p. 39
  • The regularized Christoffel measure achieved the best recovery, followed by regularized and θ-kernel diagonal measures; the base measure was least effective and failed for r ≥7, while coherence curves closely matched empirical sizes.Figure 1 reports empirical recovery sizes from 200 trials per tested sample size.Source: Figure 1 · p. 40, §6.1 · p. 40

6.2 A lognormal elliptic model with Haar spatial modes

  • For the lognormal Haar model, normalized reference active-subspace errors qualitatively followed predicted limiting slopes −1.8, −2.8, −3.8, and −4.8 across the four tested smoothness values.The curves showed plateaus and drops associated with the dyadic level structure.Source: §6.2 · p. 42
  • Across tested ranks 8, 9, 10, 15, and 20, active ridge errors decayed markedly faster than coordinate errors for s = 1, 1.5, 2, and 2.5.Fitted coordinate slopes were −1.07, −1.47, −1.90, and −2.35, and active slopes were −2.32, −3.15, −3.95, and −4.75, respectively. Agreement was close for s = 2 and 2.5, with visible discrepancies for s = 1 and 1.5.Source: §6.2 · p. 43
  • The lognormal Haar model’s empirical recovery experiment tested s = 2.5 using the base Gaussian and r = 2 Hermite θ-kernel diagonal measures.Figure 5 reports recovery sizes from 103 independent trials and includes a linear reference scaling M = 20r.Source: §6.2 · p. 43, Figure 5 · p. 44
  • The experiment tested θ ∈ {0.7, 0.8, 0.9, 1} using 103 trials and 78 sample sizes from 1 to 5000; recovery required success in at least 90% of trials.The reported sample threshold was the smallest grid value meeting that recovery criterion; each trial used its first M samples to construct the empirical projector.Source: §6.2 · p. 44
  • The base Gaussian measure required substantially fewer samples than θ-kernel diagonal measures; among the latter, recovery improved as θ increased.The passage describes this as instance-dependent: the regularized coherence estimate is a sufficient worst-case bound, not a guarantee that kernel diagonal measures outperform the base measure for every problem.Source: §6.2 · p. 44
  • Kernel diagonal sampling improved recovery in the rare-activation example but not this PDE, illustrating that its effect depends on the problem instance.For this PDE, the base measure evidently resolves the relevant gradient directions efficiently, whereas the modified measures also allocate samples to behavior allowed by the smoothness class but not prominent here.Source: §6.2 · p. 44

7 Conclusions

The paper develops finite-sample quasi-optimality bounds for empirical active subspaces in the projection-error metric, then combines them with gradient smoothness to prescribe rank, regularization, and sampling. Numerical experiments support the decay theory and show that smoothness-adapted sampling can help but may be conservative.

7 Conclusions

  • The projection-error analysis gives high-probability quasi-optimality bounds governed by regularized inverse Christoffel coherence; regularized Christoffel sampling reduces coherence to effective dimension and yields sample size of order r log r.In the positive-tail regime, the canonical regularization scale is tail energy divided by target rank; in the exact finite-rank regime, sufficiently small positive regularization gives exact recovery.Source: §7 · p. 45
  • RKHS regularity of gradient projections controls coherence and tail decay, enabling prescriptions for rank, regularization, and sample size from kernel-diagonal importance measures.The RKHS describes an a priori regularity class and may include modes absent from the particular gradient field.Source: §7 · p. 45
  • Weighted Hermite and Legendre summability verify the assumptions for lognormal Gaussian and affine uniform elliptic PDEs; the Gaussian ridge-error decay exponent doubles relative to best-ranked input coordinates.For the uniform case, the analysis explains why base-measure sampling can already be quasi-optimal.Source: §7 · p. 45
  • Experiments support the theoretical rank-r error decay rates and show that smoothness-adapted sampling can improve recovery but may be conservative when base sampling already resolves a problem.The sampling comparison is supported by both the reported experimental finding and its concise restatement.Source: §7 · p. 45
  • Further work includes adaptive refinement of kernel-diagonal sampling, sharper instance-dependent or probabilistic coherence, and extensions to broader parametric PDE classes.The proposed extensions aim to clarify how far the link between smoothness, sampling stability, and dimension reduction persists.Source: §7 · p. 45

A Auxiliary results for the active subspace setting

The appendix establishes analytic facts used in Section 2 and introduces notation for cylindrical functions and gradients.

A Auxiliary results for the active subspace setting

A.1 Product Sobolev calculus

The appendix supports the product Sobolev calculus with closability and coordinate-characterization results, using finite-dimensional smooth approximation and conditional expectations.

A.1 Product Sobolev calculus

  • Conditional expectations onto increasing finite-coordinate σ-algebras converge in L2, and the finite-dimensional approximants converge with their gradients.The convergence follows from Hilbert-valued martingale convergence and square summability of gradient coordinates.Source: §A.1 · pp. 48, 50
  • For Gaussian and uniform product measures, the appendix establishes closability of the cylindrical gradient and verifies that almost-everywhere equal cylindrical representatives have the same gradient.The proof uses coordinatewise integration by parts and density of test functions.Source: §A.1 · p. 48
  • The coordinate characterization is proved by reducing to finitely many coordinates and approximating each finite-dimensional function together with its weak gradient by smooth functions.Source: §A.1 · p. 49
  • Finite-dimensional graph density holds for Gaussian and uniform measures; the proofs use cutoffs and mollification for Gaussian inputs and extension and mollification for uniform inputs.Source: §A.1 · p. 49
  • The coordinate characterization identifies each gradient component with the corresponding weak derivative and requires square summability of those derivatives.Source: §A.1 · p. 50
  • The truncated conditional gradient has the form G(n) = Qn E[g | Fn], projecting the conditional expectation onto the first dn coordinates.Source: §A.1 · p. 50

A.2 Linear observations

Under the standing closability and observation-bound assumptions, the appendix proves that every linear observation is a Sobolev function.

A.2 Linear observations

  • Under the stated closability and observation-bound assumptions, every linear observation Xv belongs to H1.Source: §A.2 · p. 51
  • The proof approximates Xa by truncated compositions θR(Xa), whose functions and gradients converge pointwise to Xa and a.Source: §A.2 · p. 51
  • Dominated convergence gives convergence of the gradients in the required norm, and closedness of ∇ yields the claim.Source: §A.2 · p. 51

A.3 Gaussian subspace conditional Poincaré inequality

The proof establishes a conditional Poincaré bound for Gaussian subspaces by reducing finite-dimensional cases to Gaussian vectors and extending the argument to finite or infinite product measures.

A.3 Gaussian subspace conditional Poincaré inequality

  • The argument applies to both finite and infinite Gaussian product measures.It uses conditional expectation as an L2 contraction when extending the inequality.Source: §A.3 · pp. 51–52
  • The Gaussian process {Xa} is centered, with covariance given by the ℓ2(d) inner product.The proof obtains this from finite-coordinate approximations and convergence of characteristic functions.Source: §A.3 · p. 51
  • Conditional variance given the subspace is bounded by the expected gradient quadratic form with the conditional covariance Σ.The inequality holds pointwise under conditioning on XV = z.Source: §A.3 · p. 52

B.1 Joint marginals for the rare-activation Hermite model

At rank 7, the four sampling measures produce distinct joint marginals for coordinates y5 and y7, reflecting different ways of redistributing probability across coordinates.

B.1 Joint marginals for the rare-activation Hermite model

  • The joint (y5, y7) marginals show how the four sampling measures redistribute probability across coordinates.The measures are the base Gaussian, θ-kernel diagonal measure at θ = 0.8, regularized kernel diagonal measure, and regularized Christoffel measure.Source: §B.1 · p. 53, Figure 6 · p. 53
  • The base Gaussian and θ-kernel diagonal measures factorize across coordinates, while the regularized kernel diagonal measure generally does not.The latter dependence is induced by its nonfactorized Hermite coefficients.Source: §B.1 · p. 53
  • The regularized Christoffel measure mixes choices of one normalized gradient coordinate at a time, producing horizontal and vertical features in the joint marginals.The remaining coordinates retain their Gaussian law.Source: §B.1 · p. 53

B.2 Spatial modes for the lognormal elliptic model

The lognormal Haar benchmark compares reference active spatial modes with coordinate Haar functions and shows that the active directions are not simply reordered coordinate directions.

B.2 Spatial modes for the lognormal elliptic model

  • The leading reference active mode remains close to the coarsest Haar pattern for both s = 1 and s = 2.5.The figure compares the first four active modes with the first four coordinate Haar functions.Source: §B.2 · p. 53, Figure 7 · p. 54
  • The following reference active modes combine several dyadic levels rather than merely reordering the coordinate directions.Each curve is normalized by its spatial supremum norm; active-mode annotations give the norm before normalization.Source: §B.2 · p. 53, Figure 7 · p. 54
  • The figure presents the first four coordinate Haar functions in the top row and the first four reference active modes in lower rows for s = 1 and s = 2.5.Active-mode annotations report norms before the curves are normalized.Source: Figure 7 · p. 54
Loading 2609.08940v1…