Sparse Polynomial Chaos Expansions: Literature Survey and Benchmark

Motivation and sparse PCE

• Expensive, high-dimensional computer models often permit only a few evaluations, making accurate uncertainty quantification difficult without a cheap surrogate.Surrogate models approximate the original input-output relation and enable low-cost uncertainty quantification after construction.Source: §1 · p. 1
• Sparse PCE represents computational models with mostly zero coefficients, leveraging sparsity-of-effects and compressibility to reduce the evaluations needed for surrogate construction.PCE use orthonormal polynomial bases and support analytical computation of moments and Sobol’ sensitivity indices; nonintrusive PCE treat the model as a black box.Source: §1 · p. 2

Paper contributions

• The paper surveys the sparse PCE literature, develops a general framework for classifying its approaches, and numerically benchmarks selected methods on realistic models.The review addresses many recent methods that often lack thorough comparisons with previously published approaches.Source: §1 · p. 2

2.1 Regression-based polynomial chaos expansions

• For independent inputs, polynomial orthonormal basis elements are products of univariate orthonormal polynomials indexed by multi-indices; outputs admit infinite expansions, but only finitely many coefficients are computed in practice.A heuristic number of model evaluations is N ≈2P, 3P.Source: §2.1 · p. 3

2.2 Sparse PCE

• Sparse PCE adds sparsity constraints to regression, enabling fewer design points than basis functions while targeting robust, accurate surrogates rather than the sparsest possible expansion.Sparse regression uses ℓ0 or ℓ1 regularization; sparsity supports recovery or approximation in underdetermined systems and improves robustness to noise, motivating sparse PCE when OLS sampling is infeasible.Source: §2.2 · pp. 3–4

2.3 Framework: Classifying the literature on sparse PCE

• The proposed framework organizes sparse-PCE computation into candidate-basis and experimental-design selection, sparse solving, model selection, and optional iterative adaptation or enhancement.The workflow can repeat by adapting the basis or enriching the design until the selection criterion is satisfactory or cannot be reduced; the framework also enables structured comparisons and new method combinations.Source: §2.2 · p. 4, §2.3 · p. 4

2.4 Choice of basis and basis adaptation

• Basis construction balances approximation capacity against regression conditioning and evaluation-to-unknown ratios, using total-degree, hyperbolic, interaction-order, or adaptive schemes.Hyperbolic truncation with q < 1 excludes high-interaction terms while retaining univariate degrees, interaction restrictions reduce basis size in high dimensions, and adaptivity iteratively adds promising polynomials and may remove others.Source: §2.4 · p. 5
• Basis adaptation expands an initially small basis through degree, q-norm, interaction-order, or combined schemes, using either model-selection criteria or active terms from prior sparse solutions.Solution-agnostic approaches do not reuse previous solutions, whereas forward-neighbor and anisotropic-degree methods control additions based on active terms, often starting from a constant surrogate.Source: §2.4 · p. 6

2.5 Experimental design

• Experimental designs target space exploration or regression-matrix properties and include input-distribution sampling, induced sampling, and candidate-set optimization, but the review is not exhaustive.The surveyed sampling methods generally ignore computational-model evaluations; model-aware active and supervised learning are treated separately.Source: §2.5 · pp. 6–7

2.5.1 Sampling based on the input distribution

• Latinized partially stratified sampling filters main effects and low-order interactions and consistently outperforms Latin hypercube sampling in high-dimensional cases.Latin hypercube sampling improves space-filling relative to Monte Carlo and can reduce estimator variance, especially for quantities dominated by first-order interactions.Source: §2.5.1 · p. 7

2.5.2 Sampling from a different distribution

• Coherence-optimal sampling outperforms both Monte Carlo and asymptotic sampling, while asymptotic sampling outperforms Monte Carlo for low-dimensional, high-degree problems; Monte Carlo can be better for high-dimensional, low-degree problems.Coherence-based designs construct a new probability distribution and orthonormal basis with minimal coherence, whereas asymptotic sampling uses an easier-to-sample classical distribution.Source: §2.5.2 · p. 8

2.5.3 Choosing points according to an optimality criterion from a candidate set

• Candidate-set methods optimize regression-matrix criteria such as D-optimality, which maximizes the information-matrix determinant and is connected to minimizing PCE coefficient-estimate variance.Rank-revealing QR or subset selection can also handle wide matrices Ψ ∈ R^N×P with N < P, while candidate-set composition can strongly influence the result.Source: §2.5.3 · pp. 8–9
• Optimality-based designs select candidate points by improving regression-matrix properties, including column orthogonality, information-matrix determinant, mutual coherence, or average cross-correlation.S-optimal designs require N ≥ P because they rely on the information-matrix determinant, whereas near-optimal sampling does not require this condition.Source: §2.5.3 · p. 9

2.5.4 Sequential enrichment of the experimental design

• Sequential enrichment generally produces solutions with smaller validation error than nonsequential strategies, although this promising approach is not explored further in the benchmark.Additional points are selected using the latest sparse PCE solution or an augmented basis, including strategies based on S-optimal, D-optimal, coherence-optimal, expected-loss, and posterior-entropy criteria.Source: §2.5.4 · pp. 9–10

2.6 Solution of the minimization problem

• Sparse PCE regression solvers comprise convex, greedy, Bayesian compressive-sensing, and iteratively reweighted methods, with hyperparameters typically calibrated by cross-validation.FastLaplace has been shown to attain even sparser solutions, while LARS, OMP, SP, Bayesian, and reweighted approaches use distinct optimization or selection mechanisms.Source: §2.6 · pp. 10–11

2.7 Model selection criterion

• Model selection uses generalization error, approximated by validation error, to assess sparse-surrogate mean-square accuracy and decide whether to continue or stop.A relative mean-squared error is introduced to make validation error independent of model scaling.Source: §2.7 · pp. 11–12
• 2.7 Model selection criterion: Sparse PCE model selection proxies unavailable generalization error with cross-validation, LOO, modified LOO, KIC, or sparsity criteria, while double cross-validation addresses selection bias.Cross-validation may also select sparse-solver hyperparameters, but reusing its error estimate can be overly optimistic because of model-selection bias.Source: §2.7 · pp. 12–13

2.8 Further enhancements of sparse PCE

• 2.8 Further enhancements of sparse PCE: Proposed methods include preconditioning, stop-sampling, weighted regression, resampled PCE, and dimension reduction through PCA, basis adaptation, or nonlinear PCE-driven PLS.These approaches target improved accuracy, fewer model evaluations, or sparser representations in fewer coordinates; resampled PCE retains terms selected most often across solver runs.Source: §2.8 · p. 13

3.1 Benchmark design

• The benchmark compares MC, LHS, coherence-optimal, and D-optimal sampling with LARS, OMP, SP, BCS, and SPGL1 solvers.It uses modified LOO cross-validation for LARS and OMP, k-fold cross-validation for SP, BCS, and SPGL1, and also tests an SP variant using LOO cross-validation.Source: §3.1 · p. 14
• D-opt(LHS) sampling performed significantly worse than most other sampling schemes and often worse than its LHS candidate set.The benchmark therefore excludes D-optimal designs constructed from LHS candidate sets from displayed results.Source: §3.1 · p. 14
• The study evaluates relative mean-squared error (RelMSE) across several models, sparse solvers, and experimental-design techniques, focusing on practically relevant small experimental designs spanning 5–7 sizes.Each experiment is repeated 30–50 times to account for statistical uncertainty.Source: §3.1 · p. 15

3.2 Software

• The benchmark is implemented primarily in UQLab, supplemented by DOPT_PCE, SPGL1, FastLaplace, and in-house implementations for specialized sampling methods.UQLab provides MC, LHS, LARS, and OMP, while DOPT_PCE provides D-optimal sampling and SP; coherence-optimal and near-optimal sampling use in-house implementations.Source: §3.2 · pp. 15–16

3.3 Benchmark: Considered models

• The benchmark covers 11 computational models of varying complexity and input dimensionality, with aggregated results over all models and detailed analysis of four spotlight models.The spotlight models represent low-dimensional highly compressible behavior, smooth nonlinear engineering behavior, high-dimensional slow coefficient decay, and high-dimensional compressible behavior.Source: §3.3 · p. 16
• The selected models are intended as a comparative testing ground rather than a representative sample of all engineering-model classes.They include benchmark functions and a stochastic heat-diffusion PDE, spanning analytical and nonanalytical, low- and high-dimensional, smooth and highly nonlinear cases.Source: §3.3 · p. 16
• The benchmark uses 11 analytical and finite-element models, each with a fixed static total-degree basis and hyperbolic truncation; q is heuristically set to 1 for d ≤10 and 0.5 for d ≥20.Source: Table 1 · p. 17, §3.3 · p. 18

3.4 Results: Comparison of solvers

• For larger experimental designs, solver generalization errors can differ by several orders of magnitude, whereas all solvers perform similarly poorly for the smallest designs.Figure 2 compares six solvers across spotlight models using relative MSE and LHS sampling.Source: §3.4 · p. 19, Figure 2 · p. 20
• BCS, SPLOO, and OMP are often among the strongest solvers; BCS favors smaller designs, SPLOO performs well for low-dimensional models, and OMP’s denser solutions generalize well.BCS can plateau early or omit important terms at large design sizes, while OMP tends to return dense solutions.Source: §3.4 · p. 19
• BCS is the most frequent winner for low-dimensional models with small experimental designs, ranking first in 31% of runs.Aggregated rankings separate low- and high-dimensional models and small- versus large-design regimes.Source: Figure 3 · p. 21, §3.4 · p. 21
• 42% of low-dimensional, large-design runs had SPLOO as the best solver, with 73% within two times and 99% within one order of the smallest relative MSE.SPLOO outperformed the other solvers in all reported respects for this regime.Source: §3.4 · p. 22
• Across solver-only comparisons, LARS and SPGL1 perform poorly; SP and BCS often approach the best solution but are less robust, whereas OMP is robust but less accurate.For high-dimensional models, all solvers were within ten times the minimum in more than 89% of runs, reducing solver-choice impact.Source: §3.4 · p. 22
• BCS was best in 45% and 53% of high-dimensional runs for small and large designs, respectively, and within two times the minimum in 96% and 97%.BCS and OMP were within ten times the smallest relative MSE in all high-dimensional cases.Source: §3.4 · p. 22

3.5 Results: Comparison of sampling schemes together with solvers

• MC and LHS perform comparably, with LHS usually matching or improving MC for low-dimensional models, while advanced schemes help mainly with large low-dimensional designs.Coherence-optimal and D-optimal coherence-optimal sampling show mixed performance for small low-dimensional designs and match or underperform LHS for high-dimensional models.Source: §3.5 · p. 23
• Figure 4 shows considerable solver–sampling differences, especially for low-dimensional models, whereas many high-dimensional combinations achieve errors close to the minimum.The comparison aggregates rankings and robustness across separate experimental designs for low- and high-dimensional models.Source: §3.5 · p. 23, Figure 4 · p. 24
• BCS is broadly strong across sampling schemes, except that SPLOO with coherence-optimal sampling wins for low-dimensional models with large experimental designs.The results suggest BCS is preferable when information content is low, including small designs or high-dimensional models.Source: §3.5 · p. 25
• Solver rankings and robustness are mostly independent of the sampling scheme, with similar behavior observed across designs.The results aggregated by sampling scheme follow the behavior observed on LHS.Source: §3.5 · p. 25
• Plateauing at larger sample sizes indicates that the maximum accuracy attainable with the chosen basis has been reached.A larger basis may add a missing important regressor, but at fixed design size it can reduce accuracy by lowering the design-point-to-basis-function ratio and degrading the regression matrix.Source: §3.5 · p. 25

3.6 Results: Comparison of sampling schemes together with solvers, using a smaller candidate basis

• With the smaller candidate basis, relative MSE reaches a plateau at smaller experimental-design sizes, while most solver and sampling observations remain unchanged.The experiments use the Ishigami and borehole models with LARS, OMP, SP, SPLOO, and BCS across MC, LHS, coh-opt, D-opt(coh-opt), and near-opt(coh-opt) designs.Source: §3.6 · pp. 25–26
• Near-optimal sampling often matches or slightly improves coh-opt sampling and frequently achieves the smallest median error.For Ishigami it also has low variability, whereas borehole results show a larger spread with several outliers; this is consistent with (Alemazkoor and Meidani, 2018a).Source: Figure 5 · p. 26, §3.6 · p. 26, Figure 6 · p. 27

4 Discussion and conclusions

• The survey organizes sparse PCE computation into a general framework that accommodates methods developed for different computational components.The benchmark was intended to guide practitioners and provide a basis for identifying methods suited to real-world problems.Source: §4 · pp. 27–28
• Solver and sampling choices can change relative MSE by several orders of magnitude, with their effects generally more pronounced for low-dimensional models.Solver and sampling rankings are mostly independent, so designs effective for one solver tend to work well with others, and vice versa.Source: §4 · p. 27
• For low-dimensional models, BCS is best with small experimental designs, SPLOO with coherence-optimal sampling is best with large designs, and near-optimal sampling dominates when feasible.D-opt(coh-opt) is slightly preferable for BCS at small design sizes, while SPLOO appears especially robust.Source: §4 · pp. 27–28
• For high-dimensional models, BCS is by far the best solver; all solvers perform better when paired with LHS, and no advanced sampling scheme appears competitive regardless of the solver used.The numerical benchmark used 11 example functions intended to resemble real-world engineering problems and tested sparse solvers and sampling schemes on a fixed set of basis functions.Source: §4 · pp. 27–28
• The conclusions are limited by representative but incomplete benchmark coverage and by using a fixed basis selected heuristically rather than investigating degree adaptivity.Adaptive degree selection depends on accurate error estimation; OMP and BCS can underestimate generalization error, potentially changing solver and sampling rankings.Source: §4 · p. 28

A Details on experimental design sampling techniques

• Sampling affects sparse-regression recovery through the regression matrix, whose entries are basis polynomials evaluated at design points and depend on input distributions and the index set.Some schemes are costly, nontrivial, or unavailable for certain input distributions, but model evaluations usually dominate practical cost, motivating complex designs when they reduce sample needs.Source: A Details on experimental design… · p. 39
• Sampling schemes are grouped into input-distribution sampling, induced sampling from modified distributions, and methods optimizing regression-matrix properties.Examples include MC and LHS; asymptotic, coherence-optimal, and Christoffel sparse approximation; and D-optimal, S-optimal, and near-optimal designs.Source: A Details on experimental design… · p. 39

A.1 Sampling according to the input distribution

• LHS divides each input component’s quantile space into N intervals, samples one point per interval, randomly combines dimensions, and transforms points into input space.This constructs a space-filling design through an isoprobabilistic transform.Source: §A.1 · p. 40
• LHS reduces linear-regression estimator variance when main effects dominate, whereas partially stratified sampling can reduce variance when interaction terms dominate (Shields and Zhang, 2016).Latinized partially stratified sampling combines LHS and stratification to minimize estimator variance.Source: §A.1 · p. 40
• Maximin selection improves LHS space filling by choosing, among random LHS designs, the design with the largest minimal pairwise distance.These methods aim to distribute design points evenly in quantile space, so no input region is neglected.Source: §A.1 · pp. 39–40
• Stratified sampling partitions input space into disjoint strata, samples within them, and weights points by stratum probability mass to reduce statistical-estimator variance.LPSS is especially beneficial when prior knowledge identifies interacting variable groups and has been used for problems with d = 100.Source: §A.1 · p. 40
• Monte Carlo (MC) sampling from the input distribution is a special case of coherence-based theory, which provides bounds on coherence and the associated number of points needed for recovery (Doostan and Owhadi, 2011; Rauhut and Ward, 2012; Yan et al., 2012; Hampton and Doostan, 2015b).Source: §A.1 · p. 40

A.2 Sampling from a different distribution

• Sampling from a modified distribution changes the weighted ℓ1 problem through a weight function w(x), potentially improving or deteriorating the recovered solution c.The weighting matrix W is diagonal in the sampled weights, and WΨ evaluates the modified basis w(x)ψα(x).Source: §A.2 · p. 40
• To make the columns of WΨ orthonormal, design points are drawn from a suitably modified input distribution f̃_X.The modified weighted system can therefore be interpreted as using the basis ψ̃α(x) = w(x)ψα(x).Source: §A.2 · pp. 40–41

Recovery guarantees

• Under isotropy, perfect recovery of an s-sparse solution requires a number of samples proportional to µ(A, {ψα})s log(P) with high probability, with analogous guarantees in noise.Here, s is the solution sparsity and P = |A| is the number of basis functions.Source: §A.2.1 · p. 41

Coherence-optimal sampling

• Lower coherence directly reduces the samples needed for perfect recovery, motivating coherence-optimal sampling with weighted functions that remain orthonormal under a modified distribution.The weighted system is constructed as ˜ψα(x) = w(x)ψα(x), with µ(A, {˜ψα}) < µ(A, {ψα}).Source: §A.2.1 · p. 41

Weighted orthonormal systems

• The weighted functions form an orthonormal system under ˜fX, and coherence is minimal when the upper bound G equals the tight bound B.A loose bound G ≥ B can instead yield a modified distribution that is easier to sample.Source: §A.2.1 · p. 41

Unbounded bases

• For polynomial bases without a finite upper bound, analogous results can be obtained on a restricted domain where the bound is finite and isotropy remains approximate.The modified distribution is ˜fX(x) = c2G(x)2fX(x)1S(x).Source: §A.2.1 · p. 42

A.2.2 Sampling using a loose upper bound ("asymptotic sampling")

• For Legendre and Hermite polynomials, a loose upper bound G(x) ≥ max_{α∈A} |ψ_α(x)| enables analytical expressions for distributions with improved coherence (Hampton and Doostan, 2015b).Source: §A.2.2 · p. 42
• Asymptotic distributions for Laguerre and Jacobi polynomials have been implemented in COH-OPT (Hampton and Doostan, 2017).These correspond to Gamma and Beta distributions, respectively.Source: §A.2.2 · p. 42
• For Legendre polynomials, asymptotic sampling has lower coherence than standard sampling when d < p, whereas standard sampling is lower-coherence when d > p.The latter regime is more common in applications.Source: §A.2.2 · p. 42
• The sampling scheme with smaller coherence is theoretically expected to yield better recovery rates, a prediction confirmed numerically for Legendre polynomials (Hampton and Doostan, 2015b, section 5.1).Source: §A.2.2 · p. 42

A.2.3 Coherence-optimal sampling

• Coherence-optimal sampling minimizes coherence and achieves better recovery and smaller errors than standard and asymptotic sampling, but MCMC can be slow in high dimensions.For Gaussian inputs, its broader support may also reduce accuracy when simulations are less reliable far from typical operating conditions.Source: §A.2.3 · pp. 42–43

A.2.4 Christoffel sparse approximation

• Christoffel sparse approximation supports weighted least-squares convergence, performs well for low-dimensional high-degree cases, but can underperform input sampling in high dimensions.Its modified basis is not orthonormal under the selected measure, yielding weaker theoretical recovery results available only in the univariate case.Source: §A.2.4 · p. 43

A.3 Choosing points according to an optimality criterion from a candidate set

• Candidate-set design methods select points using different optimality criteria, but greedy or heuristic procedures generally find only local optima whose quality depends on the candidate set.Candidate sets may come from MC, LHS, coherence-optimal, or Christoffel sampling, and optimized samples often preserve the candidate set’s spread.Source: §A.3 · p. 44

A.3.1 D-optimal sampling

• RRQR-based D-optimal sampling selects candidate points to improve the regression matrix and remains usable when N < P, although it is not guaranteed to reach the maximal D-value.The benchmark uses RRQR because conventional D-optimal methods fail when the information matrix is singular; RRQR instead produces a local optimum.Source: §A.3.1 · p. 44

A.3.2 Quasi-optimal sampling based on the S-value

• S-value sampling greedily adds candidate rows that maximize a heuristic combining column orthogonality and information-matrix determinant, but is unsuitable when basis size exceeds design size.Because S(Ψ) = 0 when N < P, the method was excluded from the benchmark for sparse PCE without experimental-design enrichment.Source: §A.3.2 · p. 45
• The S-value-based quasi-optimal sampling algorithm performs well for sequential enrichment and least-squares regression.Source: §A.3.2 · p. 45

A.3.3 Near-optimal sampling

• Near-optimal sampling greedily selects candidate points to reduce mutual coherence and average cross-correlation in the regression matrix.The method seeks columns pointing in different directions to facilitate sparse-solution recovery, but finds only a local optimum.Source: §A.3.3 · pp. 46–47
• O(NMP^2) complexity makes near-optimal sampling prohibitively expensive for large bases, and its optimized correlations do not guarantee accurate recovery (Alemazkoor and Meidani, 2018a).Source: §A.3.3 · p. 47

A.4 Illustration of sampling schemes

• Gaussian sampling designs can place points far into low-probability regions, where engineering models may be inaccurate or fail.For degree p = 12, some points lie seven standard deviations from the mean; the asymptotic, coherence-optimal, and matrix-optimal designs show large spreads that grow with basis degree.Source: §A.4 · p. 47, Figure 8 · p. 49

B Details on sparse regression solvers

• Sparse regression can minimize the ℓ2-norm of empirical error under a sparsity-enforcing constraint; exact ℓ0-minimization requires infeasible combinatorial search, while ℓ1-minimization provides a convex relaxation.Source: B Details on sparse regression solvers · p. 50

B.1 Orthogonal matching pursuit (OMP)

• OMP greedily adds the regressor most correlated with the residual, recomputes active coefficients by least squares, and uses sparsity, error, or model-selection stopping rules.It has complexity O(mNP), but may overfit and produce unreliable LOO error estimates in basis-adaptive settings.Source: §B.1 · p. 51

B.2 Least angle regression (LARS)

• LARS is a greedy method for approximately solving ℓ1-minimization, adding regressors according to residual correlations while differing from OMP in coefficient updates.It starts with an empty model and adds regressors one by one based on their correlation with the residual.Source: §B.2 · p. 52
• LARS-LASSO removes predictors when coefficient signs change and solves (27) exactly under certain conditions.Hybrid LARS instead recomputes least-squares coefficients for each selected basis, ensuring minimal empirical error and enabling LOO-based sparsity selection.Source: §B.2 · pp. 52–53
• LARS builds nested active-basis sets by moving coefficients in an equiangular direction until a nonactive predictor matches the active predictors’ residual correlation.It returns up to m = min{N, P} models and can stop at a target sparsity or residual-error threshold.Source: §B.2 · pp. 52–53
• For N < P, LARS has computational complexity O(mNP), with matrix inversions updateable in O(mN) using Schur’s complement.OLS-based LOO computation and early stopping can reduce practical model-selection costs.Source: §B.2 · p. 53

B.3 Subspace pursuit (SP)

• SP maintains K active basis functions, adds K residual-correlated regressors batchwise, fits them by OLS, and removes the K smallest-magnitude coefficients until convergence.The augmentation and OLS steps require 2K ≤ min{N, P}.Source: §B.3 · p. 54
• Under certain assumptions, SP has theoretical guarantees, with complexity O(N(P+K^2)K) generally and O(NPK) when K^2 ∈ O(P).The algorithm performs O(K) iterations in general and O(log K) in certain cases.Source: §B.3 · p. 54
• When K is unknown, the paper proposes selecting it by leave-one-out cross-validation instead of the 4-fold procedure suggested by Diaz et al. (2018).Diaz et al. (2018) suggest testing NK = 10 values of K and choosing the smallest 4-fold cross-validation error.Source: §B.3 · p. 55
• SP and CoSAMP are related greedy methods that differ mainly in the number of regressors added at each iteration.Source: §B.3 · p. 55

B.4 SPGL1

• SPGL1 solves BPDN by repeatedly solving LASSO with spectral projected-gradient descent until the Pareto relationship φ(τ) matches the prescribed σ.It uses Newton root finding on f(τ) = σ − φ(τ), where φ maps LASSO’s τ to the equivalent BPDN residual level σ.Source: §B.4 · p. 56
• The SPGL1 Pareto front is convex, nonincreasing, and differentiable, representing the trade-off between coefficient ℓ1-norm and residual ℓ2-norm.Source: §B.4 · p. 56
• SPGL1 was among the slowest solvers and often returned rather dense solutions in numerical benchmarks on compressible models.Source: §B.4 · p. 56

B.5 Sparse Bayesian learning

• Sparse Bayesian learning embeds regression in a probabilistic framework, estimating coefficients or predictions from posterior distributions under specified hyperparameter assumptions.BCS methods differ in their hyperparameter distributions and approximate posterior-computation techniques.Source: §B.5 · p. 56
• BCS typically assumes Gaussian white measurement noise, but omitted PCE terms can instead produce correlated, heteroscedastic, non-Gaussian, nonzero-mean discrepancies.Despite this mismatch, the framework can still help find sparse solutions.Source: §B.5 · p. 56
• The general BCS setup uses Gaussian likelihoods and coefficient priors, while noise variance and coefficient variances are governed by chosen hyperparameter distributions.Figure 9a summarizes the likelihood, priors, and distributional choices underlying sparse Bayesian learning.Source: Figure 9 · p. 57, §B.5 · p. 57
• Sparsity arises through assumptions on the hyperparameter distribution that induce an effective coefficient prior with a sharp peak at zero, encouraging zero values, and a heavy tail allowing large coefficients.Source: §B.5 · p. 57
• Babacan et al. (2010) use a hierarchical BCS model with exponential coefficient-variance priors, Gamma hyperpriors, and iterative optimization of λ, β, and γ_i.For their implementation, ν = 0 gives the best results, while β is set or selected because the algorithm does not estimate it reliably; the benchmark uses cross-validation.Source: §B.5 · p. 58

B.6 Greedy stepwise regression solvers

• Greedy stepwise solvers start from an empty model, add regressors according to a selection criterion, compute coefficients, and stop or select among models using a criterion.Some methods also perform backward elimination, and Table 3 summarizes selection, coefficient-computation, and stopping choices, including OMP and LARS.Source: §B.6 · p. 58
• Except for LARS and OMP, greedy methods are heuristic, lack theoretical convergence guarantees, and often depend on multiple tuning parameters.New variants can be formed by pairing alternative regressor-selection, coefficient-computation, and model-selection methods.Source: §B.6 · pp. 58–59

C Additional results

• The appendix presents additional results that complement those shown in Sections 3.4–3.6.Source: C Additional results · p. 61
• The detailed experimental setup is referenced in Section 3 rather than repeated here.Source: C Additional results · p. 61

C.1 Comparison of sparse solvers

• Figure 10 displays boxplots of relative MSE for seven additional models.Source: §C.1 · p. 61

Comparison of sampling schemes together with solvers

• Figures 11–14 display box-plots of relative MSE against ED size for Ishigami, borehole, two-dimensional diffusion, and 100D function, showing all combinations of solvers and sampling schemes.Source: §C.2 · p. 61
• Solvers are distinguished by color, while sampling schemes use varying shades and line styles.Source: §C.2 · p. 61
• The results are additionally sliced at small and large experimental-design sizes to compare solver performance across design-size regimes.Source: §C.2 · p. 61

Solver comparison

• The remaining solver comparisons include LARS, SP, and BCS, complementing earlier results shown for OMP and SPLOO.These results are presented as boxplots of relative MSE against experimental-design size.Source: §C.3 · p. 61

Sampling-scheme comparison

• Figures 11–14 benchmark relative MSE across four sampling schemes, using five solvers for Ishigami (d = 3, p = 14, q = 1) and borehole (d = 8, p = 4, q = 1), and four solvers for two-dimensional diffusion (d = 53, p = 4, q = 0.5) and the 100D function (d = 100, p = 4, q = 0.5).Source: Figure 11 · p. 63, Figure 12 · p. 64, Figure 13 · p. 65

Plot encoding

• For the benchmark plots, sampling schemes are distinguished by shades and line styles, while solvers are coded by color and combined results follow the order MC–LHS–coh-opt–D-opt(coh-opt).This encoding is used to compare relative MSE across experimental-design schemes and solvers.Source: §C.3 · pp. 63–66

Smaller candidate basis

• Figure 15 extends the smaller-basis Ishigami and borehole comparisons to three sparse solvers and five experimental-design schemes over fifty replications.The Ishigami basis uses d = 3, p = 12, q = 1; the borehole basis uses d = 8, p = 4, q = 1.Source: Figure 15 · p. 67, §C.3 · p. 67

• Table 4 overviews articles and benchmark studies comparing sparse PCE methods, including their main results.Source: D Benchmark studies · p. 68, Table 4 · p. 69
• The listed “best method” is the method reported to deliver the smallest target error in each publication.Source: D Benchmark studies · p. 69
• Target quantities include moments, Sobol’ indices, and the sparse PCE surrogate’s generalization error.Source: D Benchmark studies · p. 69
• Table notation defines ‘≻’ as ‘better than’ and explains solver and sampling acronyms through Sections 2.5–2.6 or the cited publications.Source: D Benchmark studies · p. 69
• Sampling labels specify how candidate sets are created, while a star (*) marks a method proposed by the cited paper.Source: D Benchmark studies · p. 69