Polynomial meta-models with canonical low-rank approximations: numerical insights and comparison to sparse polynomial chaos expansions

1 Introduction

• PCE expands responses on tensor-product orthonormal polynomial bases, but the number of basis terms grows exponentially with input dimension.Sparse PCE selects a smaller set of relevant terms to improve efficiency.Source: §1 · p. 2
• Canonical decompositions express tensors as sums of rank-one components, offering a compressed polynomial representation whose parameter count grows linearly with input dimension.The paper investigates their use as low-rank approximations (LRA).Source: §1 · pp. 2–3
• The study investigates open questions in greedy canonical LRA construction, including stopping criteria and selection of optimal rank and polynomial degree.The algorithm sequentially updates polynomial coefficients along separate dimensions.Source: §1 · p. 3
• The comparison assesses LRA and sparse PCE errors on finite-element models using varying experimental-design sizes sampled with Sobol sequences and Latin hypercube sampling.The paper also briefly examines beam deflection to check LRA performance when the original model has low-rank structure.Source: §1 · p. 3

2.1 Mathematical setup

• A non-intrusive meta-model is an analytical function built from original-model evaluations at an experimental design, intended to reproduce similar statistical properties.The input and response are treated as random variables; the setup restricts attention to scalar responses, which can be handled elementwise for vector responses.Source: §2.1 · pp. 3–4

2.2 Error measures

• Generalization error measures meta-model accuracy, and when direct evaluation is impractical it can be estimated using exact-model and meta-model responses on a validation set.The validation set requires additional evaluations of the original model.Source: §2.2 · p. 4
• Empirical error uses only experimental-design evaluations, but can underestimate actual generalization error, particularly under overfitting.Cross-validation instead partitions the experimental design into training and testing sets and averages the resulting errors.Source: §2.2 · pp. 4–5

3.1 Spectral representation

• Under finite-variance assumptions, spectral representation expresses the response in a Hilbertian basis, while independence allows multivariate bases to be formed from univariate bases.The construction can also treat dependent inputs after an appropriate isoprobabilistic transformation.Source: §3.1 · p. 5

3.2 Construction of polynomial basis

• PCE uses orthonormal multivariate polynomials formed as tensor products of univariate families; isoprobabilistic transformations accommodate nonstandard or dependent inputs.Hyperbolic truncation limits interaction terms, but the number of retained terms still grows exponentially with input dimension.Source: §3.2 · pp. 6–7

3.3 Computation of polynomial coefficients

• Regression estimates PCE coefficients by minimizing residual mean-square error over the experimental design, while sparse PCE uses LAR to select predictors before OLS fitting.For high-dimensional problems, the classical coefficient count can make the required model evaluations prohibitively large.Source: §3.3 · pp. 7–8

3.4 Accuracy estimation

• PCE accuracy can be estimated with leave-one-out error computed from a single full-design expansion, with a corrected estimate used because the uncorrected error tends to be optimistic.The correction is described for PCE, unlike the LRA correction whose orthogonality assumption does not hold.Source: §3.4 · p. 8

4.1 Formulation with polynomial bases

• A canonical decomposition expresses the response as a finite sum of rank-one functions; LRA uses such decompositions to approximate the response with relatively few terms.The rank-one functions are expanded using polynomial bases orthonormal with respect to each input’s marginal distribution.Source: §4.1 · pp. 8–9
• For M=10 and p=3, PCE has 1,048,576 unknowns versus 40R for LRA, at most 400 when R≤10.The contrast becomes more pronounced for higher-degree polynomials.Source: §4.1 · p. 9

4.2 Greedy construction

• The adopted greedy algorithm increases rank by successively adding rank-one components, with alternating least squares updating coefficients along one dimension at a time.The coefficients in the other dimensions are held fixed during each dimension’s least-squares minimization.Source: §4.2 · p. 9

4.2.1 Approximation for a prescribed rank

• Each prescribed-rank correction step stops when its iteration limit is reached or successive relative empirical-error decreases fall below a threshold.The correction step may require several passes over the dimensions, which the authors identify as potentially critical to accuracy.Source: §4.2.1 · pp. 10–11
• The construction replaces PCE’s single large minimization with a sequence of small problems, including M problems of size p_i+1 per correction iteration and an updating problem of size r.The paper notes that this can reduce memory requirements in high-dimensional applications.Source: §4.2.1 · p. 11

4.2.2 Rank selection

• Rank can be selected by k-fold cross-validation or by an LOO error computed from one full-design model, avoiding the repeated construction required by cross-validation.A correction to the LOO estimate is examined, but its formal derivation assumes orthogonal regressors, which do not hold for the updating problem.Source: §4.2.2 · pp. 12–13

5 Example applications of canonical low-rank approximations

• Four applications span a five-dimensional analytical rank-one beam model and finite-element models with 10, 21, and 53 inputs.The finite-element responses are truss deflection, frame displacement, and temperature in stationary heat conduction.Source: §5 · p. 13
• The experiments assess two rank-selection criteria and correction-step stopping criteria, while varying experimental-design size.The designs use Sobol quasi-random sequences, and large Monte Carlo validation sets are used to assess accuracy.Source: §5 · p. 13
• Large validation sets are unavailable in real-life problems, where analysts typically must rely on the experimental design alone.Source: §5 · p. 13

5.1 Beam deflection

• The response is simply-supported beam mid-span deflection, modeled from five independent uncertain inputs transformed to standard normal variables and expanded with Hermite polynomials.The inputs are beam width, height, length, Young’s modulus, and load.Source: §5.1 · pp. 13–14, Table 1 · p. 14

5.1.1 Rank selection and error measures

• For most design sizes, the median 3-fold-CV rank matches the optimal rank; generalization error stays below 10^-4 and is 10^-8 or smaller for N ≥ 200, even with non-optimal ranks.The optimal rank is 1 except at N = 100, where it is 2; LOO selects rank 1 throughout.Source: §5.1.1 · p. 14, Figure 1 · p. 15
• 3-fold CV estimates generalization error better than LOO, whose estimate deteriorates at higher ranks and is deemed inappropriate for this example.The passage attributes LOO’s overly high correction factors to non-orthogonal regressors, while uncorrected errors are too optimistic.Source: §5.1.1 · p. 15, Figure 2 · p. 15

5.1.2 Stopping criterion in the correction step

• Decreasing the correction-step threshold reduces generalization error by orders of magnitude for all but the largest design, while the smallest design requires more iterations.The threshold ∆c errmin is varied from 10^-9 to 10^-4.Source: §5.1.2 · p. 16, Figure 3 · p. 16

5.2 Truss deflection

• The truss example predicts mid-span deflection from ten independent uncertain inputs, using Hermite-polynomial LRA after transforming inputs to standard normal variables.The inputs comprise six vertical loads and cross-sectional areas and Young’s moduli for horizontal and diagonal bars.Source: §5.2 · p. 16, Figure 4 · p. 17, Table 2 · p. 17

5.2.1 Rank selection and error measures

• 3-fold CV captures the increase in optimal rank as design size grows, while rank-selection effects on accuracy diminish for larger designs; LOO selects rank 1 and is optimal only for N ≤ 500.The optimal rank is 1 for N ≤ 500 and higher for larger designs; at N = 5,000, the median CV rank matches it.Source: §5.2.1 · p. 17, Figure 5 · p. 18
• For sufficiently large designs, 3-fold-CV error estimates generalization error fairly across ranks, whereas LOO is inappropriate for higher-rank models.Both error measures approximate generalization error particularly well for larger designs.Source: §5.2.1 · p. 17, Figure 6 · p. 18

5.2.2 Stopping criterion in the correction step

• A sufficiently small stopping threshold is critical for smaller designs, but small thresholds may cause unnecessary correction-step iterations for larger designs.At N = 5,000, generalization error changes within the same order of magnitude, with a decreasing but non-monotonic trend as the threshold decreases.Source: §5.2.2 · p. 18, Figure 7 · p. 19

5.3 Frame displacement

• The frame example predicts top-right, top-floor horizontal displacement from 21 correlated inputs, transformed to independent standard normal variables for Hermite-polynomial LRA.Input dependence is defined using a Gaussian copula.Source: §5.3 · pp. 19–20, Figure 8 · p. 20, Table 3 · p. 20

5.3.1 Rank selection and error measures

• Except at N = 50, 3-fold CV has little rank-selection dispersion and usually matches the optimal rank; selecting a rank above 1 can cause gross errors at N = 50.The optimal rank is 1 for smaller designs and rises to at most 2; LOO selects rank 1 throughout.Source: §5.3.1 · p. 21, Figure 9 · p. 22
• 3-fold-CV error estimates become accurate by N = 500, while LOO may substantially overestimate generalization error for higher-rank models.The error estimates approach generalization error as design size increases.Source: §5.3.1 · pp. 21–22, Figure 10 · p. 22

5.3.2 Stopping criterion in the correction step

• Changing the stopping threshold affects generalization error by about one order of magnitude at N = 200, but has negligible accuracy effects at the other tested sizes, where repeated iterations add effort.The threshold is varied from 10^-6 to 10^-1.Source: §5.3.2 · pp. 22–23, Figure 11 · p. 23

5.4 Heat conduction

• The example predicts average temperature in a square subdomain using a random conductivity field discretized by EOLE, with 53 independent standard-normal inputs and a 16,000-element finite-element model.The temperature field is solved on a two-dimensional square domain with specified boundary conditions.Source: §5.4 · pp. 23–24

5.4.1 Rank selection and error measures

• Rank 1 is optimal for every design; 3-fold CV selects it consistently for N ≥ 200, while LOO selects it throughout.For the two smaller designs, CV can select higher ranks, affecting accuracy.Source: §5.4.1 · p. 24, Figure 15 · p. 26
• At the optimal rank, 3-fold CV overall estimates generalization error better than LOO in the illustrated cases, although LOO can estimate it reasonably even at higher ranks.Both criteria identify rank 1 as optimal in the examples with N = 100 and N = 5,000.Source: §5.4.1 · p. 25

5.4.2 Stopping criterion in the correction step

• Changing the iteration cap or differential-error threshold has relatively little effect on accuracy, becoming negligible for N > 500.The comparison varies Imax from 1 to 20 and ∆c errmin from 10^-1 to 10^-7.Source: §5.4.2 · pp. 25–26

• Three-fold CV estimates generalization error more reliably than LOO, whose lack of regressor orthogonality can produce excessive correction factors, especially at higher ranks.The uncorrected LOO counterpart tends to be overly optimistic.Source: §5.5 · pp. 26–27
• For finite-element models, EDs with N ≥ 10·M yield generalization errors of order 10^-3 or smaller, even with non-optimal rank selection.For the rank-1 analytical beam, the same ED-size condition yields even higher accuracy.Source: §5.5 · p. 27
• The differential empirical-error threshold is more critical for small EDs; the authors recommend 10^-5–10^-6 and caution that lower values can cause numerical instability.Setting Imax = 50 avoids excessive iterations while achieving nearly maximum or sufficient accuracy in the examined cases.Source: §5.5 · p. 28

6.1 Beam deflection

• For the rank-1 beam, LRA is 2–3 orders of magnitude more accurate than PCE; EDs of N = 30–50 yield LRA errors of order 10^-4–10^-6.The selected LRA degree agrees with the validation-set optimum or differs by one, and ED-based errors approximate generalization errors fairly well.Source: §6.1 · p. 29, Figure 18 · p. 29

6.2 Truss deflection

• For truss deflection, LRA is more accurate with smaller EDs, but PCE errors decrease faster and PCE becomes superior for larger EDs.For LHS designs, PCE errors show greater dispersion; LRA medians remain close to those from Sobol sequences.Source: §6.2 · p. 29, Figure 19 · p. 30, Figure 20 · p. 30
• At the highest response threshold in truss deflection, median PCE conditional error is about an order of magnitude larger than LRA's.The passages also report that PCE systematically underestimates extreme responses, while the difference has only minor influence on overall generalization error.Source: §6.2 · p. 31, Figure 21 · p. 31, Figure 22 · p. 32

6.3 Frame displacement

• For frame displacement, LRA errors are smaller at smaller EDs, while PCE conditional errors exceed LRA's at higher response thresholds despite being similar at lower thresholds.The LRA generalization error is estimated fairly accurately by 3-fold CV; PCE's is slightly underestimated by LOO.Source: §6.3 · pp. 32–33, Figure 25 · p. 34, Figure 26 · p. 34

6.4 Heat conduction

• For heat conduction, LRA errors are smaller at smaller EDs, but PCE errors decrease faster; both ED-based measures estimate generalization errors fairly well.The optimal LRA degree is accurately identified by 3-fold CV for all considered EDs.Source: §6.4 · p. 33, Figure 27 · p. 35
• For heat conduction, PCE predictions of extreme responses show a negative bias absent from LRA predictions, and conditional generalization error captures the lower PCE tail accuracy.Source: §6.4 · p. 34, Figure 28 · p. 35

• Across three finite-element problems, LRA errors are lower for small EDs (approximately up to 10M–20M), while PCE errors decrease faster as ED size increases.Source: §7 · p. 36
• When generalization errors are similar, LRA better estimates extreme responses; its advantage is quantified by conditional generalization error and is promising for reliability applications (Konakli and Sudret, 2015b,a).Source: §7 · p. 36
• LRA uses a sequence of small OLS minimizations, making its construction simpler than PCE and requiring far less computer memory.The comparison uses PCE with least angle regression to retain significant basis terms and LRA with tensor-product bases and small minimization problems.Source: §7 · pp. 35, 37