Source-linked AI summary
The generalization of Latin hypercube sampling
Michael D. Shields, Jiaxin Zhang
TL;DR
The paper broadly generalizes stratified sampling and addresses challenges in choosing appropriate partially stratified sampling subspaces. It introduces Latinized stratified sampling, combining stratified and Latin hypercube designs, to reduce variance from both main effects and interactions.
Problem
Choosing appropriate subspaces for partially stratified sampling is difficult, particularly when sampling must address interactions and main effects.
Method
The paper generalizes stratified sampling and couples partially stratified sampling with Latinized stratified sampling to produce designs that are simultaneously stratified and Latin hypercube designs.
Results
Partially stratified sampling reduces variance associated with variable interactions, whereas Latin hypercube sampling reduces variance associated with main effects.
Takeaways & Limitations
Coupling Latinized stratified sampling with partially stratified sampling provides superior variance reduction when low-order interactions and main effects are both present.
Takeaways & Limitations
The paper discusses considerations, challenges, and shortcomings associated with partially stratified sampling designs.
Abstract
from arXiv · showhide
Latin hypercube sampling (LHS) is generalized in terms of a spectrum of stratified sampling (SS) designs referred to as partially stratified sample (PSS) designs. True SS and LHS are shown to represent the extremes of the PSS spectrum. The variance of PSS estimates is derived along with some asymptotic properties. PSS designs are shown to reduce variance associated with variable interactions, whereas LHS reduces variance associated with main effects. Challenges associated with the use of PSS designs and their limitations are discussed. To overcome these challenges, the PSS method is coupled with a new method called Latinized stratified sampling (LSS) that produces sample sets that are simultaneously SS and LHS. The LSS method is equivalent to an Orthogonal Array based LHS under certain conditions but is easier to obtain. Utilizing an LSS on the subspaces of a PSS provides a sampling strategy that reduces variance associated with both main effects and variable interactions and can be designed specially to minimize variance for a given problem. Several high-dimensional numerical examples highlight the strengths and limitations of the method. The Latinized partially stratified sampling method is then applied to identify the best sample strategy for uncertainty quantification on a plate buckling problem.
1. Introduction
The paper generalizes LHS within a spectrum of stratified sampling designs, introducing PSS to study intermediate stratification and LPSS to reduce variance from both main effects and interactions.
- LHS is widely used in uncertainty quantification and reduces variance associated with additive components, or main effects.
- True SS stratifies all dimensions simultaneously, whereas LHS stratifies each dimension separately and assembles samples through random pairings.
- PSS explores intermediate designs by stratifying selected orthogonal subspaces of an N-dimensional sample space.
- PSS reduces variance associated with low-dimensional interactions within high-dimensional transformations.
- LPSS combines PSS and LHS variance reductions to simultaneously address main effects and low-dimensional interactions, improving sample efficiency.
- The method is evaluated through high-dimensional demonstrations and applied to uncertainty quantification of plate buckling strength.
2. Review of Sampling Methods
The review contrasts simple random, stratified, and Latin hypercube sampling, emphasizing their different variance-reduction mechanisms and conditions.
- Simple random sampling generates independent and identically distributed realizations that are evaluated through a system transformation.
- Stratified sampling partitions the sample space into disjoint strata and samples realizations within them.
- LHS stratifies each vector-component subspace into equal-probability strata, then randomly pairs component samples without replacement.
- LHS is nonunique because random component pairings permit (M!)^(N−1) possible combinations, motivating algorithms that optimize pairings.
- Both SS and LHS reduce estimator variance relative to classical Monte Carlo, but through different statistical mechanisms.
- LHS filters main-effect variance, with stronger main effects producing greater variance reduction; monotonic functions also receive guaranteed reduction.
3. Partially Stratified Sampling
Partially Stratified Sampling (PSS) generalizes stratification by partitioning variables into orthogonal subspaces, spanning the extremes of true stratified sampling and Latin hypercube sampling. It supports variance analysis, unbiased estimation, and variance reduction when subspace structure captures interactions.
- PSS definition: PSS spans stratification designs between true stratified sampling, which stratifies all dimensions jointly, and LHS, which stratifies dimensions individually.Both true SS and LHS are special cases of PSS, using N-dimensional and one-dimensional subspaces respectively.
- PSS construction: PSS partitions an N-dimensional sample space into disjoint orthogonal subspaces, stratifies each subspace, and randomly groups lower-dimensional samples into full samples.The subspaces may have different dimensions and numbers of strata; a complete design also specifies variable groupings, sample counts, and stratum counts.
- PSS construction: In a 4D PSS 2x2 design, stratified samples are drawn within two 2D subspaces and then randomly paired to assemble 4D samples.The construction can be generalized to unequal strata, but the paper excludes that case because computing sample weights is challenging.
- Bias and response variance: PSS estimators are unbiased, and their marginal variable distributions match those generated by simple random sampling.The response variance is derived using admissible cell-pair covariances and reduces to the LHS form when each subspace is one-dimensional.
- Asymptotic properties: PSS reduces variance when covariances between cells without common coordinates are negative, while filtering additive components in subspace variables.This targets non-additive interaction components, for which low-dimensional stratification can outperform LHS.
- Asymptotic properties: PSS preserves asymptotic normality and does not increase estimator variance over SRS for sufficiently large sample sizes.The paper presents these properties as generalizations of corresponding LHS results.
4. When to use partially stratified sampling?
True stratified sampling is preferable for low-dimensional problems dominated by interactions, while LHS is strongest for additive main effects. PSS selects sampling strategies across subspaces according to the relative contributions of main and interaction effects.
- Variance mechanisms: SS and LHS reduce variance through different mechanisms: SS conditions estimates on strata, whereas LHS induces negative covariance linked to main effects.SS therefore addresses main and interactive effects, while LHS primarily filters main effects and provides no reduction for interaction components.
- Additive and product functions: LHS nearly eliminates additive-function variance, reaching approximately 10^-10 and outperforming SS by more than four orders of magnitude.SS still substantially reduces variance relative to SRS.
- Additive and product functions: For product functions, SS reduces variance by up to three orders of magnitude relative to SRS and LHS in low dimensions.Its advantage persists through moderate dimensions, including N = 10, but diminishes as dimension grows.
- Choosing SS or LHS: True SS is appropriate when the SS main-effect variance is smaller than the LHS interaction-effect variance.This criterion connects sampling choice to the relative contributions of main and interaction effects.
- Choosing SS or LHS: SS becomes superior to LHS at S12 = 0.005 for uniform inputs and S12 = 0.1 for normal inputs.For SS, interaction-effect variance remains negligible unless S12 > 0.5; LHS variance increases dramatically even with very small interactions.
5. “Latinized” Stratified Sampling
Latinized stratified sampling constructs designs that are simultaneously stratified and Latin hypercube samples. Applying LSS within PSS subspaces targets interactions while retaining main-effect variance reduction and avoids some orthogonal-array construction challenges.
- LSS construction: LSS constructs a true stratified sample that is simultaneously an LHS on a chosen subspace.The procedure draws an LHS, imposes compatible strata, and selects points without replacement within each stratum.
- LSS construction: The LSS procedure repeats stratum-wise sampling while marking used LHS cells off-limits for later strata.This preserves the Latin constraint across the resulting sample set.
- Relation to OA-LHS: Under equal-probability strata, LSS is equivalent to an OA-LHS, but it is easier to construct because it does not require large orthogonal arrays.Under equivalence conditions, it inherits OA-LHS variance-reduction properties.
- LSS with PSS: Applying LSS to PSS subspaces targets different interaction orders and variable combinations without generating large orthogonal arrays.Disjoint subspaces can simultaneously address, for example, three-variable and two-variable effects.
- LSS with PSS: LPSS reduces variance for both main effects and interactions, making subspace selection less costly when suspected interactions are uncertain.If variables interact, both effects are reduced; if they do not, the text reports no variance increase.
6. Demonstration Problems
High-dimensional examples compare PSS and LPSS designs on polynomial, Rosenbrock, and Schwefel functions. LPSS generally performs strongly when both main effects and interactions matter, but very high-dimensional interactions remain difficult to reduce.
- High-dimensional polynomial functions: LPSS-2KI1K−2KI and LPSS-250 consistently provide the largest variance reductions in the polynomial tests, especially with strong main effects and interactions.The two LPSS designs perform nearly identically.
- High-dimensional polynomial functions: PSS-250 is less effective than LHS when main effects are strong, but becomes increasingly competitive as interaction strength grows.With weak main effects, it is nearly always superior to LHS.
- High-dimensional polynomial functions: When no main effects are present, LHS provides no variance reduction, whereas all PSS and LPSS methods provide considerable reduction.This pattern is reported for the N(0,1) Case 4 polynomial test.
- Rosenbrock function: For the Rosenbrock function, PSS and LPSS are considerably more effective than LHS, and 4D partial stratification reduces variance more than 2D stratification.The authors attribute the latter to reducing variance from more interactions.
- Schwefel’s Problem 1.2: For shifted-mean Schwefel inputs, LHS reduces variance by more than an order of magnitude, while LPSS remains effective by reducing both main-effect and interaction variance.PSS is slightly less effective because it cannot reduce the significant main effects.
- Limitations: Very high-dimensional interactions currently lack an effective variance-reduction method, although they typically occur only in select circumstances.This is identified as a limitation of both the proposed methods and existing techniques.
7. Application to plate buckling
The plate-buckling application seeks a stratification that minimizes Monte Carlo uncertainty in a six-dimensional problem with strongly interacting material, geometric, and imperfection variables. LPSS-4112 provides the greatest variance reduction among the tested designs.
- Problem and sampling designs: The plate-buckling uncertainty-quantification problem is six-dimensional and contains strongly interacting parameters controlling buckling strength and variability.The study evaluates PSS and LPSS designs that group variables according to structurally relevant interactions.
- Results: LHS outperforms all three PSS designs because filtering main effects is more important than filtering interactions for this problem.All methods use 625 samples.
- Results: All three LPSS designs outperform LHS by reducing variance associated with both main effects and interactions.LPSS therefore improves on the separate main-effect advantage of LHS and interaction-focused advantage of PSS.
- Results: LPSS-4112 is the most effective sample design for the plate-buckling problem.Its effect is described as similar, though not identical, to constructing an LHS on λ, δ0, and η.
8. Conclusions
The paper generalizes LHS into a spectrum of PSS designs and combines PSS with LSS to reduce variance from both variable interactions and main effects. The resulting LPSS method is applied to high-dimensional examples and a plate buckling uncertainty-quantification problem.
- LHS and true stratified sampling form the extremes of a spectrum whose intermediate designs are Partially Stratified Samples.
- PSS designs reduce variance from variable interactions, whereas LHS reduces variance from main effects.
- PSS designs have use-related challenges and shortcomings that the paper addresses by coupling them with Latinized stratified sampling.
- LSS produces designs that are simultaneously LHS and fully stratified, and under certain conditions is equivalent to an Orthogonal Array-based LHS but simpler to obtain.
- LPSS combines LSS with PSS and provides superior variance reduction in many high-dimensional applications when low-order interactions and main effects are present.
- The methods are evaluated through numerical examples and applied to a plate buckling problem in structural mechanics.