PaperlayerView exact revision on arXiv

Source-linked AI summary

Sparsity by Default: The Theory and Practice of ARD in Gaussian Process Regression for Variable Selection

Jia Cai

arXiv:2609.33550v1stat.MLcs.LG

TL;DR

In GP regression, variable selection is difficult because irrelevance concerns whether an entire nonlinear function is constant along an input, not whether a single coefficient vanishes. This paper explains ARD’s marginal-likelihood lengthscale mechanism, surveys its theory and alternatives, and concludes that ARD remains useful while providing only effective sparsity.

  • Problem

    For nonparametric functions, input irrelevance is a whole-function property—constancy along a coordinate—rather than a zero coefficient, so linear-model selection machinery does not directly apply.

    Source: §1 · p. 1

  • Method

    The paper explains ARD’s marginal-likelihood mechanism and surveys its theory and alternatives, including Bayesian shrinkage, projection, and scalable penalized-likelihood approaches.

    Source: §5.2 · p. 17, §6.1.2 · p. 20, §6.2.2 · p. 23

  • Results

    ARD remains a low-cost default in low-to-moderate dimensions, while high-dimensional selection is being addressed with strong sparsity priors and integrated inference.

    Source: §9 · pp. 30–31

  • Takeaways & Limitations

    The authors recommend ARD where it is trustworthy and alternatives when its specific limitations matter.

    Source: §3.6 · p. 13

    ARD yields effective sparsity: irrelevant inputs get small but positive relevances rather than exact zeroes.

    Source: §2.5 · p. 8

Abstract

from arXiv · show

Automatic relevance determination (ARD) is the standard device for input selection in Gaussian process (GP) regression. By giving the covariance kernel a separate lengthscale for every input and learning those lengthscales by maximizing the marginal likelihood, ARD lets the data decide which coordinates matter: irrelevant inputs receive very large lengthscales and are effectively switched off. We trace this mechanism to the Bayesian Occam's razor embodied in the marginal likelihood, derive the gradient through which it prunes inputs, and emphasize that ARD delivers effective rather than exact sparsity. We review the algorithms used in practice and the rules that turn lengthscales into selections, and we survey the asymptotic theory, distinguishing the fixed-domain identifiability obstruction on the lengthscales from the high-dimensional selection-consistency guarantees recently established for hierarchical GP priors, and noting what remains open for plain ARD. We compare ARD with spike-and-slab priors, sparse axis-aligned and global-local shrinkage priors including the Bayesian lasso and horseshoe, penalized likelihood kriging, sensitivity and projection criteria, and additive kernels. We argue that ARD endures because of its seamless integration with kernel learning, universal software support, and low cost, and we close with its limitations and remedies.

1 Introduction and Historical Context

In nonparametric regression, input irrelevance concerns whether an entire function is constant along a coordinate, making selection harder than checking whether a linear-model coefficient is zero. The section introduces ARD as a kernel-based response to this challenge, illustrates its effective pruning, and traces its development and related methods.

1 Introduction and Historical Context

  • Irrelevance in a general function means constancy in one input across all values of the others, a whole-function property rather than a zero coefficient.That makes nonparametric selection require machinery beyond the standard linear-model toolbox.Source: §1 · p. 1
  • ARD gives each input its own kernel lengthscale and learns these by marginal-likelihood optimization; very large lengthscales effectively switch inputs off.The reciprocal lengthscale acts as a relevance score, so selection needs no separate algorithm or cross-validation tuning parameter.Source: §1 · p. 2
  • The survey focuses on ARD and neighboring methods, not general GP instruction or comprehensive high-dimensional nonparametric regression.It aims to explain intuition for statisticians familiar with regression and some kernel concepts.Source: §1 · p. 2

1.1 A motivating illustration

  • With N = 120 observations, ARD assigned the relevant input a lengthscale near 1.7 and the inert input one near 1.4×10^2.The posterior mean varies along x1 but is nearly flat along x2, without a separate selection step.Source: §1.1 · p. 3, Figure 1 · p. 3

1.2 Historical development

  • MacKay introduced evidence-based, per-input regularization in Bayesian neural networks, where large regularization constants shrink an input’s outgoing weights toward zero.The survey reports that his 1993 ASHRAE energy-prediction entry won by a substantial margin.Source: §1.2 · p. 3
  • Neal’s Bayesian sampling of relevance parameters and neural-network-to-GP limit connected neural-network ARD to GP lengthscales.Williams and Rasmussen (1996) brought per-dimension lengthscales into mainstream GP machine learning; Rasmussen and Williams (2006) gave the canonical exposition.Source: §1.2 · p. 4
  • The relevance vector machine prunes basis functions or training points, unlike input ARD, which selects variables.Wipf and Nagarajan (2008) related ARD to reweighted ℓ1 problems, while Qi et al. (2004) proposed selecting relevances by leave-one-out predictive performance after noting evidence-maximizing ARD can overfit.Source: §1.2 · p. 4

2 Theoretical Foundations of ARD

Gaussian process regression represents an unknown function with a covariance kernel, and ARD gives each input its own learned lengthscale. The marginal likelihood balances fit against complexity, pruning irrelevant inputs through its gradient, but yielding soft sparsity and an optimization landscape that can produce start-dependent selections.

2.1 Gaussian process regression in one page

  • A Gaussian process places a prior on the function through a covariance kernel and gives a test-point predictive mean and variance in closed form.The predictive variance provides uncertainty quantification.Source: §2.1 · pp. 4–5

2.2 The ARD kernel

  • ARD replaces the shared squared-exponential lengthscale with one per input, so a very large lengthscale makes the covariance—and fitted function—effectively independent of that coordinate.The reciprocal lengthscale 1/ℓ_j is interpreted as relevance; the same per-dimension treatment extends to Matérn kernels.Source: §2.2 · p. 5

2.3 The marginal likelihood and the Bayesian Occam’s razor

  • The marginal likelihood combines a data-fit term with a log-determinant complexity penalty, automatically balancing fit and flexibility when selecting lengthscales.Marginalizing the latent function supplies the penalty; without that integration, joint likelihood optimization would drive lengthscales toward zero to fit noise.Source: §2.3 · p. 6, Figure 2 · p. 7

2.4 The gradient of the marginal likelihood

  • The marginal-likelihood gradient pits a residual-fit trace against a complexity trace; for an irrelevant input, the optimizer increases its lengthscale until the pruning force weakens.The covariance-derivative entries decay like ℓ_d^-3, leading to a large but finite plateau.Source: §2.4 · pp. 7–8

2.5 Effective sparsity, not exact sparsity

  • ARD yields effective rather than exact sparsity: irrelevant inputs receive small but positive relevances, so a separate thresholding rule is needed to select a subset.An artificial inert reference input can calibrate a noise floor, while the selection boundary remains the analyst’s choice.Source: §2.5 · p. 8, Figure 3 · p. 9

2.6 A cautionary subtlety: the landscape is not convex

  • The marginal-likelihood objective is generally non-convex, so different initializations can converge to different lengthscales and relevance patterns.This motivates multi-start optimization and fully Bayesian alternatives.Source: §2.6 · p. 9

3 Algorithmic Implementations and Thresholding

The section reviews evidence maximization, Bayesian sampling, and variational approximations, then explains how relevance estimates become selections and when to use ARD or alternatives.

3.1 Type-II maximum likelihood by gradient ascent

  • Type-II fitting optimizes log marginal likelihood with log-scale positive hyperparameters and multiple restarts to address the non-convex objective.Standardization, coordinate-wise median-distance initialization, and checking restart spread help interpret and diagnose the fit.Source: §3.1 · p. 10

3.2 Fully Bayesian inference by sampling

  • Fully Bayesian ARD integrates over hyperparameters and summarizes posterior relevance, trading greater uncertainty and selection fidelity for the cost of hundreds or thousands of GP iterations.Hamiltonian Monte Carlo is one sampling option; slice sampling is a gradient-free alternative.Source: §3.2 · p. 11

3.3 Variational and approximate variants

  • Variational approaches range from scalable Type-II-like lengthscale estimation to inclusion-probability models and predictive-ARD optimization.The scalable spike-and-slab variational GP returns inclusion probabilities at SVGP-like cost, while predictive-ARD optimizes predictive rather than marginal likelihood.Source: §3.3 · p. 11

3.4 From relevances to selections: thresholding rules

  • Selection rules include fixed cutoffs, relevance gaps, inert-reference calibration, and posterior inclusion thresholds, each imposing a decision rule beyond ARD’s soft relevance ranking.Fixed cutoffs depend on problem scale; gap rules are unclear when relevances vary smoothly.Source: §3.4 · p. 12
  • Model-X knockoffs provide finite-sample false-discovery-rate control, a guarantee raw lengthscale thresholding cannot offer.The cited nonparametric additive-model adaptation combines knockoffs with random-feature approximations and subsampling.Source: §3.4 · p. 12

3.5 Software

  • ARD kernels and marginal-likelihood optimization are available across major GP libraries, with some packages also supporting automatic differentiation, GPUs, and scalable variational GPs.Fully Bayesian inference is available through probabilistic-programming interfaces and SAAS implementations.Source: §3.5 · p. 12

3.6 A practical decision guide

  • For small samples and low-to-moderate dimension, the guide recommends exact ARD with standardized inputs, restarts, and inert-reference calibration; for high dimension or large samples, it recommends stronger priors or scalable approximations.The scalable spike-and-slab variational GP is recommended when both scale and calibrated selection are required.Source: §3.6 · p. 13
  • When selection itself matters, the guide favors methods with inclusion probabilities, established consistency, or prediction-based relevance over raw lengthscales, especially for mixed linear and nonlinear effects.Sensitivity-based relevance is recommended when comparing linear and nonlinear effects of equal predictive importance.Source: §3.4 · p. 12, §3.6 · p. 13

4 Asymptotic Properties and Theoretical Guarantees

The survey distinguishes an infill-sampling obstruction to consistent estimation of individual GP lengthscales from support-recovery guarantees for hierarchical GP priors under high-dimensional random design. Those guarantees do not establish consistency for plain marginal-likelihood ARD, whose lengthscale-based selections remain heuristic.

4.1 Identifiability under fixed-domain asymptotics

  • Under fixed-domain infill, Zhang (2004) shows that only the exponential covariance ratio σ^2/φ is consistently estimable, not σ^2 and φ individually.Equivalent Gaussian measures explain why observations within a bounded domain cannot distinguish the individual parameters; this obstructs unconditional consistency claims for raw ARD lengthscales.Source: §4.1 · p. 14

4.2 Selection consistency under high-dimensional asymptotics

  • Jiang and Tokdar (2021) establish vanishing posterior probability of selecting the wrong subset for hierarchical GP priors when the truth is sparse and has finite smoothness.Their result assumes i.i.d. random design with growing predictor dimension; the proof uses finite smoothness to penalize over-specified models, and rougher truths permit larger candidate dimensions.Source: §4.2 · pp. 14–15
  • Chu et al. (2011) establish oracle-type selection consistency for the mean component of a spatial linear model using covariance tapering and a one-step sparse estimator.This guarantee is for a penalized-likelihood construction, not plain ARD.Source: §4.2 · p. 15

4.3 Reconciling the two results

  • The results are compatible because Zhang (2004) concerns covariance-hyperparameter identifiability under infill, whereas Jiang and Tokdar (2021) concern support recovery with growing-dimensional i.i.d. design.A discrete inclusion pattern can be recoverable even when continuous covariance parameters are not consistently estimable.Source: §4.3 · p. 15

4.4 Implications for plain ARD

  • Plain marginal-likelihood ARD with lengthscale thresholding has no established selection-consistency guarantee in the results surveyed.The hierarchical guarantee changes the inferential target to a discrete inclusion vector; direct consistency theory for plain ARD remains open.Source: §4.4 · pp. 15–16

5 Advantages, Disadvantages, and Failure Modes

ARD is convenient, uncertainty-aware, and flexible, but its effective sparsity becomes harder to interpret and threshold as dimension grows. Its main boundaries are weak selection margins, uncertain lengthscales, computational cost, and no general selection guarantee for plain thresholding.

5.1 Advantages

  • ARD infers relevance jointly with function fitting, fits into standard GP libraries with little added implementation effort, and supplies posterior predictive uncertainty.Its flexibility also accommodates nonlinear effects and interactions, while inverse lengthscales provide an interpretable ranking.Source: §5.1 · p. 16

5.2 Disadvantages

  • Marginal-likelihood optimization is multimodal, and its reported relevance pattern depends on finding a good mode, often through random restarts.ARD also adds D hyperparameters and retains the GP cost of O(N^3) computation and O(N^2) storage per covariance factorization.Source: §5.2 · pp. 16–17
  • As D grows, irrelevant and weakly relevant inputs can have overlapping relevances, while distance concentration weakens the kernel-gradient signal for distinguishing them.The weakest relevant inputs are therefore especially difficult to resolve by thresholding.Source: §5.2 · p. 17
  • Under fixed-domain asymptotics, individual Matérn lengthscales may be weakly identified even when predictive fit is excellent, so relative rankings can be more stable than magnitudes.Relative-scale reporting and Bayesian propagation of hyperparameter uncertainty are suggested when magnitudes matter.Source: §5.2 · pp. 17–18
  • In Figure 4, the two strongest relevant inputs remain separated from inert inputs at D = 40, but the weakest loses nearly all thresholding margin as D grows.The eroding margin is graceful degradation, not a collapse of ARD.Source: §5.2 · p. 17, Figure 4 · p. 18
  • ARD can overstate nonlinear effects relative to equally predictive linear effects because lengthscales measure variation rate rather than contribution to prediction.It ranks rather than selects, so thresholding is required; evidence maximization can also overfit, especially when D is large relative to N.Source: §5.2 · p. 18

5.3 Summary of the trade-off

  • ARD suits low-to-moderate dimensions when quick, interpretable, uncertainty-aware rankings are wanted, but alternatives merit consideration for exact zeros, calibrated inclusion probabilities, balanced effect comparisons, or formal guarantees.Plain lengthscale thresholding has no general theorem guaranteeing recovery of the true active set.Source: §5.2 · pp. 18–19, §5.3 · p. 19

6 The Frontier Landscape of GP Variable Selection

The section maps GP variable-selection alternatives by how they express and discover relevance, then compares them with ARD on practical criteria.

6 The Frontier Landscape of GP Variable Selection

  • The alternatives are organized by the strategy each uses to express and discover input relevance.The section returns to why ARD remains the default despite principled alternatives.Source: §6 · p. 19
  • The comparison asks why ARD remains the default despite the range of available methods.This is the organizing question for the survey of alternatives.Source: §6 · p. 19
  • Table 1 summarizes the methods along practical comparison axes.The passage does not specify the table's individual axes or results.Source: §6 · p. 19

6.1 Probabilistic exact-sparsity methods

Exact-sparsity approaches replace ARD’s soft lengthscale ranking with probabilistic inclusion, shrinkage priors, or explicit penalties. They trade added modeling or computation for exact zeros, uncertainty-aware selection, or stronger high-dimensional behavior.

6.1.1 Spike-and-slab priors on relevance

  • Spike-and-slab priors put a point mass at zero and a continuous slab on each relevance, yielding posterior inclusion probabilities rather than ARD’s point lengthscales.Recent variational GP implementations produce these probabilities at a cost comparable to ordinary sparse variational GPs.Source: §6.1.1 · p. 19
  • The inert-input calibration device uses a guaranteed-irrelevant artificial input to define a noise floor for judging genuine inputs.This reference-distribution idea can also be grafted onto ARD.Source: §6.1.1 · p. 19

6.1.2 Fully Bayesian relevance and the SAAS prior

  • Fully Bayesian relevance integrates over lengthscales with MCMC, exploring optimizer multimodality and returning posterior relevance distributions at substantially greater computational cost.Each sampler iteration still requires a cubic covariance factorization.Source: §6.1.2 · p. 20
  • SAAS uses a global half-Cauchy scale to shrink inverse lengthscales toward zero and local heavy-tailed scales to let a few inputs escape; its inference uses the No-U-Turn sampler.The authors report that sampling rather than maximizing is essential for recovering sparse structure in high dimension.Source: §6.1.2 · p. 20

6.1.3 Global–local shrinkage: the Laplace and horseshoe priors

  • The Laplace prior is the Bayesian lasso counterpart, but its single global penalty and light tails cannot both strongly suppress noise and leave large signals unshrunk.Its posterior mean is not sparse, despite a sparse posterior mode.Source: §6.1.3 · p. 21
  • The horseshoe combines a spike at zero with heavy tails, while SAAS applies a horseshoe-type global–local hierarchy to GP inverse squared lengthscales.For GP lengthscales, the cited work reports near-minimax, dimension-adaptive posterior contraction; this is a prior on kernel relevances, not literally the normal-means horseshoe estimator.Source: §6.1.3 · p. 21

6.1.4 Penalized-likelihood kriging

  • Penalized-likelihood kriging adds an explicit sparsity-inducing penalty to relevance parameters, producing exact zeros within a frequentist point-estimation workflow.The tuning parameter can be selected by cross-validation or an information criterion; Bayesian bridge GP regression instead imposes norm constraints and samples from the constrained posterior.Source: §6.1.4 · p. 22

6.2 Dependence, sensitivity, and projection methods

Dependence and predictive-sensitivity criteria assess relevance outside ARD’s lengthscale objective, while projection methods target low-dimensional directions rather than individual coordinates. The choice depends on whether the aim is predictive relevance, coordinate selection, or rotated structure.

6.2.1 Kernel-dependence and sensitivity criteria

  • HSIC measures input-response dependence and is zero if and only if they are independent for characteristic kernels; backward-elimination selectors use it to remove weak contributors.These selectors are model-agnostic rather than GP-specific.Source: §6.2.1 · p. 22
  • Predictive-sensitivity scores measure changes in GP predictions under input perturbations, including expected squared predictive-mean derivatives averaged over the input distribution.Unlike lengthscales, these scores assess effect on prediction and avoid confusing rapid small-amplitude variation with large-amplitude trends.Source: §6.2.1 · pp. 22–23
  • Projection-predictive selection projects a reference GP onto sparse submodels and chooses the smallest one that predicts almost as well.Sensitivity and projection methods require a fitted reference GP and additional sensitivity computation or submodel search.Source: §6.2.1 · p. 23

6.2.2 Active subspaces and linear projection

  • Active subspaces use dominant eigenvectors of the average gradient outer product to identify directions of variation, addressing rotated structure that coordinate-wise ARD can mis-specify.General projections are less interpretable as variable lists; sparse projection methods seek a middle ground by encouraging directions to involve few original inputs.Source: §6.2.2 · p. 23

6.3 Structured additive kernels

Structured additive kernels replace a flat relevance ranking with interpretable contributions from inputs and interactions. Additive models can also distinguish zero, linear, and nonlinear feature effects.

6.3 Structured additive kernels

  • Structured sums replace a flat relevance ranking with a decomposition of the function by input and interaction order.Order variances indicate relevance at the interaction-order level, while lengthscales can carry ARD within additive terms.Source: §6.3 · p. 24, §6.3.1 · p. 24

6.3.1 Additive and compositional kernels

  • Additive kernels decompose variance by inputs and interactions, complement ARD, and can mitigate dimensionality problems through low-order structure.Structure search over kernel expressions is combinatorial; additive models are especially useful when interactions or variance decomposition matter.Source: §6.3.1 · p. 24
  • VARD classifies each feature as exactly zero, linear, or nonlinear within a sparse additive regression model.It estimates one-dimensional feature components with a variational ARD procedure rather than inferring effect type from a product-kernel lengthscale.Source: §6.3.1 · p. 24

6.4 Why ARD remains popular

ARD remains popular because it fits into ordinary GP kernel learning with little extra machinery, is widely available, and is inexpensive to try. Its empirical performance is good in low-to-moderate dimensions, while alternatives address needs it does not meet as well.

Why ARD remains popular

  • ARD adds no new inference machinery: it uses the ordinary kernel and marginal-likelihood optimization, and is available in major libraries.Its output is an immediately interpretable relevance ranking, and trying it costs little.Source: §6.4 · p. 24
  • In low-to-moderate dimensions and with modest sample sizes, ARD performs well empirically and recovers relevant inputs in illustrated cases.It uses a few gradient-based optimizations rather than thousands of MCMC iterations.Source: §6.4 · pp. 24–25
  • Alternatives are useful when users need exact zeros, calibrated probabilities, mixed linear and nonlinear effects, or formal guarantees.The relative advantages of alternatives include exact sparsity and inclusion probabilities, but come with a computation trade-off for MCMC-based methods.Source: §6.4 · pp. 24–25

7 Scalability and Large-N, Large-p Regimes

Exact ARD inherits GP computation costs and adds optimization and statistical challenges as input dimension grows. Scalable approximations and sparse-covariance methods extend relevance learning to larger datasets and high-dimensional settings.

7 Scalability and Large-N, Large-p Regimes

  • ARD scalability is limited by GP computation, with additional optimization and reliability challenges as input dimension grows.Approximations are compatible with ARD kernels and retain per-dimension relevance estimates.Source: §7 · p. 25, §7.4 · p. 26, §7.5 · p. 26

7.1 The cubic bottleneck

  • ARD fitting costs O(N^3) per factorization and O(N^2) memory; each evidence-maximization step requires a fresh factorization, while multi-start multiplies that cost by the number of restarts.At N = 10^5, dense factorization is infeasible on commodity hardware.Source: §7.1 · p. 25

7.2 Gradients and their cost

  • Given the Cholesky factorization, all D+2 likelihood derivatives cost O(N^2D) total, so factorization—not gradient formation—is the bottleneck.Analytic gradients enable practical evidence maximization and are available in GP libraries.Source: §7.2 · p. 26

7.3 Ill-conditioning and the nugget

  • Large lengthscales and nearby inputs can make the covariance matrix ill-conditioned; a nugget stabilizes factorization but slightly biases fits toward smoothness.The nugget must be large enough for numerical stability but small enough to avoid distorting the model.Source: §7.3 · p. 26

7.4 The dimensional burden of ARD specifically

  • High dimension makes ARD harder both computationally and statistically: its larger hyperparameter space complicates optimization, while irrelevant lengthscales crowd selection thresholds.The passage identifies strongly regularized Bayesian priors and scalable spike-and-slab methods as alternatives in this regime.Source: §7.4 · p. 26

7.5 Scalable approximations

  • Inducing-point SVGP, structured or low-rank methods, NNGP, and Vecchia offer scalable options, with the choice guided by dimension and data geometry.ARD relevances remain available through approximate marginal-likelihood or variational optimization; exact ARD is suitable up to a few thousand observations.Source: §7.5 · pp. 27–28
  • N = 10^6 and D = 10^3: VGPR runs in well under an hour, comparable to lasso and regression trees on the same data.VGPR uses a penalized Vecchia log-likelihood with sequential covariate selection and an adaptive bridge penalty.Source: §7.5 · p. 28

8 Evaluation Frameworks and Application Domains

Evaluation should distinguish variable selection from prediction because their metrics can disagree. Applications use ARD and alternatives according to their data regimes and decision needs.

8 Evaluation Frameworks and Application Domains

  • Evaluation criteria differ depending on whether the goal is selecting inputs or predicting responses.The section distinguishes known-support selection studies from prediction-focused assessment.Source: §8 · p. 28

8.1 Evaluating variable selection

  • For known active sets, precision, recall, and false discovery rate assess selection; ARD rankings are evaluated with threshold-swept ROC or precision–recall curves.Bayesian methods with inclusion probabilities can also be assessed for calibration, while prediction calls for out-of-sample error or log predictive density; the two metric families can disagree.Source: §8.1 · pp. 28–29

8.2 Application domains

  • ARD appears in emulation, spatial statistics, genomics, engineering, and Bayesian optimization, with methods chosen to match dimensionality and analysis goals.The survey describes ARD as an immediate first choice, with alternatives used when dimension, exact selection, calibrated uncertainty, or interaction-focused analysis calls for them.Source: §8.2 · pp. 29–30

9 Discussion and Open Problems

ARD is widely used because marginal-likelihood optimization offers a low-cost, interpretable relevance ranking, but it provides only effective sparsity and lacks general selection-consistency guarantees. The survey maps its limitations to alternatives and open problems, while emphasizing that modern GP variable-selection methods increasingly combine complementary approaches.

9 Discussion and Open Problems

  • ARD offers only effective sparsity, no calibrated inclusion probabilities or standard-form hyperparameter uncertainty, and no general selection-consistency guarantee.It can also misrank relevance when linear and nonlinear effects are mixed.Source: §9 · p. 30
  • In low-to-moderate dimensions, ARD’s marginal-likelihood mechanism usually ranks relevance correctly at negligible added cost, helping explain its widespread use.It is the default anisotropic kernel, fitted through the same optimization and available as a library option.Source: §9 · p. 30
  • Match alternatives to ARD’s failure regime: spike-and-slab for exact sparsity, SAAS for high-dimensional selection, and sensitivity or projection methods for prediction impact.Additive or compositional kernels expose interactions, active subspaces address rotated structure, and penalized kriging offers consistency theory.Source: §9 · p. 30
  • Inducing-point, structured, and Vecchia approximations relax the cubic bottleneck, enabling reported variable selection at a scale of one million observations and one thousand inputs.Cao et al. (2022) report this scale.Source: §9 · pp. 30–31
  • Strong sparsity priors with integrated uncertainty address high-dimensional selection, while scalable variational methods add inclusion probabilities to GP inference.The survey identifies these as developments that narrow ARD’s limitations within its own framework.Source: §9 · p. 31
  • Open problems include selection consistency for marginal-likelihood ARD, prediction-centered relevance for mixed linear and nonlinear effects, and calibrated thresholding rules.The proposed consistency direction is to target identifiable microergodic parameters; the inert-reference threshold is useful but ad hoc.Source: §9 · p. 31
  • GP variable-selection methods increasingly combine continuous lengthscales, inclusion indicators, shrinkage, structured decompositions, and prediction-centered criteria rather than treating them as rivals.ARD’s lasting contribution is the idea that relevance can be learned automatically as a by-product of fitting rather than imposed in advance.Source: §9 · pp. 31–32

Funding

The funding statement reports no funding.

Funding

Loading 2609.33550v1…