PaperlayerView exact revision on arXiv

Source-linked AI summary

Scalable Variable Selection under Predictor Dependence with Adaptive Virtual Dummies

Taulant Koka, Michael Muma

arXiv:2609.31257v1eess.SPstat.ME

TL;DR

High-dimensional variable selection needs scalable error control, while predictor dependence can distort competition between real predictors and spherical dummies. The paper adapts virtual dummies using conditional Gaussian projections and shares real-design computations across paths; it reports stable empirical FDR and substantial runtime gains at large scale.

  • Problem

    Under predictor dependence, spherical dummies can misrepresent null competition, while repeated forward paths recompute shared real-design products.

    Source: §1 · p. 1

  • Method

    Sphere AG fits conditional Gaussian projections for inactive predictors and maps draws onto the remaining sphere radius, while a shared lazy Gram cache reuses real-design products across paths.

    Source: §1 · p. 1, §2.1 · p. 2, §2 · p. 2

  • Results

    Sphere AG yields stable empirical FDR with some power loss in highly dependent regimes, while the cached method delivers more than a sixfold speedup and runs at p = 100000 where tested T-Rex+DA and Model-X are not viable.

    Source: §5 · p. 4

  • Takeaways & Limitations

    The combined approach preserves paths while scaling to p = 100000, where tested T-Rex+DA and Model-X are not viable.

    Source: §5 · p. 4

    The pooled Gaussian fit may miss a small subset of strongly response-aligned nulls, and future work targets localized or mixture-based laws with finite-sample FDR guarantees.

    Source: §5 · p. 4

Abstract

from arXiv · show

Reliable high-dimensional variable selection requires scalable error-controlling methods. The Terminating-Random Experiments (T-Rex) selector estimates the false discovery rate (FDR) by aggregating early-terminated forward-selection paths in which predictors compete with synthetic dummies. We address two remaining challenges: i) predictor dependence can bias dummy-predictor competition; ii) computation is wasted on recomputing terms shared across experiments. Building on memory-efficient virtual dummies, which sequentially sample projections from their exact conditional law, we estimate the conditional Gaussian law of inactive predictors and map draws onto the remaining sphere radius. A shared lazy Gram cache computes the response product and each requested Gram column once across experiments. Simulations across three covariance structures show that uniform spherical dummies can exceed the target FDR, whereas the proposed method empirically controls FDR. Caching yields more than a sixfold speedup, and the method remains feasible with 100000 predictors, where competing FDR-controlling methods become computationally impractical.

1. INTRODUCTION

High-dimensional variable selection must identify response-associated predictors while controlling irrelevant reports. The introduction motivates T-Rex, then presents dependence-adaptive virtual dummies and shared caching to address statistical and computational limitations.

1. INTRODUCTION

  • Reliable high-dimensional variable selection seeks response-associated predictors while controlling irrelevant reports through the false discovery rate (FDR).Source: §1 · p. 1
  • When p ≥ n, standard Benjamini–Hochberg and Benjamini–Yekutieli p-values are unavailable without assumptions or pre-screening, which can reduce power.Source: §1 · p. 1
  • T-Rex aggregates early-terminated forward-selection paths where real predictors compete with dummies; T-Rex+DA addresses dependence but adds grouping and threshold-search costs and can be conservative.Explicitly storing and processing an n × L dummy matrix can also dominate resources at very large scales.Source: §1 · p. 1
  • Virtual dummies generate only path-required coordinates, but independent spherical dummies ignore anisotropic null competition and repeated paths recompute shared real-design products.Source: §1 · p. 1
  • The paper proposes Sphere AG, which fits conditional Gaussian projections of inactive predictors and maps draws to the remaining sphere radius, plus shared lazy Gram caching across paths.The cache computes the response product once and each requested real Gram column once, without constructing the full Gram matrix.Source: §1 · p. 1

2. DEPENDENCE-ADAPTIVE VIRTUAL DUMMIES

Under grouped-factor dependence, ordinary spherical T-Rex increasingly exceeds the target FDR as dependence grows, while Sphere AG remains empirically below it, with some power loss under strong dependence. Sphere AG adapts dummy projections to inactive predictors’ path-dependent distribution while preserving spherical geometry.

2. DEPENDENCE-ADAPTIVE VIRTUAL DUMMIES

  • Under dependence, inactive-real projections are anisotropic and path-dependent, whereas spherical dummy projections are not, motivating adaptation of the dummy law.The method retains T-Rex stopping, voting, and FDP-estimation rules, changing only the conditional law used to generate dummy projections.Source: §2 · pp. 1–2
  • As ρ grows, spherical T-Rex increasingly exceeds the target FDR, while Sphere AG remains empirically below it, with power loss under strong dependence.The comparison is under grouped-factor dependence; Fig. 1 covers 100 paired trials and marks the target FDR at 0.2.Source: §2 · p. 2, Figure 1 · p. 2
  • Virtual dummies reveal coordinates along an adaptive basis as needed, storing O(Lk) scalars after k directions rather than O(nL) dummy entries.If a dummy is selected, its unrevealed component is completed from its conditional law; under rotational invariance, this reproduces explicit-dummy selection exactly.Source: §2 · p. 2

2.1. Sphere-constrained adaptive Gaussian dummies

  • Sphere AG estimates inactive predictors’ conditional Gaussian projection law and maps draws onto the remaining sphere radius to preserve unit norm.When a dummy is selected, its remaining component is sampled uniformly on the corresponding sphere; outside the isotropic special case, this approximates inactive predictors’ path-dependent projection distribution.Source: §2.1 · p. 2
  • The geometric guarantee does not establish FDR control: fitted moments depend on the design and selected path, so Sphere AG’s FDR behavior is evaluated empirically.The method therefore lacks the null–dummy exchangeability used by the standard T-Rex proof.Source: §2.1 · p. 3

3. SHARED LAZY GRAM CACHING

Shared lazy Gram caching avoids repeated real-design products across T-Rex experiments while computing Gram columns only when first requested. A memory budget limits retained columns without forming the full Gram matrix.

3. SHARED LAZY GRAM CACHING

  • The shared cache computes X⊤y once and each requested real Gram column X⊤x_j on its first request, then reuses it across experiments.It retains a memory budget and does not form the full p × p Gram matrix.Source: §3 · p. 3
  • Under virtual dummies, the real design remains the shared cost-dominating object, while virtual-dummy projections are generated along each realized path.OMP’s residual correlations combine persistent response and real-Gram terms with experiment-specific virtual-dummy projections; LARS also uses the cached real Gram columns.Source: §3 · p. 3
  • Caching leaves inferential output unchanged in exact arithmetic, and when rounding does not change candidate ordering.Cached and direct multiplication return the same inner products.Source: §3 · p. 3
  • With sufficient cache capacity, M_G counts distinct real predictors selected by any experiment, so repeated requests across paths increase reuse.The cost accounting separates the initial response product, first-time real-column requests, and experiment-specific dummy directions.Source: §3 · p. 3

4. EXPERIMENTS

Across the reported SNR and dependence comparisons, Sphere AG stays below the target FDR for both solvers, with power varying by setting. The experiments also report a trade-off in the Table 2 comparison and describe the covariance settings used.

4. EXPERIMENTS

  • Unless otherwise stated, simulations use n = 500, p = 1000, p1 = 10, B = 20, and L = 3p, with columns centered and normalized to unit norm.SNR sweeps fix block-Toeplitz dependence at ρ = 0.9, Student-t3-driven block loadings at ρ = 0.4, and grouped-factor dependence at ρ = 0.3; dependence sweeps fix SNR at one. Both block models use blocks of size 100.Source: §4 · p. 3
  • Sphere AG stays below the target FDR for both solvers, while its TPP rises with SNR and falls under grouped-factor dependence.T-Rex+DA’s FDR rises with dependence, especially for LARS in the grouped-factor model; Model-X+ stays below target, while Model-X exceeds it at several points.Source: Figure 2 · p. 4, §4 · p. 4
  • Sphere AG remains below the target with increasing power; spherical LARS has FDR near .885 and selects about 89 variables.The tested Model-X implementation would need 80 GB RAM and could not run on the 64 GB machine, while the T-Rex+DA grouping step was not viable.Source: §4 · p. 4

5. CONCLUSION

Sphere AG adapts dummy projections to predictor dependence while retaining spherical geometry, and combines virtual sampling with shared caching for scalable selection. It shows stable empirical FDR with some power loss in challenging dependence, but its pooled fit may miss strongly response-aligned nulls.

  • Sphere AG learns path-conditional projections from inactive predictors while preserving the spherical geometry induced by standardization.Source: §5 · p. 4
  • Sphere AG yields stable empirical FDR, with some power loss in challenging highly dependent regimes.Source: §5 · p. 4
  • More than sixfold speedup comes from virtual sampling and shared lazy Gram caching; Sphere AG runs at p=100000 where tested T-Rex+DA and Model-X are not viable.The large-scale grouped-factor study used n=1000 and 100 trials per SNR; Model-X and T-Rex+DA could not be run.Source: §5 · p. 4, Table 2 · p. 4
  • The pooled Gaussian fit may miss a small subset of strongly response-aligned nulls.Future work will focus on localized or mixture-based projection laws with finite-sample FDR guarantees.Source: §5 · p. 4
Loading 2609.31257v1…