PaperlayerView exact revision on arXiv

Source-linked AI summary

$L_0$ Feature Selection via Monte Carlo Tree Search in High-Dimensional Financial Regression

Cameron Mouangue, Tristan Cazenave

arXiv:2610.13574v1cs.LGq-fin.CPq-fin.ST

TL;DR

In high-dimensional financial regression, Lasso can shrink correlated signals and exact sparse selection is computationally difficult and often optimized on training error. The paper applies validation-guided MCTS with GNRPA to feature-subset selection; its linear ensemble reaches out-of-sample R2 = 0.6961, and selected subsets outperform convex baselines on this dataset.

  • Problem

    Lasso shrinkage can restrict selection when informative financial signals are spread across correlated variables, while exact L0 selection is computationally difficult and commonly optimized on training error rather than out-of-sample performance.

    Source: §1 · p. 1

  • Method

    The framework formulates sparse feature selection as a sequential decision process and uses GNRPA tree search to score candidate subsets by validation performance, with a lexicographic objective for subset size and multicollinearity.

    Source: §1 · pp. 1–2

  • Results

    A linear ensemble of selected subsets reaches out-of-sample R2 = 0.6961, and sparse subsets outperform convex baselines out of sample on this dataset.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    MCTS can accommodate objectives such as favoring lower-MaxVIF subsets when explanatory performance is similar, while the linear ensemble preserves additive performance attribution.

    Source: §5 · p. 8

    The improved out-of-sample fit comes with higher computational cost and tuning burden than convex baselines.

    Source: §5 · p. 8

Abstract

from arXiv · show

Feature selection is a central problem in high-dimensional financial regression, where large numbers of engineered variables and strong correlations among predictors reduce model stability and interpretability. $L_1$ regularization (Lasso) is widely used for sparse regression, but its shrinkage can be restrictive when informative signals are spread across correlated variables. Exact sparse regression avoids this shrinkage but requires solving a combinatorial optimization problem that is computationally difficult and is typically fit to minimize in-sample rather than out-of-sample error. This paper presents a feature selection framework based on Monte Carlo Tree Search with Generalized Nested Rollout Policy Adaptation (GNRPA). Feature selection is formulated as a sequential decision process, and candidate subsets are scored by their out-of-sample validation performance rather than training error. The method also supports a lexicographic objective accounting for subset size and multicollinearity. We study this approach on a contemporaneous explanatory regression task: explaining the weekly return of the NASDAQ-100 index from 1,072 engineered features spanning market, macroeconomic, and factor data. This isolates the value of subset selection under multicollinearity from the separate problem of predictive forecasting. On this task, the proposed method identifies sparse feature subsets with higher out-of-sample explanatory power than an optimized Lasso baseline, reaching an out-of-sample $R^2$ of 0.6961 versus 0.6313 for Lasso. These results suggest that Monte Carlo Tree Search is a practical alternative to convex regularization for sparse explanatory feature selection in financial regression, with potential value for factor attribution and risk decomposition where transparency matters as much as out-of-sample fit.

1 Introduction

In high-dimensional financial regression, correlated predictors can make Lasso shrinkage restrictive, while exact sparse selection is computationally difficult and often optimizes training error. The paper uses validation-guided MCTS for subset selection and reports improved out-of-sample explanatory performance on a 1,072-feature NASDAQ-100 task.

1 Introduction

  • Lasso shrinkage can discard or over-shrink correlated financial signals, while exact L0 selection avoids shrinkage but is NP-hard and may overfit when optimized on training error.These trade-offs motivate validation-based subset selection for noisy financial data.Source: §1 · p. 1
  • The framework adapts GNRPA within MCTS to search sequentially over feature subsets using validation performance and a lexicographic objective for subset size and multicollinearity.It supports fixed-cardinality and adaptive stopping strategies.Source: §1 · pp. 1–2
  • The study evaluates the approach on 1,072 engineered features for weekly NASDAQ-100 returns, with predictors observed at the same time as the target.Accordingly, reported R2 values measure explanatory regression rather than out-of-sample return prediction.Source: §1 · pp. 1–2
  • The method identifies smaller feature subsets with improved out-of-sample explanatory performance over optimized convex regularization on this dataset.The task explains weekly NASDAQ-100 returns using contemporaneous predictors, not return forecasting.Source: §1 · pp. 1–2

2 Related Work

Prior financial feature-selection methods trade off variable-level interpretability, shrinkage, and computational scalability, while many do not directly optimize out-of-sample performance. The paper situates validation-scored GNRPA subset search within MCTS and reinforcement-learning approaches to combinatorial optimization.

2.1 Feature Selection for Financial Regression

  • PCA and neural-network selection offer alternatives, but may not produce globally sparse linear models; Lasso retains original variables yet can shrink correlated signals.Exact subset selection avoids coefficient shrinkage, unlike Lasso.Source: §2.1 · p. 2
  • Exact-subset methods do not directly optimize out-of-sample performance; mixed-integer solvers can become prohibitive, while coordinate descent can stall when collinearity requires coordinated changes.The latter methods minimize penalized training error.Source: §2.1 · p. 2

2.2 Monte Carlo Tree Search and GNRPA

  • MCTS estimates candidate-action values through repeated simulations, while GNRPA adapts rollout policies using temperature control and heuristic biases for higher-dimensional search.Prior work also used reinforcement learning to learn search policies over large discrete spaces.Source: §2.2 · p. 2, §2.1 · p. 2
  • The paper adapts GNRPA to sparse financial regression by treating feature selection as sequential decisions and evaluating candidate subsets on validation performance.This application addresses an area described as having received limited attention.Source: §2.2 · p. 2

3 Methodology

The methodology combines a contemporaneous financial dataset with validation-guided sequential subset search, using correlation-aware heuristics and adaptive stopping.

3.1 Dataset

  • The 1,072-feature dataset models contemporaneous weekly NASDAQ-100 returns, so its setup is explanatory rather than lead-lag forecasting.Features include market, macroeconomic, and factor variables; technical indicators use lookbacks of 5 to 252 trading days.Source: §3.1 · p. 3

3.2 Problem Formulation

  • The MDP constructs subsets sequentially by selecting an unchosen feature or stopping, and scores terminal subsets on validation performance.Search supports fixed-cardinality and adaptive modes.Source: §3.2 · p. 3

3.3 Search Heuristics

  • An mRMR heuristic guides search through the large subset space by trading off target relevance against redundancy with selected features.The redundancy measure can average across the subset, use RMS, or focus on the top-k correlated features; endogenous stopping uses top-3 RMS.Source: §3.3 · pp. 3–4

3.4 Adaptive Search

  • Adaptive search removes the need to pre-specify subset size and ranks terminal subsets by validation R^2, then smaller size, then lower MaxVIF.MaxVIF serves as a tie-breaker for multicollinearity.Source: §3.4 · pp. 4–5
  • The stopping bias balances adding features against termination, increasing near Kmax while allowing exploration near Kmin.The endogenous-stopping configuration also uses conservative base-level and larger higher-level learning rates, with root-level early stopping after P iterations without improvement.Source: §3.4 · p. 4

4 Experimental Results

The experiments compare validation-guided GNRPA configurations with financial regression baselines on a held-out NASDAQ-100 explanatory task. They also examine the trade-offs among explanatory fit, sparsity, computational cost, and ensemble stability.

4.1 Experimental Setup and Baselines

  • On the held-out January 2024–March 2026 window, GNRPA configurations were evaluated as explanatory regressions, with validation and hyperparameter tuning kept separate from the test period.The internal validation window immediately preceded the test period; the authors note a trade-off between regime-representative validation and protection against regime change.Source: §4.1 · p. 5
  • Lasso achieved OOS R2 of 0.6313 with 60 features, but tightening it to 14–16 features reduced OOS R2 to 0.39–0.43, versus 0.6648 for GNRPA.Elastic Net selected 55 features with OOS R2 of 0.6243, while forward stepwise selection stopped at five features with OOS R2 of 0.2489.Source: §4.1 · p. 5

4.2 Explanatory Performance

  • 0.6961 OOS R2 for the soft-voted ensemble versus 0.6313 for optimized Lasso, with 59 combined variables and 1671.6 seconds of search time.Table 1 reports performance, execution time, and sparsity over the 562-day test window.Source: §4.1 · p. 5, Table 1 · p. 6, §4.2 · p. 6
  • The endogenous-stopping ensemble reached mean R2 of 0.6682 and worst-run R2 of 0.650, while tuned cardinality reached mean R2 of 0.6648 but had a worst run of 0.519.The tuned-cardinality configuration was faster and its mean was significant versus Lasso; endogenous stopping's mean was also significant versus Lasso.Source: §4.2 · p. 6

4.3 Implementation Details and Hyperparameter Optimization

  • Independent runs were combined by soft voting, with batches of six added until aggregate validation R2 changed by less than 0.01 across two consecutive batches.Because Lasso and Elastic Net are deterministic at a fixed regularization strength, comparisons include both individual stochastic-search runs and their ensemble.Source: §4.3 · p. 6

4.4 Interpretability and Ensemble Stability

  • The selected 15-feature subset forms a distributed-lag model, including opposing VIX lag coefficients and consecutive negative coefficients for high-yield spread changes.The subset's composition is shown in Table 4, and Figure 2 displays its correlation matrix.Source: §4.4 · p. 6, Table 4 · p. 7, Figure 2 · p. 7
  • The endogenous-stopping ensemble used 59 unique features versus 74 for tuned cardinality, suggesting more consistent feature selection across runs while retaining linear performance attribution.The authors identify selection frequency and inter-run Jaccard similarity as possible future stability measures.Source: §4.4 · p. 6

5 Discussion and Conclusion

The paper presents validation-guided L0 feature selection as a way to avoid convex shrinkage while retaining interpretable linear models. On this dataset, the linear ensemble reaches an out-of-sample R^2 of 0.6961, with higher computational cost and evaluation limited to one index and window.

  • The framework formulates validation-guided L0 subset selection as a sequential Markov Decision Process, avoiding convex shrinkage bias while retaining linear-model interpretability.Source: §5 · p. 8
  • 0.6961 out-of-sample R^2 is reached by a linear ensemble of selected subsets, which preserves additive performance attribution and outperforms convex baselines on this dataset.Source: §5 · p. 8
  • When explanatory performance is similar, maximum Variance Inflation Factor guides the search toward subsets with lower internal multicollinearity.Source: §5 · p. 8
  • The improved out-of-sample fit comes with higher computational cost and tuning burden than convex baselines, though reported single-workstation Python runtimes are conservative.Source: §5 · p. 8
  • The evaluation covers one index and one out-of-sample window, January 2024–March 2026; rolling windows and additional asset classes remain future directions.Source: §5 · p. 8
  • GNRPA builds solutions incrementally and adapts its policy, unlike genetic algorithms or simulated annealing, while cardinality-constrained index tracking is a proposed finance application.Source: §5 · p. 8

Related papers

Loading 2610.13574v1…