PaperlayerView exact revision on arXiv

Source-linked AI summary

VARX-L: Structured Regularization for Large Vector Autoregressions with Exogenous Variables

William Nicholson, David Matteson, Jacob Bien

arXiv:1508.07497v4stat.AP

TL;DR

Large VARX models face dimensionality and estimation problems even though including more component and exogenous series can support forecasting. The paper introduces VARX-L, a structured regularization framework using efficient penalized estimation and reference-model shrinkage. Across simulations and macroeconomic applications, its procedures consistently outperform benchmark procedures, with performance robust across sparsity structures and forecast horizons.

  • Problem

    Large VARX models are difficult to estimate and forecast when low-frequency data contain many endogenous and exogenous series, despite the value of including relevant variables.

  • Method

    VARX-L is a structured penalized multivariate regression framework that uses convex optimization for high-dimensional VARX estimation and can shrink toward reference models such as a vector random walk.

  • Results

    The proposed procedures consistently outperform benchmark procedures in simulations and macroeconomic forecasting applications, with performance robust across sparsity structures and forecast horizons.

  • Takeaways & Limitations

    VARX-L provides a flexible framework for forecasting high-dimensional, low-frequency VARX systems, including settings where the number of series approaches or exceeds the sample length.

  • Takeaways & Limitations

    The paper focuses solely on forecasting applications, and its current implementation requires a coherent maximal lag selection.

Abstract

from arXiv · show

The vector autoregression (VAR) has long proven to be an effective method for modeling the joint dynamics of macroeconomic time series as well as forecasting. A major shortcoming of the VAR that has hindered its applicability is its heavy parameterization: the parameter space grows quadratically with the number of series included, quickly exhausting the available degrees of freedom. Consequently, forecasting using VARs is intractable for low-frequency, high-dimensional macroeconomic data. However, empirical evidence suggests that VARs that incorporate more component series tend to result in more accurate forecasts. Conventional methods that allow for the estimation of large VARs either tend to require ad hoc subjective specifications or are computationally infeasible. Moreover, as global economies become more intricately intertwined, there has been substantial interest in incorporating the impact of stochastic, unmodeled exogenous variables. Vector autoregression with exogenous variables (VARX) extends the VAR to allow for the inclusion of unmodeled variables, but it similarly faces dimensionality challenges. We introduce the VARX-L framework, a structured family of VARX models, and provide methodology that allows for both efficient estimation and accurate forecasting in high-dimensional analysis. VARX-L adapts several prominent scalar regression regularization techniques to a vector time series context in order to greatly reduce the parameter space of VAR and VARX models. We also highlight a compelling extension that allows for shrinking toward reference models, such as a vector random walk. We demonstrate the efficacy of VARX-L in both low- and high-dimensional macroeconomic forecasting applications and simulated data examples. Our methodology is easily reproducible in a publicly available R package.

1 Introduction

Large VARs can incorporate relevant and disaggregated economic information but are difficult to estimate and forecast because their parameter spaces grow rapidly. The paper introduces VARX-L, a structured regularization framework designed for high-dimensional VARX forecasting with computationally efficient estimation and flexible shrinkage.

  • Motivation: Unrestricted VARs allow every series to depend linearly on the past values of all included series, creating overparameterization when data are low-frequency.Large unrestricted VARs can produce inaccurate forecasts and be sensitive to changes in economic variables.
  • Related approaches: Existing parameter-reduction approaches range from subjective, theory-based restrictions and Minnesota priors to empirical Bayesian and lasso-based regularization.Their trade-offs include restrictive assumptions, hyperparameter requirements, computational expense, and limited variable selection in some approaches.
  • Motivation: Large VARs may nevertheless improve forecasts by retaining potentially relevant variables and incorporating disaggregated measurements linked to aggregate outcomes.Prior work cited in the paper reports forecasting improvements for macroeconomic aggregates such as the Consumer Price Index.
  • Related approaches: Lasso-VAR methods are computationally efficient in high dimensions, perform variable selection, and readily compute multi-step forecasts with prediction intervals.The paper presents these properties as advantages over Bayesian approaches.
  • VARX-L contribution: VARX models add unmodeled exogenous variables, but standard subset-based selection can require fitting all models across lag orders for endogenous and exogenous series.This creates an additional dimensionality challenge for VARX estimation.
  • VARX-L contribution: VARX-L adapts structured regularization and convex optimization to estimate high-dimensional VARX models efficiently while imposing minimal assumptions on parameter structure.The framework supports forecasting when the numbers of component series and exogenous variables approach or exceed the series length, and includes shrinkage toward a vector random walk.

2 Methodology

VARX-L is a structured penalized multivariate regression framework for large VARX models. It reduces overparameterization by imposing sparsity patterns that reflect lagged, endogenous, and exogenous structure while enabling tractable estimation.

  • VARX formulation: A VARX_k,m(p,s) models a k-dimensional series using endogenous lags, exogenous lags, an intercept, and white-noise innovations.The model estimates k(1 + kp + ms) regression parameters without regularization.
  • VARX-L framework: VARX-L applies convex penalties to endogenous and exogenous coefficient matrices to reduce the VARX parameter space through sparsity.The framework uses penalty functions P_y(Φ) and P_x(β), with one penalty parameter shared across coefficients after standardizing series to a common scale.
  • Lag Group VARX-L: Lag Group VARX-L groups coefficients by lag, shrinking entire related groups toward zero and supporting structured variable selection without within-group sparsity.This structure is useful when series have comparable dynamics and can also act as a lag-selection tool.
  • Sparse Group VARX-L: Sparse Group VARX-L variants combine group and lasso penalties, allowing both group-level and within-group sparsity.The additional parameter α controls within-group sparsity, so active groups may contain coefficients set to zero.
  • Other penalty structures: Basic VARX-L uses an unstructured lasso, whereas Endogenous-First VARX-L uses a nested penalty that prioritizes endogenous coefficients before exogenous counterparts.The Endogenous-First structure requires an exogenous coefficient row to be nonzero only when the corresponding endogenous row is nonzero.
  • Estimation and selection: The proposed estimation procedures are computationally tractable in high dimensions, including an efficient group-lasso solution based on block coordinate descent and trust-region subproblems.Penalty parameters are selected sequentially in a rolling manner, with a procedure based on minimizing h-step-ahead mean-square error.
  • Own/Other Group VARX-L: Own/Other Group VARX-L separates each endogenous lag matrix into own-lag and cross-series groups, reflecting that diagonal coefficients may differ from off-diagonal coefficients.The grouping can set all coefficients associated with a series or exogenous relationship to zero as a unit.

3 High-Dimensional Macroeconometrics

The paper evaluates VARX-L regularization in two high-dimensional and one low-dimensional macroeconomic applications, using rolling cross-validation to select penalties and comparing forecasts across horizons. Across applications, structured sparse VARX-L procedures generally outperform benchmarks, with own/other-lag groupings often performing especially well.

  • The applications use US macroeconomic indicators in two high-dimensional settings and Canadian indicators with US series as exogenous predictors in a low-dimensional setting.
  • 3.1 Practical Implementation: Rolling cross-validation selects the penalty by minimizing h-step-ahead mean squared forecast error over sequential forecast origins.The procedure separates initialization, training, and forecast-evaluation periods and adds observations sequentially.
  • Macroeconomic Applications: Most VARX-L procedures substantially outperform benchmarks at both horizons, with Own/Other Group and Sparse Own/Other Group VARX-L achieving the best performance in one application.The results support distinguishing a series’ own lags from lags of other series.
  • Macroeconomic Applications: Sparsity appears crucial: Bayesian VAR, factor, and information-criterion VARX benchmarks generally perform worse than VARX-L procedures across the reported applications.The factor model’s poor performance also provides evidence against a low-rank structure in these applications.
  • Macroeconomic Applications: In the low-dimensional Canadian application, Sparse Lag Group VARX-L achieves superior performance at both horizons, and every VARX-L procedure beats its corresponding VAR-L.These results indicate that exogenous predictors contribute useful out-of-sample information in this application.
  • Model Confidence Sets: The model-confidence-set procedure distinguishes VARX-L models from benchmarks and retains an Own/Other Group variant in every application.In the Canadian application, no VAR-L models enter the model confidence set at either horizon.

4 Extending the VARX-L for Unit-Root Nonstationarity

The paper extends VARX-L to nonstationary macroeconomic series by shrinking coefficients toward reference matrices, including a vector random walk, rather than transforming the data to stationarity. In an application with untransformed series, Minnesota VARX-L procedures outperform the random walk, while structural choices affect performance across horizons.

  • Stationarity transformations can be labor-intensive and subjective, destroy information about long-run relationships, and are problematic for series with negative values.
  • The extension shrinks VARX-L coefficients toward known constant matrices, allowing reference models such as a vector random walk to represent mild non-stationarity.The Minnesota VARX-L uses a reference matrix that implements a variant of the Minnesota prior.
  • Application: The nonstationary application forecasts 20 untransformed macroeconomic series using 20 exogenous predictors and shrinks toward a vector random walk.
  • Results: Every Minnesota VARX-L procedure outperforms the random walk at both horizons; Own/Other Group Minnesota VARX-L is best at h = 1, while Basic VARX-L is best at h = 4.
  • Results: Forecasting performance depends substantially on structure: lag-based groupings perform relatively poorly at h = 1 but improve relative to other methods at h = 4.Their reduced performance is attributed to inability to distinguish the diagonal random-walk component from coefficients on other lags.
  • Results: AIC and BIC are uninformative in this nonstationary setting, while Bayesian VAR and factor-model forecasts are inferior to the VARX-L procedures and naive random walk.
  • Results: Considerable sparsity is imposed across horizons, and the results indicate that greater sparsity may be appropriate when shrinking toward a reference model rather than toward zero.

5 Simulation Scenarios

Simulation experiments evaluate VARX-L methods across matched and mismatched sparsity patterns, showing that structured penalties generally improve forecasting over benchmark procedures while the best structure depends on the data-generating pattern.

  • Scenario 1: Unstructured Sparsity: Scenario 1’s unstructured sparsity favors Basic VARX-L, while Sparse Own/Other VARX-L is the strongest structured alternative.Approximately 90 percent of coefficients are zero, with each coefficient independently active with 10 percent probability.
  • Scenario 2: Lag Group Sparsity: Scenario 2’s lag-group sparsity favors Lag Group VARX-L, and all structured approaches substantially outperform Basic VARX-L and the benchmark procedures.The results are attributed to effectively leveraging strong signals from exogenous predictors.
  • Scenario 3: Hybrid Sparsity: Scenario 3’s hybrid sparsity favors Basic VARX-L followed closely by Sparse Lag Group VARX-L, while information-criterion methods and BGR perform poorly.The benchmarks cannot account for within-lag sparsity, and BGR cannot perform variable or lag order selection.
  • Overall simulation findings: Across scenarios, VARX-L methods substantially outperform benchmark procedures and remain fairly robust when the imposed group structure does not match the true sparsity pattern.Scenario-specific winners generally correspond to structures that best represent the underlying coefficient pattern.

6 Conclusion

The conclusion presents VARX-L as a scalable and flexible structured-regularization framework that incorporates endogenous and exogenous predictors. Across simulations and macroeconomic applications, the proposed methods outperform benchmarks and remain robust across sparsity structures and forecast horizons.

  • Contribution: VARX-L reduces the parameter space while incorporating useful information from endogenous and exogenous predictors.The framework is presented as amenable to high-dimensional VARX settings.
  • Contribution: VARX-L models scale with data dimension and accommodate a wide variety of potential dynamic structures.
  • Empirical findings: Each proposed method consistently outperforms benchmark procedures in simulations and macroeconomic forecasting applications.
  • Empirical findings: Forecast performance appears robust across multiple sparsity structures and forecast horizons, while structured models tend to outperform Basic VARX-L on actual macroeconomic data.
  • Extensions: The paper focuses solely on forecasting applications, although the framework could be extended to structural analysis and policy evaluation.
  • Limitations: The current implementation requires coherent maximal-lag selection, and choosing lag order from data frequency can lead to overfitting.An increasing lag penalty is possible but would require multidimensional penalty selection and subjective specification of its functional form.

A.1 Compact Matrix Notation

The appendix rewrites the VARX model in compact matrix notation by stacking observations, lagged predictors, coefficient matrices, and residuals.

  • Matrix construction: Φ concatenates endogenous lag coefficient matrices, and β concatenates exogenous lag coefficient matrices.
  • Compact representation: B combines Φ and β, while U stacks the residual vectors; the VARX equation and least-squares objective then use these compact matrices.

A.2 Intercept Term

The appendix separates the intercept from the regularized coefficients, derives it from the centered data representation, and then applies coordinate-descent updates to the coefficient matrix.

  • Intercept treatment: The intercept ν̂ is not regularized and is derived separately from the unpenalized portion of the objective.
  • Intercept treatment: The intercept is obtained using row means of Y and Z, after which both matrices are treated as centered.
  • Coefficient updates: Coordinate descent updates each B_jr using a partial residual and a subgradient-based optimality condition.
  • Coefficient updates: The resulting update uses the soft-threshold operator, with the signum function and positive-part expression defining its components.

A.3.2 Lag Group VARX-L

The Lag Group VARX-L subproblem is transformed into a trust-region formulation and solved efficiently with block coordinate descent, active-set iterations, and eigenvalue-based calculations.

  • The one-lag problem for Φ(q) uses a partial residual and incorporates weighting into the penalty parameter λ.
  • The transformed subproblem is a trust-region problem whose radius ∆ corresponds to the optimal solution.
  • Block coordinate descent solves the transformed subproblems, with KKT conditions and spectral results characterizing the optimum.
  • The optimal ∆ is found by solving a root condition, while the relevant norm is computed efficiently through an eigen-decomposition of Gq.
  • An active-set algorithm cycles through groups and then updates only nonzero groups until convergence.
  • The active-set approach considerably reduces computation time when λ is large and most model coefficients are zero.

A.3.3 Own/Other Group VARX-L

The Own/Other Group VARX-L formulation transforms the grouped lag problem into least squares subproblems so own and other lags can be partitioned and optimized.

  • Because own and other lag groups are not proper submatrices, Equation (2.3) is transformed into a least squares problem.
  • The transformation defines new quantities and expresses the own-lag block subproblem for group qq in the transformed formulation.
  • The optimum for the own-lag block is characterized by a zero-subgradient condition.
  • After the least squares transformations, a slightly adapted version of Algorithm 3 can be applied.

A.3.4 Sparse Lag Group VARX-L

The sparse and endogenous-first VARX-L procedures combine proximal or gradient-based optimization with acceleration, active-set updates, and computational shortcuts for structured penalties.

  • Sparse Lag Group VARX-L: Sparse Group VARX-L uses gradient-based methods by decomposing the objective into differentiable and nondifferentiable components.
  • Sparse Own/Other Group VARX-L: Sparse Own/Other Group VARX-L follows almost the exact same procedure as its lag counterpart after least squares transformations.
  • Sparse Lag Group VARX-L: The sparse update linearizes the unpenalized quadratic loss around the current estimate and adds the penalty term.
  • Sparse Lag Group VARX-L: Nesterov acceleration yields a convergence rate of 1/j2, compared with 1/j for the standard proximal method.
  • Sparse Lag Group VARX-L: The method uses a constant step size based on a Lipschitz constant, whose matrix quantity can be computed from the maximum eigenvalue.
  • Sparse Lag Group VARX-L: The maximum eigenvalue is computed with the power method, retaining its eigenvector as a warm start during cross-validation and forecast evaluation.
  • Sparse Lag Group VARX-L: The active-set procedure updates all groups initially, then enters inner updates for active groups and repeats until convergence.
  • Endogenous-First VARX-L: Endogenous-First VARX-L uses proximal gradient descent with row-wise decoupling and closed-form nested block updates.

A.4 Banbura et al. (2009) Implementation

The Banbura et al. implementation uses a normal inverted Wishart prior represented through dummy observations and calibrated prior parameters.

  • The Bayesian VAR uses a normal inverted Wishart prior implemented through additional dummy observations.
  • The prior variance scales σ1, …, σk are estimated using univariate autoregressive models.
  • The intercept prior ϵ is set to a very small number, and δ is set to zero in forecasting applications except the Minnesota VARX-L application.
  • BGR additionally shrinks Π = (Ik − Φ1 − · · · − Φp) toward zero to impose a bound on the sum of coefficients.
  • The prior means μ1, …, μk capture the unconditional average level of each series, while τ is set to 10λ as a loose prior.
  • The augmented matrices Y* and X* permit the posterior mean to be calculated in closed form.

A.5 Penalty Grid Selection

This section identifies starting values for penalty grids and lists procedures used to estimate and tune VARX-L models, including cross-validation and active-set updates.

  • Table 17 reports starting values of the penalty grid for each procedure, with ρ_q denoting the number of variables in group q.
  • Basic VARX-L uses an iterative procedure over k groups and kp + ms terms until a desired threshold is reached.
  • Cross-validation is provided for Basic VARX-L to select tuning parameters over a training range indexed by t.
  • The lag-group procedure applies proximal updates to grouped coefficients and can use an active-set strategy.
  • The estimation routines include threshold and block updates, while sparse lag-group and endogenous-first variants use dedicated inner-loop or proximal procedures.
Loading 1508.07497v4…