Source-linked AI summary
Sparse Causal Discovery in Multivariate Time Series
Stefan Haufe, Guido Nolte, Klaus-Robert Mueller, Nicole Kraemer
TL;DR
The paper addresses how to recover causal interactions in multivariate time series when the underlying graph is sparse. It proposes joint sparsity across the lagged coefficients for each series pair, and reports that Group Lasso and Ridge Regression with multiple testing outperform competitors across simulated scenarios, though not always significantly.
Problem
Sparse causal graphs are common assumptions in applications, but standard OLS and Ridge VAR fitting produce dense coefficients, complicating sparse causal recovery.
Source: §1 · p. 2Method
The paper proposes Group Lasso to jointly sparsify all lag coefficients for each series pair, and also considers Ridge Regression followed by multiple significance testing.
Source: §3.2 · pp. 4–5, §3.1 · p. 4Results
Group Lasso and Ridge Regression outperform their competitors in all simulated scenarios, though not always significantly.
Source: §4.2 · p. 7Takeaways & Limitations
The two approaches were shown to outperform standard methods in simulated scenarios, and Group Lasso is expected to handle larger problems if the sparsity assumption is correct.
Source: §5 · p. 8The approach assumes linear underlying dynamics, and the cited Granger-causality treatment fails when common factors are not observable.
Source: §1 · p. 2
Abstract
from arXiv · showhide
Our goal is to estimate causal interactions in multivariate time series. Using vector autoregressive (VAR) models, these can be defined based on non-vanishing coefficients belonging to respective time-lagged instances. As in most cases a parsimonious causality structure is assumed, a promising approach to causal discovery consists in fitting VAR models with an additional sparsity-promoting regularization. Along this line we here propose that sparsity should be enforced for the subgroups of coefficients that belong to each pair of time series, as the absence of a causal relation requires the coefficients for all time-lags to become jointly zero. Such behavior can be achieved by means of l1-l2-norm regularized regression, for which an efficient active set solver has been proposed recently. Our method is shown to outperform standard methods in recovering simulated causality graphs. The results are on par with a second novel approach which uses multiple statistical testing.
1 Introduction
The paper frames causal discovery as estimating interactions among multivariate time series, using VAR coefficients at time lags to represent causal influence. Because causal graphs are often assumed sparse, it proposes jointly sparsifying all lag coefficients for each pair of series and evaluates the approach against standard methods and multiple testing.
1 Introduction
- A causal influence is represented by a nonzero VAR coefficient at at least one time lag, under the paper’s linear-dynamics assumption.The introduction first describes Granger causality through improved prediction from a series’ past values, compared with using only the target series’ past.Source: §1 · pp. 1–2
- Sparse causal graphs motivate estimating VAR models with regularization, since OLS and Ridge Regression are known to produce dense coefficients.The paper also considers Ridge Regression followed by multiple statistical testing.Source: §1 · p. 2
- The proposed sparse approach jointly zeros all autoregressive coefficients for a series pair when no causal relation is present.The paper evaluates this approach and Ridge Regression with multiple testing against standard methods in simulations.Source: §1 · p. 2
2 Background
The background defines VAR-based causal influence through nonzero lagged coefficients and surveys sparse estimation and testing approaches. It distinguishes coefficient sparsity methods from strategies that test for significant dependencies.
2 Background
- A linear VAR process of order P models each time point from P past values, and causal influence is indicated by a nonzero coefficient at at least one lag.The coefficient matrices are estimated from a sample of the multivariate time series to conduct causal inference.Source: §2 · pp. 2–3
2.1 Sparsity
- Lasso-based VAR estimation has been proposed, but higher-order extensions may instead estimate partial correlations between time-lagged series.Valdes-Sosa et al. (2005) consider order 1; Arnold et al. (2007) extend to higher orders using the partial-correlation transformation.Source: §2.1 · p. 3
2.2 Testing
- Testing strategies detect significant partial correlations after estimating them, with shrinkage proposed for high-dimensional data.The cited partial-correlation testing approach was tested for P = 1 and is described as straightforward to extend to higher-order VARs.Source: §2.2 · p. 3
3 Our Approach
The approach presents Ridge regression with multiple testing and direct Group Lasso estimation as strategies for sparse VAR causal discovery. Group Lasso selects coefficient groups jointly across time lags.
3.1 Ridge Regression and Multiple Testing
- Ridge regression estimates VAR coefficients with regularized least squares, but its coefficients are generally nonzero and therefore yield a fully connected graph.The approach sparsifies the estimates through statistical testing based on approximate coefficient distributions.Source: §3.1 · pp. 3–4
- Under the null hypothesis, the Ridge coefficients are modeled as jointly normal, enabling adjusted p-values for simultaneous tests of individual coefficients.Hypotheses are rejected when adjusted p-values fall below the predefined significance level.Source: §3.1 · p. 4
3.2 Group Lasso
- Group Lasso imposes joint sparsity on coefficients for each time-series pair, so a whole group can be set to zero across lags.Its ℓ1,2 norm is an ℓ1 norm over a vector of ℓ2 norms.Source: §3.2 · pp. 4–5
- An active-set solver reduces memory use and computation time by applying the SOCP solver to a much smaller active set of coefficient groups.The optimization problem is convex but non-differentiable.Source: §3.2 · p. 5
4 Simulations
The simulations compare four methods for recovering causal structure across noise conditions and model-order settings. Group Lasso and Ridge Regression outperform their competitors across scenarios, though not always significantly.
4 Simulations
- The experiments compare Group Lasso, Lasso, Ridge Regression with multiple testing, and complete Granger Causality, with and without the true model order.When the order is unknown, the reconstruction uses P = 10; the model order could also be estimated, for example by cross-validation.Source: §4 · p. 5
4.1 Setup
- Each simulated series has M = 7 variables and T = 1000 observations, with 10 causal interactions in a stable VAR process of order P = 5.Ten instances are generated for each of three conditions: no added noise, uncorrelated white noise, and mixed noise from an instantaneous mixture of AR processes.Source: §4.1 · pp. 5–6
- Performance is assessed with ROC curves and AUC, averaged over 10 problem instances, with standard errors computed for AUC.The ROC curves allow assessment across different operating regimes, including regimes with very few false positives.Source: §4.1 · p. 6
- The methods vary their decision thresholds or regularization parameters to produce solutions ranging from completely sparse to completely dense.Granger Causality uses a thresholded score; Ridge Regression varies its significance level after cross-validation, while Lasso and Group Lasso vary their regularization constants.Source: §4.1 · p. 6
4.2 Results and Discussion
- Across all scenarios, Group Lasso and Ridge Regression outperform competitors, though not always significantly.Ridge Regression is slightly better in the noiseless condition, Group Lasso has a visible but insignificant advantage with white noise, and they are on par under mixed noise.Source: §4.2 · p. 7, Table 1 · p. 7
- Group Lasso’s ROC curve exceeds Lasso’s, while knowing the true model order provides hardly any significant advantage in these simulations.The authors interpret Lasso’s lower ROC curve as a tendency to be too dense.Source: §4.2 · p. 7
5 Conclusion
The paper presents Group Lasso for causal discovery in multivariate time series and discusses Ridge Regression with multiple testing as an alternative. Both approaches outperform standard methods in simulated scenarios; application to real-world problems remains future work.
5 Conclusion
- The paper presents a Group Lasso approach and a novel VAR-modeling alternative: Ridge Regression followed by multiple testing.The authors state that both approaches outperform standard methods in simulated scenarios.Source: §5 · pp. 7–8
- The authors identify real-world application as future work and expect the Group Lasso approach to scale to larger problems if the sparsity assumption holds.Their proposed route splits the task into M independent subproblems and uses an active-set solver with strong regularization.Source: §5 · p. 8