PaperlayerView exact revision on arXiv

Source-linked AI summary

Detecting causal associations in large nonlinear time series datasets

Jakob Runge, Peer Nowack, Marlene Kretschmer, Seth Flaxman, Dino Sejdinovic

arXiv:1702.07007v2stat.MEphysics.ao-phstat.AP

TL;DR

Observational time-series causal discovery is difficult because experiments are often unavailable and datasets combine high dimensionality, nonlinearities, and limited samples. The paper introduces PCMCI, which selects relevant conditions and combines that causal-discovery strategy with linear or nonlinear conditional independence tests. Across climate and synthetic datasets, the method reports higher detection power with controlled false positives, enabling multivariate causal-network reconstruction.

  • Problem

    Causal relationships must be inferred from observational time series despite high dimensionality, nonlinear dependencies, limited samples, and often impractical interventions.

  • Method

    PCMCI removes irrelevant conditioning variables through condition selection and applies conditional independence tests designed for highly interdependent time series, including linear and nonlinear options.

  • Results

    The method yields higher detection power while controlling false positives around the chosen 5% significance level across small- to large-scale analyses.

  • Takeaways & Limitations

    PCMCI enables multivariate causal analyses that can more credibly reconstruct time-lagged causal networks from observational time series.

  • Takeaways & Limitations

    The method focuses on time-lagged dependencies and does not address causal directionality for contemporaneous links.

Abstract

from arXiv · show

Identifying causal relationships from observational time series data is a key problem in disciplines such as climate science or neuroscience, where experiments are often not possible. Data-driven causal inference is challenging since datasets are often high-dimensional and nonlinear with limited sample sizes. Here we introduce a novel method that flexibly combines linear or nonlinear conditional independence tests with a causal discovery algorithm that allows to reconstruct causal networks from large-scale time series datasets. We validate the method on a well-established climatic teleconnection connecting the tropical Pacific with extra-tropical temperatures and using large-scale synthetic datasets mimicking the typical properties of real data. The experiments demonstrate that our method outperforms alternative techniques in detection power from small to large-scale datasets and opens up entirely new possibilities to discover causal networks from time series across a range of research fields.

Introduction

Causal discovery from observational time series is needed to reconstruct dependencies in complex systems where interventions are impractical. High dimensionality, nonlinearities, limited samples, and spurious correlations motivate a method that improves reliable multivariate network reconstruction.

  • Motivation: Observational causal networks are important for studying climate and neuroscience systems when interventions are expensive, unethical, or impractical.Climate-model mechanisms may be limited by imperfect realism, increasing the value of observational time series.
  • Challenge: High-dimensional climate hypotheses require testing many possible drivers beyond an initially correlated pair of processes.The interconnected climate system can quickly turn a two-variable question into a high-dimensional causal discovery problem.
  • Challenge: Causal discovery must reconstruct time-lagged links while distinguishing direct effects from autocorrelation, indirect links, and common drivers.Pairwise correlations can create spurious associations, producing denser networks than the underlying causal structure.
  • Existing limitations: Standard high-dimensional Granger-style regression can have very low detection power when datasets contain hundreds of variables but only a few hundred samples.This curse of dimensionality has contributed to applications being limited largely to bivariate analyses.
  • Contribution: The proposed method targets tens to hundreds of variables with linear or nonlinear time-delayed dependencies and sample sizes of a few hundreds or more.The authors report reliable false-positive control and higher detection power without significantly increasing computational demand.

Causal discovery

The climate example illustrates how irrelevant conditioning variables and dimensionality reduce causal detection power. PCMCI addresses this by selecting relevant conditions before applying conditional independence tests, including tests suited to nonlinear dependencies.

  • Motivation: Conditioning on irrelevant variables and high dimensionality are two main causes of low detection power in autoregressive causal analyses.The motivating example uses a tropical Pacific–British Columbia teleconnection to demonstrate this dilemma.
  • Climate example: Lagged correlation found Nino →BCT at lag two months with effect size ≈0.3 and p < 10^-4, but also suggested the reverse direction.Lagged correlation therefore cannot establish causal directionality or the correct coupling lag.
  • Climate example: FullCI reduced the Nino →BCT effect size to 0.1 at lag 2, with p = 0.037, after conditioning on the past.Including an additional variable can further reduce power when it explains away part of the partial correlation.
  • Climate example: 53% detection at a 5% significance level was obtained for the true Nino_t−2 →BCT_t link when the effect size was 0.09.Adding more variables reduced detection power to 40% despite the same causal-link coefficient, making the true link likely to be overlooked.
  • PCMCI approach: PCMCI combines PC1 condition selection with a conditional independence test to avoid conditioning on irrelevant variables in time series graphs.Its conditioning strategy accounts for autocorrelation and aims to improve power while controlling false positives.
  • Conditional independence tests: GPDC uses Gaussian-process regression and distance correlation for additive nonlinear dependencies, whereas CMI uses a non-parametric conditional-mutual-information estimator.The greater generality of GPDC or CMI can reduce power for linear relationships with small samples.

Results

Across linear and nonlinear synthetic experiments and a climate teleconnection, PCMCI maintained high detection power in high-dimensional, autocorrelated time series while generally controlling false positives. Its condition-selection step reduced conditioning dimensionality, and its MCI statistic provided effect sizes useful for ranking causal links.

  • Climate teleconnection: On the climate example, PCMCI detected the Nino−2 → BCT link with effect size ≈0.10 and detection power above 80% even in the high-dimensional case.It also estimated the causal effect strength ≈0.14 for autoregressive pairs with similar power despite differing autocorrelations.
  • Linear relationships: PCMCI showed nearly equal power for weakly and strongly autocorrelated links up to N = 20, while PC1 detected more than 80% of true parents through N = 100.At N = 100, over 80% of selected conditions were false positives, but the condition set remained only a small fraction of FullCI’s conditions.
  • Linear relationships: PCMCI maintained high detection power when the dimensionality Nτmax exceeded the sample size, outperforming FullCI and PC across tested network sizes.For N = 10, 99% of links had detection power greater than 70%, whereas Lasso consistently overlooked a substantial fraction of links.
  • Nonlinear relationships: For nonlinear models, PCMCI achieved the highest GPDC-based power with only slight declines as networks grew, while mostly controlling false positives.With CMI, PCMCI also controlled false positives correctly and exceeded PC’s power except for some nonlinear link types.
  • Causal strength: The MCI statistic produced larger effect sizes than FullCI and sometimes correlation, supporting higher detection power and meaningful ranking of causal links.Correlation values for links with equal causal effects ranged from zero to high values, whereas MCI was shown to estimate causal strength.

Discussion and conclusion

Large-scale causal discovery must balance broader multivariate adjustment against reduced detection power and false-positive control. PCMCI addresses this trade-off through condition selection and a conditional independence test designed for highly interdependent time series, while its causal interpretation depends on standard assumptions and is focused on time-lagged dependencies.

  • Adding irrelevant variables increases dimensionality, can reduce effect sizes, and may cause important causal links to be overlooked.
  • PCMCI removes irrelevant conditions and uses a conditional independence test for highly interdependent time series.Condition selection improves power, while MCI controls false positives for highly autocorrelated data.
  • PCMCI accommodates different conditional independence tests, discrete or continuous series, and multivariate variables as graph nodes.This supports causal associations at different aggregation levels and across multiple interdependent layers.
  • The method focuses on time-lagged dependencies, leaving contemporaneous causal directionality to other techniques.
  • A causal interpretation assumes causal sufficiency and the Causal Markov Condition, among other technical assumptions.
  • Multivariate causal analyses can make reconstructed causal networks more credible than predominantly bivariate analyses.The paper connects this possibility to growing data availability across scientific disciplines.

Materials and Methods

The paper describes its main method and supplementary material, with reproducible software and documentation available online.

  • The main text describes the novel method introduced, while further material is provided in the Supplement.
  • Software reproducing the examples and numerical experiment results is available online.
  • The Tigramite repository includes comprehensive documentation of the method.

Funding

The work received support from multiple academic, governmental, regional, and computing-resource funders, with no conflicts of interest declared.

  • J.R. received funding from a postdoctoral award by the James S. McDonnell Foundation.
  • The project was supported by the European Regional Development Fund, the German Federal Ministry of Education and Research, and the Land Brandenburg.
  • The Potsdam Institute for Climate Impact Research provided high-performance computing resources for the project.

S1 Causal discovery

PCMCI reconstructs time-series causal graphs through condition selection followed by momentary conditional independence testing. Its design targets lagged, high-dimensional dependencies while allowing flexible conditional independence tests and parameter choices.

  • Causal discovery method (PCMCI): The time-series graph represents variables at each time point and directed links across time lags, with contemporaneous dependencies left undirected.
  • Causal discovery method (PCMCI): PCMCI estimates time-series graphs with a two-step procedure: condition selection followed by MCI causal discovery testing.
  • MCI causal discovery: MCI tests variable pairs across candidate time delays while conditioning on selected parents and can use different test statistics.The maximum delay, significance threshold, and number of driver conditions are free parameters.
  • Condition-selection: Condition selection iteratively removes variables when conditional independence is accepted for conditioning subsets of increasing cardinality.The fast PC1 variant limits the number of tested combinations per iteration to qmax = 1.
  • Parameter choices: PCMCI with pX = 3 was slightly faster and more powerful than MCIall in the reported experiments.It also reduced the power difference between weakly and strongly autocorrelated links.
  • Alternative methods: The PC method produced mostly over-conservative p-values, while PC1+MCI0 and the residual-based approach failed to control false positives.

S2 Conditional independence tests

The framework combines a causal discovery procedure with conditional independence tests spanning linear, nonlinear residual-based, and fully non-parametric approaches. These tests differ in assumptions, computational properties, and ability to capture smooth or highly non-smooth dependencies.

  • Test framework: PCMCI can be combined with any conditional independence test; the paper implements ParCorr, GPDC, and conditional mutual information tests.The tests estimate different dependence measures with associated test statistics.
  • Partial correlation: Partial correlation uses regression residuals and an analytical null distribution, but applies only to multivariate Gaussian data and linear dependencies.It estimates X and Y conditional on Z through multivariate regression, then tests residual correlation.
  • GPDC: GPDC combines Gaussian-process regression with distance correlation on uniformized residuals to measure linear and nonlinear association.Its underlying assumption is an additive functional dependency.
  • GPDC: Figure 3D shows that GPDC identifies conditional independence in a quadratic relationship that ParCorr does not.The example uses X = Z2 +ηX and Y = −Z2 +ηY with X ⊥⊥Y | Z.
  • Conditional mutual information: CMI is zero if and only if X ⊥⊥Y|Z, and the discrete CMIsymb variant estimates it from symbol frequencies using permutation testing.The continuous estimator uses nearest-neighbor entropy estimation and a local permutation scheme.
  • Conditional mutual information: The non-parametric CMI test is locally data-adaptive, enabling detection of highly non-smooth dependencies that GP regression may miss.Its finite-sample variance and convergence rates are not established, leaving limited theoretical guidance for the estimator.

S3 Properties of PCMCI

PCMCI has polynomial worst-case complexity and theoretical consistency under causal sufficiency, faithfulness, and the causal Markov condition. Its MCI step supports calibrated false-positive control and can yield larger effect sizes and higher power than FullCI, though GPDC may show inflated false positives in high dimensions.

  • Computational complexity: PCMCI has polynomial worst-case complexity in the number of variables and maximum time lag.The stated worst-case complexity is N^3τmax^2 + N^2τmax when α-optimization is excluded.
  • Consistency: Consistency of population PCMCI requires causal sufficiency, faithfulness, the causal Markov condition, and consistent conditional independence tests.Under these assumptions, the estimated graph converges to the true graph in the infinite-sample population setting.
  • Consistency: The consistency result is weaker than uniform consistency because it does not provide an error-probability convergence rate.Uniform consistency would require further assumptions excluding almost-unfaithful dependencies.
  • False-positive control: MCI tests are well calibrated under the iid innovation-term argument, and experiments found expected false-positive rates for strongly autocorrelated and nonlinear dependencies.The approach outperformed pre-whitening and block-shuffling in the reported experiments, whereas GPDC showed inflated false-positive rates in high-dimensional settings.
  • Effect size and power: MCI has an effect size at least as large as FullCI and generally achieves higher power through lower-dimensional conditioning and stronger test statistics.The two effect sizes are equal when the additional FullCI conditioning variables are independent of Y given W.
  • Causal strength: MCI can estimate causal strength from the coupling coefficient and noise variances without depending on parent-induced or autocorrelation-related dependencies in linear additive models.The paper reports that this makes causal signals easier to detect against confounding-driver or autocorrelation noise.

S4 Numerical experiments

The numerical experiments use synthetic time-series networks whose known causal structure reproduces high dimensionality, nonlinearity, strong autocorrelation, and time-lagged dependencies. Performance is assessed across network configurations using false-positive and true-positive rates, runtimes, and autocorrelation regimes.

  • Experimental design: The synthetic model reproduces four challenges of complex-system time series: high dimensionality, nonlinearity, strong autocorrelation, and time-lagged causal dependencies.The underlying relationships are known, allowing causal discovery methods to be evaluated against the generated ground truth.
  • Experimental design: Twenty ensemble members are generated for each combination of variable count N, link count L, and coupling strength c.Links are randomly selected between distinct variables, with coupling signs sampled as positive or negative with equal probability.
  • Experimental design: The experiments vary linear and nonlinear link functions, including 50% linear links and two specified nonlinear functions in nonlinear-model experiments.Autocorrelation coefficients are sampled from different sets for weakly and more strongly autocorrelated ensemble members.
  • Experimental design: Stationarity is enforced by retaining models whose corresponding linearized vector autoregressive systems pass a unit-root test.The network topologies generally use L = N links, with additional experiments involving L = 2N and bivariate settings.
  • Evaluation metrics: False-positive and true-positive rates are estimated from 100 time-series realizations per model, with figures separating weakly and strongly autocorrelated links.The reported sampling error for an estimated FPR of 0.05 or TPR of 0.95 is roughly 0.02.
  • Runtime evaluation: Runtime comparisons are reported across network sizes and time-series lengths, with averages and standard deviations shown across model setups.Runtime estimates were obtained on Intel Xeon E5-2667 v3 8C processors, so they depend on implementation details.

S5 Tigramite software package

PCMCI is implemented in the Tigramite Python package, which provides causal-inference components and visualization tools for time series analysis.

  • Software package: Tigramite implements PCMCI and multiple linear and nonlinear conditional independence tests for time-series causal inference.The package also includes plotting functions for time series, lag functions, and causal graphs.

S6 Algorithms

The supplementary algorithms define PCMCI as a condition-selection stage followed by MCI testing, with optional multiple-testing adjustment, alongside an adaptive-lasso inference procedure.

  • Condition selection: The condition-selection algorithm initializes candidate parents, iteratively tests conditional independence, removes nonsignificant parents, and sorts the remaining parents by test-statistic values.The algorithm uses a maximum condition dimension and a limit on tested conditioning-set combinations.
  • MCI causal discovery: PCMCI applies the MCI causal-discovery step to estimated parent sets across variables and time lags up to τmax.Contemporaneous links at τ = 0 are left undirected in the stated algorithm.
  • Multiple-testing adjustment: The MCI procedure returns p-values or FDR-adjusted q-values together with test-statistic values.False-discovery-rate adjustment is optional and uses significance level α.
  • Adaptive lasso: Adaptive lasso standardizes predictors and responses, iteratively rescales features, solves a cross-validated lasso problem, and fits OLS on the active set for p-values.The implementation uses TimeSeriesSplit with five splits for choosing λn and permits up to kmax iterations.

S7 Supplementary Tables

The supplementary tables define the conditional independence tests, synthetic-model configurations, and comparison methods used throughout the numerical experiments.

  • Conditional independence tests: Table S2 summarizes conditional independence tests for X ⊥⊥Y | Z, assuming standardized, continuously valued data.The table also specifies Gaussian-process implementation choices and defines D_Z as the cardinality of Z.
  • Model configurations: Table S3 lists model configurations for the experiments, with 100 time-series realizations generated per configuration to evaluate false and true positives.The configurations use the model given in Eq. (S60) and specify coupling functions such as f^(1)(x) = x.
  • Compared methods: The supplementary experiments compare FullCI, Lasso, PC, PC1+MCI3, PC1+MCIall, ParCorr, dCor, GPDC, MI, and CMI.The relevant method subsets depend on the conditional independence test used.

S8 Supplementary Figures

The supplementary figures evaluate causal-discovery methods across linear and nonlinear models, varying dimensionality, link density, sample size, thresholds, and observational noise. They also document method-specific power, false-positive control, and runtime behavior.

  • Experimental settings: The experiments vary the numbers of variables, links, and time-series length across linear and nonlinear models.Linear settings include L = N or L = 2N; nonlinear settings include L = N.
  • Condition selection: The supplementary figures examine how PC threshold α affects ParCorr, GPDC, and CMI condition selection, with AIC-based α optimization used for most ParCorr experiments.The full model setup is described in Sect. S4 and Tab. S3.
  • Linear models: For linear ParCorr experiments, PCMCI outperforms FullCI, Lasso, and PC, although Lasso misses some links even at the largest sample size.Power increases with larger samples for all methods, while runtime does not linearly increase with sample size.
  • Nonlinear models: For nonlinear GPDC experiments, power increases with sample size, FullCI better controls false positives at larger samples, and runtime scales as ∼T 3.The runtime scaling makes GPDC less suitable for large sample sizes.
  • Nonlinear models: CMI power increases with sample size, while its runtime does not increase quadratically like the individual CMI tests.The experiments use fixed N = 5 and L = N for these nonlinear models.
  • Observational noise: Observational noise up to 25% of the dynamical-noise standard deviation has minor effects across methods, whereas comparable noise levels cause stronger degradation and poorer false-positive control.The noise experiments use T = 150, N = 20, and L = N.
Loading 1702.07007v2…