Source-linked AI summary
Amortized Causal Discovery: Learning to Infer Causal Graphs from Time-Series Data
Sindy Löwe, David Madras, Richard Zemel, Max Welling
TL;DR
Existing time-series causal discovery methods often fit separate models for samples with different causal graphs, although those samples may share causal dynamics. The paper proposes Amortized Causal Discovery, a variational framework that predicts graph relations with an amortized encoder and models their dynamics with a decoder. Experiments report significant performance gains over prior work, including noisy and hidden-confounding settings, while leaving assumptions and realism of simulated data as limitations.
Problem
Existing methods refit models for samples with different causal graphs, so they cannot exploit potentially shared dynamics across samples.
Method
Amortized Causal Discovery uses a single amortized encoder to predict graph edges across samples and a decoder to model time-series dynamics under those edges.
Results
ACD achieves significant performance gains over existing methods in fully observed settings and with noise and hidden confounding.
Takeaways & Limitations
The framework can infer causal relations in previously unseen samples without refitting and can pool statistical strength across samples.
Takeaways & Limitations
The experiments use simulated data, which the authors note are not particularly realistic because real-world data are more complex and potentially misspecified.
Abstract
from arXiv · showhide
On time-series data, most causal discovery methods fit a new model whenever they encounter samples from a new underlying causal graph. However, these samples often share relevant information which is lost when following this approach. Specifically, different samples may share the dynamics which describe the effects of their causal relations. We propose Amortized Causal Discovery, a novel framework that leverages such shared dynamics to learn to infer causal relations from time-series data. This enables us to train a single, amortized model that infers causal relations across samples with different underlying causal graphs, and thus leverages the shared dynamics information. We demonstrate experimentally that this approach, implemented as a variational model, leads to significant improvements in causal discovery performance, and show how it can be extended to perform well under added noise and hidden confounding.
1. Introduction
Causal discovery methods often refit models when samples have different causal graphs, despite shared dynamics across samples. Amortized Causal Discovery instead learns shared dynamics while predicting graph-specific causal relations, improving performance and supporting unseen samples.
- Motivation: Causal discovery methods commonly assume one underlying graph and refit a model when that assumption fails.Samples can differ in graph structure while sharing dynamics that describe causal effects.
- Amortized Causal Discovery: Amortized Causal Discovery trains one model to infer causal relations across samples with different graphs but shared dynamics.The framework is designed to pool information across samples rather than discard shared dynamics.
- Amortized Causal Discovery: The framework separates causal-relation prediction from dynamics modeling using an amortized encoder and a decoder.The encoder predicts graph edges, while the decoder models system dynamics under those relations.
- Model and evaluation: ACD can infer causal relations in previously unseen samples without refitting and can improve robustness under hidden confounding.The hidden-confounding extension models unobserved variables with the amortized encoder.
- Model and evaluation: A variational ACD model applies to multivariate, nonlinear time-series data.The paper evaluates this model across causal discovery datasets in fully observed, noisy, and hidden-confounding settings.
2. Background: Granger Causality
Granger causality infers time-series causal relations from temporal precedence by testing whether one series improves prediction of another. Existing approaches typically fit and optimize a separate predictive model for each sample before extracting a graph.
- Definition: Granger causality treats X as causing Y when past X improves prediction of future Y.The paper uses a nonlinear definition based on dependence of the predictive function on past variables.
- Definition: Under observed variables and no instantaneous connections, Granger relations correspond to causal relations in the underlying DAG.This equivalence depends on both assumptions stated in the background.
- Standard workflow: Existing Granger discovery methods define a predictive function and fit its parameters by minimizing a loss on a test sequence.The framework covers linear, autoregressive, and deep-learning approaches.
- Standard workflow: They estimate the graph by applying a fixed function to the fitted parameters, such as thresholding learned weights.For some methods, zero outgoing weights indicate that one series does not Granger-cause another.
- Limitation: When samples have different graphs, separate parameter optimization prevents these methods from using information shared across samples.This limitation motivates amortizing the inference process.
3. Amortized Causal Discovery
Amortized Causal Discovery separates sample-specific causal graphs from dynamics shared across samples, enabling graph inference across differing structures without refitting for each sample. Its encoder–decoder framework can generalize to new samples, support test-time adaptation, and extend to probabilistic modeling and hidden confounding.
- Assumptions: The framework assumes causes temporally precede effects and that causal relations and structural equations are invariant to time.Temporal precedence supports identifiability without hidden confounders, while time invariance simplifies the dynamics model.
- Framework: ACD learns sample-specific causal graphs while modeling shared dynamics across samples with different underlying graphs.The framework uses an encoder for graph inference and a decoder for the shared dynamics.
- Model: The causal discovery encoder maps an input sample to a graph, while the decoder models its dynamics and training combines a sample-wise loss with graph regularization.The regularizer can penalize graphs with more edges, encouraging sparsity.
- Inference: After training, ACD can infer a causal graph for a new test sequence directly or refine graph predictions by minimizing the learned decoding loss through test-time adaptation.Direct prediction uses the amortized encoder; test-time adaptation optimizes an instantiated graph estimate.
- Probabilistic implementation: The probabilistic implementation uses qφ(z|x) to predict graph edges and pθ(x|z) to model time-series dynamics, with negative log-likelihood and KL-divergence terms.The resulting objective is a variational lower bound, and the encoder uses a graph neural network.
- Hidden confounding: ACD is designed to handle hidden confounding by extending the encoder to predict a hidden variable and feeding that prediction into the decoder.The paper considers both a global confounder affecting causal-relation strength and a time-varying hidden variable with observed-variable-like behavior.
4. Related Work
Prior causal-discovery methods cover constraint-based, mixed-effects, amortized, temporal, and relation-inference approaches, but ACD explicitly targets causal graphical structure while sharing dynamics across differing graphs.
- Constraint-based methods infer a single graph or equivalence class across samples and do not model time-series dynamics.
- Several related approaches use mixed-effects models, amortized deep learning, graph neural networks, attention, or meta-learning for relational inference.
- Relation-inference approaches are not explicitly designed to infer causal graphical structure.
- Its probabilistic implementation extends NRI with test-time adaptation, hidden-confounder algorithms, and a proof connecting zero-edge functions to Granger causality.
5. Experiments
Experiments evaluate ACD on fully observed physics and fMRI data, then test robustness to observation noise and hidden variables. ACD performs strongly across these settings, including low-data, noisy, and confounded cases.
- 5. Experiments: ACD is evaluated with AUROC on Kuramoto, Particles, and Netsim datasets, including fully observed, noisy, and hidden-variable settings.
- Fully Observed Amortized Causal Discovery: ACD greatly outperforms compared Granger-causal-discovery approaches on Kuramoto without fitting to test samples.
- Fully Observed Amortized Causal Discovery: With fewer than 50 training samples, test-time adaptation outperforms previous approaches on Kuramoto, while performance improves with more training data.
- Fully Observed Amortized Causal Discovery: 0.999 AUROC is achieved on Particles, with encoder inference best in high-data settings and test-time adaptation improving low-data performance.
- Fully Observed Amortized Causal Discovery: ACD remains comparable to single-graph methods on Netsim, where all samples share one underlying connectivity graph.
- Noisy Data: ACD is robust to a certain amount of observational noise on both Particles and Kuramoto.
- Latent Temperature: Under hidden temperature, the latent-variable model outperforms mean-temperature and fixed-temperature baselines across all tested temperatures.
- Latent Temperature: At α = 2, temperature prediction reaches 0.888 R2, 0.966 AUROC, and 0.644 accuracy.
6. Conclusion
The paper introduces ACD for inferring causal relations across samples with different graphs but shared dynamics, reporting gains over prior methods in observed, noisy, and confounded settings. It also identifies untestable shared-dynamics and graph-edge assumptions, simulated-data realism limits, and the need for theoretical guarantees.
- The probabilistic implementation reports significant performance gains over existing methods in fully observed settings, with noise, and under hidden confounding.
- The framework’s assumptions about shared dynamics and graph edges are not verifiable in practice.
- All experiments use simulated data, which the authors describe as not particularly realistic because real-world data is more complex and potentially misspecified.
- The contribution is primarily empirical, leaving conditions for guaranteed identification of correct causal structures as future work.
B.1.1. DATASETS
The experiments use two physics simulations and simulated fMRI data, with differing graph structures and sample configurations. The Netsim setup is included mainly to test flexibility in a classical single-graph setting rather than generalization.
- The physics simulations model Kuramoto oscillators and spring-connected particles, using asymmetric connectivity to represent causal rather than correlational relations.
- The physics datasets contain 50,000 training and 10,000 validation samples, with 5 time-series and longer test sequences for future prediction evaluation.
- Netsim simulates directed connections among brain regions using BOLD imaging data.
- The selected Netsim data contain 50 subjects with one shared graph, 15 brain regions, and sequences of length T = 200.
- Because Netsim has only 50 samples, the same points are used for training, validation, and testing rather than applying a standard split.
- Netsim is intended to show reasonable performance in the classical single-graph setting, not to demonstrate generalization ability.
B.1.2. ARCHITECTURE AND HYPERPARAMETERS
The implementation combines amortized graph inference with neural dynamics modeling, using dataset-specific encoders, multi-step decoder training, and several comparison baselines. The full framework is broader than the particular Markov-order implementation used here.
- Architecture: The amortized encoder uses a graph neural network with two edge-propagation steps along the causal graph.
- Architecture: MLP encoders are used for particles and Netsim, while Kuramoto uses a 1D CNN with attentive pooling.
- Architecture: Test-time adaptation removes the encoder and optimizes a non-amortized variational distribution over graphs.
- Architecture: The decoder uses one edge-propagation step and predicts multiple future steps by feeding predicted means back as inputs for k = 10 steps.
- Architecture: The decoder masks corresponding edges, making edge-type ordering non-arbitrary in this implementation.
- Scope: Although the implementation assumes a full-time graph of Markov order 1, the general ACD framework could use recurrence to remove that assumption.
B.2. Additional Experimental Result - Training Curves
Training curves on 100 particle samples show encoder overfitting in AUROC, whereas decoder NLL exhibits less overfitting.
- The encoder overfits the training samples, as indicated by its AUROC curve.
- The decoder shows less overfitting, as indicated by its negative log-likelihood performance.
C.1. Temperature Experiments
The temperature experiments model an unobserved temperature with a bounded uniform posterior and evaluate both future prediction and temperature recovery. ACD with Latent predicts the unobserved temperature well, particularly in ordering, while its future-prediction gains are slight and noisier.
- The experiment uses a CNN encoder and a simulation decoder matching the true generative ODE process.
- The latent temperature is represented with a uniform posterior whose support must remain within the prior interval [0, 4α].The prior is uniform over [0, 4α].
- The posterior parameters define a bounded interval on [0, 1], which is then scaled to the desired temperature range [0, 4α].The model samples the temperature from the scaled uniform distribution and uses it in the decoder and KL term.
- ACD with Latent shows a slight future-prediction improvement over baselines across α values, although MSE is a noisier indicator.
- Its temperature predictions are conservative, tending more often toward temperatures that are too low than too high.Higher temperatures incur larger MSEs because the systems are more chaotic and less predictable.
- ACD with Latent predicts the unobserved temperature quite well, especially its ordering as measured by correlation and AUROC.
C.2. Unobserved Time-series
The unobserved time-series experiments test whether ACD with a latent variable can reconstruct trajectories unavailable during observation. ACD with Latent outperforms mean imputation and approaches the fully observed baseline more closely than the None and Mean baselines, with stronger benefits when more observed series are influenced.
- A two-layer bidirectional LSTM with latent dimension 256 models the unobserved time-series.
- ACD with Latent predicts the unobserved trajectory more accurately than Mean imputation.
- ACD with Latent recovers fully observed baseline performance better than the None and Mean imputation baselines, though it remains worse than the Supervised baseline.
- The benefit of adding a latent variable strengthens as more observed particles are influenced by the unobserved particle.
- The follow-up experiment assumes that the unobserved series influences all observed series but is not influenced by them, making trajectory prediction easier.
- Predicted trajectories are initialized from the first half of the ground-truth sequence and then compared with the remaining trajectories.