PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Temporally Causal Latent Processes from General Temporal Data

Weiran Yao, Yuewen Sun, Alex Ho, Changyin Sun, Kun Zhang

arXiv:2110.05428v4stat.MLcs.AIcs.LG

TL;DR

Recovering causal latent variables from observations is difficult because they are not generally uniquely identifiable, and temporal methods have often assumed independent sources. This paper establishes conditions for identifying temporally causal latent processes from nonlinear mixtures and introduces LEAP to enforce them. Experiments report reliable identification across dependency structures and better performance than baselines that do not use history or nonstationarity information.

  • Problem

    Causally related latent variables are not generally uniquely recoverable from observations, and temporal nonlinear ICA work has not established how temporal structure identifies causally related latent factors and their relations.

    Source: §1 · pp. 1–2

  • Method

    The paper establishes identifiability conditions for nonparametric nonstationary and parametric latent processes, then uses LEAP, a VAE extension with a constrained causal process prior, to enforce them.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    Experiments report reliable recovery under different dependency structures and that LEAP considerably outperforms methods that do not leverage history or nonstationarity information.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors report recovery of time-delayed latent processes from nonlinear mixtures without sparsity or minimality assumptions, describing this as one of the first such works.

    Source: §1 · p. 2

    The theory assumes no instantaneous causal influence between latent processes and causal influences that do not change across regimes.

    Source: §5 · p. 9

Abstract

from arXiv · show

Our goal is to recover time-delayed latent causal variables and identify their relations from measured temporal data. Estimating causally-related latent variables from observations is particularly challenging as the latent variables are not uniquely recoverable in the most general case. In this work, we consider both a nonparametric, nonstationary setting and a parametric setting for the latent processes and propose two provable conditions under which temporally causal latent processes can be identified from their nonlinear mixtures. We propose LEAP, a theoretically-grounded framework that extends Variational AutoEncoders (VAEs) by enforcing our conditions through proper constraints in causal process prior. Experimental results on various datasets demonstrate that temporally causal latent processes are reliably identified from observed variables under different dependency structures and that our approach considerably outperforms baselines that do not properly leverage history or nonstationarity information. This demonstrates that using temporal information to learn latent processes from their invertible nonlinear mixtures in an unsupervised manner, for which we believe our work is one of the first, seems promising even without sparsity or minimality assumptions.

1 INTRODUCTION AND RELATED WORK

The section motivates identifying time-delayed causal latent processes from nonlinear observations, a problem not resolved by prior methods focused on observed variables, independent sources, or restricted latent-graph settings. It presents identifiability conditions for nonparametric nonstationary and parametric processes and introduces LEAP to enforce them.

1 INTRODUCTION AND RELATED WORK

  • Prior latent-graph methods were limited to linear relations, sparsity or minimality assumptions, or many measured children, while quick-adaptation approaches lacked established identifiability results.These constraints motivate a method for recovering temporally causal latent variables from observations.Source: §1 · p. 1
  • Temporal nonlinear ICA results used auxiliary information to identify independent sources, but that assumption may distort recovery when latent factors have causal, especially nonlinear, relations.The section identifies learning temporally related latent factors and their causal structure from temporal observations as an open question.Source: §1 · pp. 1–2
  • The paper establishes identifiability of latent factors and causal influences in both nonparametric nonstationary and parametric settings from invertible nonlinear mixtures.The parametric setting uses linear time-delayed influences; the nonparametric setting uses unknown functions of delayed parents and noise.Source: §1 · p. 2
  • LEAP extends VAEs with a learned causal process prior that enforces the Independent Noise condition and models possible nonstationarity through flow-based estimators.The framework is designed to enforce the paper’s assumed conditions through training constraints.Source: §1 · p. 2
  • The work extends identifiability theory to temporally causally related latent processes and reports recovery from nonlinear mixtures without sparsity or minimality assumptions.It leverages nonstationarity or functional and distributional constraints on temporal relations; the authors describe it as one of the first such works.Source: §1 · p. 2

2 IDENTIFIABILITY THEORY

The section defines identifiability in representation-function space and presents two conditions that ensure recovery of temporally causal latent processes from nonlinear mixtures. Under either condition, the processes are identifiable up to componentwise transformations.

2.1 IDENTIFIABILITY PROPERTY

  • Identifiability is defined in representation-function space: observationally equivalent models must recover latent variables up to permutation and componentwise invertible transformation.Observational equivalence means the learned and true models have matching joint distributions of observations.Source: §2.1 · p. 3
  • Conditional independence relations are also identifiable once the latent processes are identified up to componentwise transformations.The section states that these relations characterize time-delayed causality in a time-delayed causally sufficient system.Source: §2.1 · p. 3

2.2 OUR PROPOSED CONDITIONS

  • Nonstationary noise with independent noise terms and sufficient variability across regimes ensures componentwise identifiability for nonparametric processes.The theorem assumes third-order differentiable transitions and an injective mixing function differentiable almost everywhere.Source: §2.2 · p. 3
  • For parametric processes, generalized Laplacian process noise and a full-rank state transition matrix at some lag ensure componentwise identifiability.The theorem assumes linear additive transitions and an injective mixing function differentiable almost everywhere.Source: §2.2 · pp. 3–4

3 LEAP: LATENT TEMPORALLY CAUSAL PROCESSES ESTIMATION

LEAP extends VAEs with a causal-process prior that enforces the paper’s identifiability conditions. It models temporal transitions and noise, infers latent states from full sequences, and uses separate reconstruction designs across data types.

3 LEAP: LATENT TEMPORALLY CAUSAL PROCESSES ESTIMATION

  • LEAP extends VAEs by enforcing the identifiability conditions as constraints for learning latent causal processes.Its architecture combines a learned causal-process prior with an inference network and encoder-decoder.Source: §3 · p. 4

3.1 CAUSAL PROCESS PRIOR NETWORK

  • The causal-process prior enforces independent noise by reparameterizing transition priors into factorized noise distributions.For nonparametric transitions, learned inverse functions map latent states and their history to noise terms; parametric transitions use lag-specific matrices.Source: §3.1 · p. 4, §3.1.1 · pp. 4–5

3.1.1 TRANSITION PRIOR MODELING

  • Nonparametric inverse transition functions produce noise estimates, and a low-triangular-Jacobian transformation supports evaluating the joint latent distribution.The change-of-variables formula and independent-noise condition allow factorized noise distributions to evaluate that distribution.Source: §3.1.1 · pp. 4–5

3.1.2 NONSTATIONARY NOISE ESTIMATION

  • Flow-based estimators fit noise distributions separately by component and regime, while a discriminator encourages independence across noise terms.The discriminator distinguishes estimated noises from batch-permuted noises, and its density-ratio estimate of total correlation is added to the ELBO.Source: §3.1.2 · p. 5

3.1.3 STRUCTURE ESTIMATION

  • Sparsity regularization and pruning are used to visualize causal relations, but the identifiability results do not rely on sparse relations.For nonlinear relations, pruning can produce sparse relations even when the true relations may not follow the causal additive assumption.Source: §3.1.3 · pp. 5–6

3.2 INFERENCE NETWORK

  • A bidirectional GRU approximates the posterior over latent sequences with an isotropic Gaussian, and the KL term is estimated by sampling.The prior distribution is learned by the causal-process network rather than specified in advance.Source: §3.2 · p. 6

3.3 ENCODER AND DECODER

  • The decoder likelihood reconstructs observations from inferred latent states, with MLP or CNN architectures and losses selected by data type.For videos with multiple objects, separate CNNs extract visual features and object locations; the decoder uses those locations to reconstruct scenes.Source: §3.3 · p. 6

3.4 OPTIMIZATION

  • The VAE and noise discriminator are trained jointly: VAE parameters use the augmented ELBO, while the discriminator distinguishes estimated residuals from permuted residuals.The discriminator learns to approximate a density ratio for estimating total correlation.Source: §3.4 · p. 6

4 EXPERIMENTS

The experiments evaluate whether LEAP recovers temporally causal latent processes under synthetic and real-world conditions, including assumption violations. Across synthetic benchmarks, LEAP recovers latent processes and causal relations; on real datasets, it outperforms baselines that do not use history information.

4.1 SYNTHETIC EXPERIMENTS

  • The synthetic experiments use 8 latent dimensions, lag L = 2, and a random three-layer MLP with LeakyReLU units for mixing.They include datasets designed for NP and VAR conditions and five types that violate the assumptions; metrics include MCC and causal-relation recovery.Source: §4 · p. 6, §4.1 · p. 7
  • On NP datasets, LEAP recovers latent processes and causal relations (SHD=5), while baselines lacking history or nonstationarity information and those assuming independent sources fail or distort recovery.LEAP-VAR gains partial identifiability; Figure 3 compares MCC, causal skeletons, estimated and true factors, and MCC trajectories.Source: §4.1 · p. 7, Figure 3 · p. 7
  • On VAR datasets, LEAP recovers latent processes and transition matrices, while baselines without history or assuming independent sources fail to recover them.The causal process prior and nonstationary flow improve identifiability, and the noise discriminator further increases MCC and reduces variance; Table 2 reports module contributions to MCC.Source: §4.1 · p. 7, Table 2 · p. 7, Figure 4 · p. 8
  • Assumption-violation tests show partial identifiability with changing causal relations or low-rank transitions, but instantaneous relations and Gaussian noise distort results.For NP processes, nonstationarity is necessary; MCC trajectories differed marginally between 15 and 20 regimes, suggesting full identifiability does not always require 2n + 1 = 17 regimes.Source: §4.1 · p. 7, Figure 5 · p. 8

4.2 REAL-WORLD APPLICATIONS: PERCEPTUAL CAUSALITY

  • On KiTTiMask and Mass-Spring system, LEAP’s parametric method considerably outperforms baselines that do not use history information.For CMU-MoCap, whose true latent variables are unknown, the authors qualitatively compare reconstructed processes and skeletons with baselines.Source: §4.2 · p. 8, Figure 6 · p. 8
  • On KiTTiMask, LEAP recovers three latent variables corresponding to pedestrian-mask vertical position, horizontal position, and scale, with an almost diagonal transition matrix.The gap from SlowVAE is relatively small because the learned latent processes seem independent, a case in which the VAR method reduces to SlowVAE.Source: §4.2 · p. 8, Figure 7 · p. 9
  • On the Mass-Spring system, LEAP recovers time-delayed cross-causal relations between ball locations and the spring connections with SHD=0.Figure 8 shows the recovered causal variables and entries of B1,2.Source: §4.2 · p. 8, Figure 8 · p. 9
  • For CMU-MoCap, three latent variables seem to represent pitch, yaw, and roll; pitch and roll are coupled, while yaw has independent dynamics.The latent coordinates show smooth cyclic patterns across 12 trials with slightly different walking dynamics, modeled as separate nonstationary-noise regimes.Source: §4.2 · p. 9, Figure 9 · p. 9

5 CONCLUSION AND FUTURE WORK

The paper establishes two provable conditions for identifying temporally causal latent processes from nonlinear mixtures. Its assumptions exclude instantaneous latent causal influence and changing causal influences across regimes; extending the theory and framework to these settings remains future work.

5 CONCLUSION AND FUTURE WORK

  • Two provable conditions identify temporally causal latent processes from observed nonlinear mixtures.The conditions are validated on datasets with the properties they require.Source: §5 · p. 9
  • The theory assumes no instantaneous causal influence between latent processes and causal influences that do not change across regimes.The authors note that these assumptions may not hold for some temporal data; instantaneous relations can distort identifiability, with their amount controllable by time resolution.Source: §5 · p. 9
  • Extending the identifiability theory and framework to changing causal relations and instantaneous dependencies remains future work.Experiments demonstrate possibilities for generalizing the results to changing dynamics, but the theory is not established for changing causal relations.Source: §5 · p. 9

Supplementary Materials for Learning Temporally Causal

The supplement discusses the assumptions behind the two identifiability conditions, their applicability, and how latent-space ambiguity is handled in the proof setup. Key boundaries include causal sufficiency, adequate temporal resolution, informative nonstationarity, and nonsingular state transitions.

Supplementary Materials for Learning Temporally Causal

  • The supplementary materials include assumption explanations and identifiability proofs, dataset and metric details, model and training specifications, additional experiments, and related work.The supplement also describes comparisons with prior work and the connection between theory and the training framework.Source: Supplementary Materials for Learning… · p. 12

A.2.1 INDEPENDENT NOISE (IN) CONDITION

  • Without latent confounders, causal sufficiency implies process noise is independent of its parent nodes, while temporal precedence makes the graph acyclic.The supplement connects independent noise in structural equation models to causal sufficiency.Source: §A.2.1 · p. 14
  • The IN condition is applicable when sampling is frequent enough to observe dynamics and latent causal confounders are absent; instantaneous relations may require added sparsity or minimality assumptions.The authors identify instantaneous latent dependencies as future work.Source: §A.2.1 · p. 14

A.2.2 NONSTATIONARY NOISE AND SUFFICIENT VARIABILITY CONDITION

  • The nonstationary condition keeps transition functions fixed across regimes while allowing noise distributions to vary, with sufficient variability requiring diverse regime effects.The modulation can vary distributional form, scale, or location; the authors describe nonstationary noise as relevant to EEG/MEG, natural video, and financial volatility.Source: §A.2.2 · pp. 14–15

A.2.3 GENERALIZED LAPLACIAN NOISE CONDITION

  • The parametric condition uses generalized Laplacian noise as an L1-sparse temporal prior, suited to sharp effects on some but not all latent factors between adjacent video frames.The prior connects to slow feature analysis and can fit perturbation shapes by changing its distribution parameters.Source: §A.2.3 · p. 15

A.2.4 NONSINGULAR STATE TRANSITIONS CONDITION

  • If state transitions are low rank, a lower-dimensional projection can be observationally equivalent, so the condition requires full-rank transitions to prevent that ambiguity.The supplement illustrates this with a two-dimensional VAR process that reduces to a single linear AR process under projection.Source: §A.2.4 · pp. 15–16

A.3.1 PRELIMINARIES

  • The proof setup represents latent-space ambiguity between observationally equivalent models with a map h, which the identifiability argument seeks to restrict to permutation and componentwise nonlinearities.The supplement describes the learned mixing map as the true mixing composed with this latent-space map.Source: §A.3.1 · p. 16

A.3.2 PROOF OF THEOREM 1

Theorem 1’s proof uses observational equivalence and sufficient variability to restrict latent-space ambiguity. The supplement also states the parametric theorem’s conditions, compares the results with prior theories, and explains how the estimation framework implements selected assumptions.

A.3.2 PROOF OF THEOREM 1

  • Under nonstationary noise, independent noise, and sufficient variability, Theorem 1 ensures componentwise identifiability of temporally causal latent processes.The proof restricts the latent ambiguity to a permutation with componentwise nonlinearities.Source: §A.3.2 · p. 17
  • Sufficient variability makes the regime-difference matrix nonsingular, forcing the proof’s linear-system coefficients to zero and restricting the latent-space Jacobian.The argument derives relations from observational equivalence, applies the independent-noise assumption, and collects the resulting equations across regimes.Source: §A.3.2 · pp. 17–19

A.3.3 PROOF OF THEOREM 2

  • The parametric theorem assumes generalized Laplacian process noise and at least one full-rank state transition matrix.The proof extends prior identifiability reasoning to time-delayed causal relations and multiple time lags.Source: §A.3.3 · p. 19

A.4 COMPARISONS WITH EXISTING THEORIES

  • Unlike PCL, SlowVAE, and iVAE, the paper targets time-delayed causal relations among latent sources; its conditions extend to nonstationarity or temporal functional and distributional constraints without sparsity or minimality assumptions.The parametric condition extends Laplacian innovations to multiple-lag vector autoregressive transitions, while the nonparametric condition allows unconstrained transition forms.Source: §A.4 · p. 21

A.5 CONNECTING THEORIES TO MODEL

  • The framework enforces conditional factorial properties through the causal transition prior and residual independence, models nonstationary noise with flow-based density estimators, and leaves sufficient variability and nonsingular transitions as data properties.Flow models are separated by regime and component; initialization differs between nonparametric and parametric processes.Source: §A.5 · p. 22

B.1 SYNTHETIC DATASET

The synthetic evaluation includes datasets satisfying the proposed assumptions and variants violating individual conditions. The nonparametric and parametric datasets use different noise and transition setups.

B.1 SYNTHETIC DATASET

  • Seven synthetic datasets comprise two that satisfy the assumptions and five that violate conditions in the proposed theorems.The setup uses latent size 8, lag number 2, and a random three-layer MLP mixing function.Source: §B.1 · p. 22
  • The nonparametric dataset uses 150,000 points, Gaussian noise whose variance is modulated across 20 regimes, and a two-layer MLP transition function.Variants restrict the number of observed regimes to assess insufficient variability and stationary noise.Source: §B.1 · p. 22
  • The parametric dataset uses 50,000 points with i.i.d. Laplacian noise and uniformly sampled transition-matrix entries; violation datasets include low-rank transitions, Gaussian noise, regime-varying causal relations, and instantaneous causal relations.The low-rank variant uses rank 4 rather than full rank 8; the Gaussian variant sets α_i = 2 rather than α_i < 2.Source: §B.1 · pp. 22–23

B.2 REAL-WORLD DATASET

The real-world evaluation uses three public datasets spanning pedestrian masks, a mass-spring video, and human motion capture. It also defines graph-distance and latent-recovery metrics.

B.2 REAL-WORLD DATASET

  • Three public datasets cover pedestrian segmentation, a five-degree-of-freedom mass-spring system, and human motion capture; CMU MoCap uses 62 joint-location measurements.The CMU data comprise 12 walk trials from Subject 7.Source: §B.2 · pp. 23–24

B.3 EVALUATION METRICS

  • SHD counts edge insertions, deletions, or flips needed to transform one causal graph into another.It is a Minimum Edit Distance variant for causal discovery.Source: §B.3 · p. 24
  • MCC evaluates latent-factor recovery by matching each ground-truth factor to an estimated variable using absolute correlations and an assignment problem.Pearson or Spearman correlations are used depending on whether recovered factors have componentwise invertible nonlinearities.Source: §B.3 · p. 24

C IMPLEMENTATION DETAILS

  • The implementation-details section introduces architecture, hyperparameter selection and sensitivity analysis, and training settings.Source: C IMPLEMENTATION DETAILS · p. 24

C.1 NETWORK ARCHITECTURE

LEAP combines observation-specific encoders and decoders with temporal inference and a learned causal-process prior. Its architecture varies across synthetic, motion-capture, and video data.

C.1 NETWORK ARCHITECTURE

  • A bidirectional GRU processes observations in both temporal directions, while a second inference module uses inferred past latents to compute each posterior.Skip connections link the modules; the first L latent variables use an isotropic Gaussian prior, and later priors come from the learned transition network.Source: §C.1 · p. 24

3.1.2 CNN-Encoder

  • MLP encoders and decoders serve synthetic and motion-capture data, while video uses CNNs and a separate keypoint predictor for Mass-Spring objects.The Mass-Spring decoder retrieves object features using predicted locations to reconstruct scenes.Source: §C.1 · p. 24, §3.1.2 · p. 26, §4.1 · p. 26
  • For Mass-Spring video, a feature extractor and spatial-softmax keypoint predictor provide object features and locations to the scene decoder.The keypoint predictor outputs two-dimensional locations for each object.Source: §C.1 · p. 24, §3.1.2 · p. 26

C.1 NETWORK ARCHITECTURE

  • The causal-process prior uses inverse transitions, Jacobian log-determinants for nonparametric transitions, and componentwise spline flows to model estimated-noise distributions.VAR transitions use lag-specific state-transition matrices and have log-determinant zero under the additive-noise assumption.Source: §C.1 · p. 25
  • Ablation variants add the learned causal-transition prior and then nonstationary flow estimators, with the latter variant excluding the noise discriminator.The causal-prior variant uses MSE in place of flow density estimators when estimating the KL divergence.Source: §C.1 · p. 25

3.2 KiTTiMask-Encoder

  • The KiTTiMask encoder maps semantic-segmented frames through convolutional layers to a latent-sized feature representation.Source: §3.2 · p. 26

4.1 CNN-Decoder

  • The Mass-Spring refiner takes sampled latent sequences and reconstructs RGB video scenes.Source: §4.1 · p. 26

4.2 KiTTiMask-Decoder

  • The KiTTiMask decoder maps latent sequences through dense and transpose-convolution layers to estimated mask frames.Source: §4.2 · p. 26

C.2.1 HYPERPARAMETER SELECTION AND SENSITIVITY ANALYSIS

Across the tested latent sizes and maximum lags, MCC varied only marginally in both parametric and nonparametric settings. The analysis finds greater sensitivity to β and γ than to σ.

C.2.1 HYPERPARAMETER SELECTION AND SENSITIVITY ANALYSIS

  • β and γ had larger effects on identifiability than σ, while results were robust to latent size and maximum-lag choices.Validation data were used to set hyperparameters whose effects were more consequential.Source: §C.2.1 · p. 27

Parametric (VAR) Dataset

  • MCC scores were around 0.9 across VAR latent sizes n ∈ [4, 6, 8] and lags L ∈ [2, 3, 4], with marginal differences.The selected values were [β, γ, σ] = [3E-3, 9E-3, 1E-6].Source: §C.2.1 · p. 27

Nonparametric (NP) Dataset

  • MCC scores were around or above 0.85 across NP latent sizes n ∈ [6, 7, 8] and lags L ∈ [1, 2, 3], with marginal differences.The selected values were [β, γ, σ] = [2E-3, 2E-2, 1E-6].Source: §C.2.1 · pp. 27–28

C.2.2 TRAINING

Training uses AdamW for the VAE and SGD for the noise discriminator, with early stopping based on validation ELBO. The authors also describe stability choices for latent size and learning rates.

C.2.2 TRAINING

  • VAE training used AdamW for up to 200 epochs, stopping after five epochs without validation-ELBO improvement; four-seed mean performance and standard deviation were reported.The learning rate was 0.002 and batch size was 32; the discriminator used SGD at learning rate 0.001.Source: §C.2.2 · p. 28

Computing Hardware

  • Experiments ran on a machine with two GeForce GTX 1080 Ti GPUs, each with 11GB memory.The CPU was an Intel i7-7700K with 32 logical CPU units.Source: §C.2.2 · p. 28

Training Stability

  • For real-world data, the latent size was set slightly above the true size; AdamW and a higher VAE learning rate than discriminator rate were used as stability measures.The larger latent size was intended to include meaningful latents, while the learning-rate difference was intended to prevent extreme discriminator extrapolation.Source: §C.2.2 · p. 28

D.1 COMPARISONS BETWEEN LEAP AND BASELINES ON CMU-MOCAP DATASET

On CMU-Mocap, LEAP uses three active latent variables to represent walking with smooth transitions and recovers relations between causal factors; on Mass-Spring, it recovers object locations and the underlying skeleton.

D.1 COMPARISONS BETWEEN LEAP AND BASELINES ON CMU-MOCAP DATASET

  • Three of eight latent variables represent walking with smooth transitions under LEAP, while BetaVAE and SlowVAE use all eight.The three variables encode pitch, yaw, and roll rotations of walking cycles.Source: Figure D.1 · p. 29, §D.1 · p. 29

D.2 MASS-SPRING SYSTEMS

  • The learned Mass-Spring representation disentangles five objects and encodes the balls’ horizontal and vertical locations, with the recovered skeleton matching the underlying processes at SHD=0.The latent variables are visualized as keypoint heatmaps, and the skeleton is reconstructed from estimated state-transition matrices.Source: §D.2 · p. 29, Figure D.3 · p. 30

D.1 COMPARISONS BETWEEN LEAP AND BASELINES ON CMU-MOCAP DATASET

  • LEAP recovers cross-relations between walking factors, linking pitch and roll dynamics while yaw remains independent; BetaVAE and SlowVAE recover only independent relations.The inferred relations are described as closer to reality than the independent transitions recovered by the baselines.Source: §D.1 · p. 29, Figure D.2 · p. 30

E EXTENDED RELATED WORK

  • Prior nonlinear ICA work used temporal dependencies or nonstationarity to identify independent sources, while Yang et al. (2021) added a linear causal layer for causal variables.The cited approaches include TCL, PCL, HMM-based modeling, and VAE modeling of nonstationary regimes.Source: E EXTENDED RELATED WORK · p. 30

Related papers

Loading 2110.05428v4…