PaperlayerView exact revision on arXiv

Source-linked AI summary

Causal Representation Learning for Instantaneous and Temporal Effects in Interactive Systems

Phillip Lippe, Sara Magliacane, Sindy Löwe, Yuki M. Asano, Taco Cohen, Efstratios Gavves

arXiv:2206.06169v2cs.LGcs.AIstat.ML

TL;DR

Temporal causal representation methods assume no instantaneous relations, although effects may occur faster than the observation rate. iCITRIS jointly learns minimal causal variables and their temporal and instantaneous graph from intervened sequences, and accurately recovers both in three video datasets.

  • Problem

    Prior temporal-sequence methods assume causal variables are conditionally independent within each time step, limiting their identifiability results when effects occur faster than the measurement rate.

    Source: §1 · p. 1, §2.2 · p. 3

  • Method

    iCITRIS learns minimal causal variables and their instantaneous and temporal graph from temporal observations with known intervention targets, using interventions that remove instantaneous parents.

    Source: §1 · p. 2

  • Results

    iCITRIS accurately identifies causal variables and their graph in three video datasets.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    With partially-perfect interventions and known targets, the method can identify minimal causal variables and their graph despite instantaneous effects.

    Source: §1 · p. 2

    The method assumes known, non-deterministically related intervention targets and partially-perfect interventions; it is limited to acyclic graphs.

    Source: §6 · p. 9

Abstract

from arXiv · show

Causal representation learning is the task of identifying the underlying causal variables and their relations from high-dimensional observations, such as images. Recent work has shown that one can reconstruct the causal variables from temporal sequences of observations under the assumption that there are no instantaneous causal relations between them. In practical applications, however, our measurement or frame rate might be slower than many of the causal effects. This effectively creates "instantaneous" effects and invalidates previous identifiability results. To address this issue, we propose iCITRIS, a causal representation learning method that allows for instantaneous effects in intervened temporal sequences when intervention targets can be observed, e.g., as actions of an agent. iCITRIS identifies the potentially multidimensional causal variables from temporal observations, while simultaneously using a differentiable causal discovery method to learn their causal graph. In experiments on three datasets of interactive systems, iCITRIS accurately identifies the causal variables and their causal graph.

1 INTRODUCTION

Prior temporal causal representation methods assume variables are conditionally independent within each measured time step, but effects faster than the measurement rate can violate that assumption. iCITRIS addresses instantaneous effects by jointly identifying causal variables and their graph, using interventions that remove instantaneous parents.

1 INTRODUCTION

  • Effects faster than the measurement or frame rate can create instantaneous causal relations, violating prior methods’ within-step conditional-independence assumption.The introduction gives a light switch and bulb as an example of an intervention affecting another variable within the same time step.Source: §1 · p. 1
  • Instantaneous-edge orientations are not determined by temporal ordering, so identifying variables and learning their graph must be solved jointly.Without the variables, the graph cannot be identified; without the graph, variables may not be conditionally independent.Source: §1 · pp. 1–2
  • With known-target, partially-perfect interventions, the paper proves that minimal causal variables and their temporal and instantaneous graph are identifiable under its assumptions.Partially-perfect interventions remove instantaneous parents; without them, causal variables are generally not identifiable.Source: §1 · p. 2
  • iCITRIS integrates differentiable causal discovery into its prior and accurately recovers causal variables and their graph in three video datasets.The method extends CITRIS, which assumes all causal relations are temporal.Source: §1 · p. 2
  • Unlike prior temporal-sequence work, iCITRIS does not require counterfactual observations or data labeled with the true causal variables, but does require known intervention targets.The related-work discussion identifies these as the setting’s requirements and contrasts them with earlier approaches.Source: §1 · p. 2

2 RELEVANT BACKGROUND AND DEFINITIONS

The background defines temporal intervened sequences as latent causal processes observed through high-dimensional measurements, with known intervention targets. It then describes minimal causal variables and the CITRIS framework for identifying them from temporal data.

2.1 TEMPORAL INTERVENED SEQUENCES

  • TRIS models K latent causal variables across time, with each variable’s parents drawn from the previous time step and intervention indicators representing which variables were intervened upon.The setup assumes known intervention indicators and time-invariant, causally sufficient, faithful graphs.Source: §2.1 · pp. 2–3
  • High-dimensional observations are noisy, entangled views of the causal variables, and the observation function is assumed bijective for unique identification.Causal variables may be multidimensional; interventions can affect only part of a variable, motivating a split into intervention-dependent and intervention-independent parts.Source: §2.1 · p. 3, §2.2 · p. 3

2.2 MINIMAL CAUSAL VARIABLES AND IDENTIFIABILITY CLASS

  • A minimal causal variable is the intervention-dependent part of a potentially multidimensional causal variable, with the split chosen to maximize information in the remaining part.The setup distinguishes the intervention-dependent and intervention-independent parts of a variable.Source: §2.2 · p. 3
  • CITRIS maps observations to latents, assigns latent dimensions to causal variables, and models each group conditioned on the previous step and its intervention; temporal dependencies can then be pruned to recover the graph.Its identifiability result requires maximizing latent information while fitting data and assumes no intervention target is a deterministic function of another.Source: §2.2 · p. 3

3 IDENTIFYING CAUSAL VARIABLES WITH INSTANTANEOUS EFFECTS

The section extends temporal intervened sequences to allow acyclic instantaneous effects and establishes when minimal causal variables and their graph can be identified. It motivates partially-perfect interventions and describes a model that learns instantaneous and temporal dependencies.

3.1 ITRIS AND CHALLENGES OF INSTANTANEOUS EFFECTS

  • Allowing instantaneous effects enlarges the model space and can make causal variables and their graph unidentifiable from observational distributions alone.The section illustrates this with alternative representations that model the same observation distribution.Source: §3.1 · p. 4
  • iTRIS permits acyclic causal relations among variables in the same time step, but instantaneous-parent variables are not identifiable without partially-perfect interventions on them.A partially-perfect intervention removes all parents in the instantaneous graph; the assumed intervention setup subsumes TRIS soft interventions when the instantaneous graph is empty.Source: §3.1 · p. 4

3.2 IDENTIFYING THE MINIMAL CAUSAL VARIABLES IN ITRIS

  • The theoretical identifiability analysis assumes continuous causal variables with full-support distributions and additional conditions that break distributional symmetries.The experiments extend the results to categorical and circular variables and limited-support distributions.Source: §3.2 · p. 5
  • iCITRIS learns a directed acyclic graph over latent variable groups for instantaneous relations, while modeling temporal relations through previous-step conditioning and pruning.The latent groups are aligned with causal variables; the temporal graph is learned implicitly and can be pruned after training.Source: §3.2 · p. 5
  • Under the stated constraints, iTRIS identifies minimal causal variables and their complete instantaneous and temporal graph, extending CITRIS’s result to instantaneous effects.The theorem requires maximizing latent information and minimizing graph edges while fitting the likelihood, with no deterministically related intervention variables.Source: §3.2 · p. 5

4 CAUSAL REPRESENTATION LEARNING WITH INSTANTANEOUS EFFECTS

iCITRIS extends CITRIS to temporal sequences with instantaneous causal effects and jointly learns causal variables and their graph. It combines differentiable graph discovery with optimization strategies intended to stabilize this joint learning.

4.1 BASELINE: CITRIS

  • iCITRIS extends the CITRIS VAE prior to model instantaneous effects while retaining its latent-variable learning setup.CITRIS uses an encoder and decoder to approximate an invertible map and trains by maximum likelihood on p(Xt+1|Xt, It+1).Source: §4.1 · p. 6, §4 · p. 6

4.2 LEARNING THE INSTANTANEOUS CAUSAL GRAPH

  • The model jointly optimizes the latent-variable distribution and graph distribution using a likelihood objective, with NOTEARS or ENCO for differentiable graph discovery.ENCO uses interventions to update edge orientations and is merged into a joint update of graph and distribution parameters.Source: §4.2 · p. 6
  • NOTEARS enforces acyclicity through a continuous adjacency-matrix constraint and uses sampled graphs to mask latent-variable parents.The constraint is added as a regularizer whose weight increases during training.Source: §4.2 · p. 6

4.3 STABILIZING THE OPTIMIZATION PROCESS

  • Jointly learning variables and graph can be unstable because each task depends on the other, motivating the proposed stabilization approaches.Incorrect graphs and local minima are identified as risks of this chicken-and-egg optimization.Source: §4.3 · p. 6
  • A graph-learning schedule freezes graph parameters initially, then warms up their learning rate while latent variables begin aligning with intervention variables.Early graph gradients are noisy and uninformative, so the initial fit uses an arbitrary graph.Source: §4.3 · p. 6
  • For fully perfect interventions, an MI estimator regularizes latent parents and children toward independence under interventions.The method trains a discriminator to distinguish joint samples from samples with shuffled parents, while the latents optimize in the opposite direction.Source: §4.3 · p. 7

5 EXPERIMENTS

Across three video datasets, the experiments evaluate causal-variable identification and graph recovery under instantaneous effects. iCITRIS-ENCO performs well across the tested graph structures and sizes, including visually complex and game-like environments.

5.1 EXPERIMENTAL SETTINGS

  • The Voronoi benchmark uses random, chain, and full instantaneous graphs with single-target perfect interventions and image observations generated through a normalizing flow.Evaluation reports variable-correlation R2 scores and graph Structural Hamming Distance; the datasets vary in difficulty and are compared with common causal representation methods.Source: §5.2 · p. 7, §5.1 · p. 7, §5 · p. 7

5.2 2D COLORED CELLS WITH CAUSAL EFFECTS: VORONOI BENCHMARK

  • iCITRIS-ENCO identifies variables and graphs with minor errors on random and chain graphs up to 9 variables, and outperforms other models on full graphs.Across graph structures and sizes, ENCO outperforms NOTEARS, whose edge-orientation errors worsen variable identification as graphs grow.Source: §5.2 · pp. 7–8, Figure 2 · p. 8

5.3 3D OBJECT RENDERINGS: INSTANTANEOUS TEMPORAL CAUSAL3DIDENT

  • The Causal3DIdent experiment uses partially-perfect interventions that remove instantaneous parents while retaining temporal dependencies, and evaluates seven variables with categorical and circular domains.Its instantaneous graph contains eight edges among the seven variables.Source: §5.3 · p. 8
  • On Instantaneous Temporal Causal3DIdent, iCITRIS-ENCO identifies variables well and recovers most instantaneous relations, with up to two errors on average.Temporal recovery has more false positives from minor correlations; iCITRIS-ENCO performs best in identifying variables and graph, while NOTEARS misorients several edges.Source: §5.3 · pp. 8–9, Table 1 · p. 8

5.4 REAL GAME DYNAMICS: CAUSAL PINBALL

  • On Causal Pinball, iCITRIS-ENCO identifies variables and the instantaneous graph with minor errors despite deterministic effects and other violated assumptions.CITRIS entangles variables more strongly, while iVAE has difficulty identifying all variables.Source: §5.4 · p. 9

6 CONCLUSION AND DISCUSSION

iCITRIS jointly identifies minimal causal variables and instantaneous and temporal graphs from intervened sequences, with accurate recovery reported across three video datasets. Its theoretical identifiability results rely on specific intervention conditions, and the method is limited to acyclic graphs.

6 CONCLUSION AND DISCUSSION

  • iCITRIS jointly learns minimal causal variables and their instantaneous and temporal graph from intervened sequences, with accurate recovery reported across three video datasets.The method is intended for temporal sequences with potentially instantaneous effects.Source: §6 · p. 9
  • The identifiability theorem requires known intervention targets that are non-deterministically related and partially-perfect interventions that can remove instantaneous parents.Without the first condition variables may become entangled; without the latter, nonexistent instantaneous relations may be predicted.Source: §6 · p. 9
  • iCITRIS is limited to acyclic graphs, although instantaneous cycles could occur at low frame rates.Learning intervention targets from action sequences is also identified as future work.Source: §6 · p. 9

SUPPLEMENTARY MATERIAL CAUSAL REPRESENTATION LEARNING FOR INSTANTANEOUS AND TEMPORAL EFFECTS

The paper frames iCITRIS as a step toward causal reasoning from high-level observations and discusses potential applications and risks. It identifies unknown intervention targets as a barrier to interactive applications and proposes combining causal representation learning with object-centric methods as a possible direction.

A BROADER IMPACT

  • The paper positions causal representation learning from pixels as a route toward reasoning about objects and abstract concepts, with reinforcement learning as a possible future application.A robotic system could interact with an environment, but the work assumes interventions with known targets.Source: A BROADER IMPACT · p. 17
  • Combining iCITRIS with unsupervised object-centric learning is proposed as a possible way to use objects as a prior disentanglement of information.The paper suggests this combination could relax further assumptions and open additional applications.Source: A BROADER IMPACT · p. 17
  • Incorrect application could be used to justify false causal relations, so users are responsible for applying the method within ethical boundaries and critically evaluating its outputs.The authors say this responsibility will be emphasized in the public code license.Source: A BROADER IMPACT · p. 17

B REPRODUCIBILITY STATEMENT

The appendices document reproducibility resources, situate iCITRIS among related approaches, and lay out notation and assumptions used in the identifiability proofs.

B REPRODUCIBILITY STATEMENT

  • Code and dataset-generation resources are provided; experiments use at least three seeds, and the reported total compute is approximately 80 GPU days.The Voronoi benchmark uses five seeds; the compute estimate excludes hyperparameter search and research trials.Source: B REPRODUCIBILITY STATEMENT · p. 17

C EXPANDED RELATED WORK

  • The related-work review contrasts ICA-style conditional-independence setups and counterfactual-pair methods with iCITRIS’s temporal intervened-sequence setting.It states that iCITRIS does not require counterfactuals or data labeled with the true causal variables.Source: C EXPANDED RELATED WORK · p. 18

D PROOFS

  • The proof appendix explains its notation and assumptions before presenting the identifiability proof in multiple steps.It also includes a proof of Lemma 3.2.Source: D PROOFS · p. 19

D.1 PRELIMINARIES

  • The preliminaries formalize the observation-to-latent mapping, assignments of latent dimensions to causal factors, and the learned directed acyclic graph.The proof uses limiting density of discrete points (LDDP) as an entropy measure invariant to invertible transformations.Source: §D.1 · pp. 19–20

D.2 ASSUMPTIONS FOR IDENTIFIABILITY

  • The assumptions discussion explains why the conditions for identifying the causal variables and graph are needed and what violations can cause.Source: §D.2 · p. 20

D.2.1 ASSUMPTION 1: THE INTERVENTIONS ON THE CAUSAL VARIABLES REMOVE INSTANTANEOUS PARENTS

The proof explains why identifiability requires partially-perfect interventions and additional conditions on intervention targets, distributions, and graph structure. It then shows how optimal representations identify minimal causal variables and their graph.

D.2.1 ASSUMPTION 1: THE INTERVENTIONS ON THE CAUSAL VARIABLES REMOVE INSTANTANEOUS PARENTS

  • Interventions must remove instantaneous parents; without partially-perfect interventions, distinct representations can fit the observations, leaving causal variables unidentifiable.The example shows that soft interventions may not distinguish causal relations introduced by the observation function from relations in the true model.Source: §D.2.1 · pp. 20, 22

D.2.2 ASSUMPTION 2: THE INTERVENTION VARIABLES ARE NOT A DETERMINISTIC FUNCTION OF EACH OTHER

  • Intervention targets must distinguish causal variables: variables always jointly intervened upon or passively observed cannot be disentangled, and their graph may remain unidentified.The passage notes that graph identification can fail for arbitrary distributions when only joint interventions on two variables are observed.Source: §D.2.2 · p. 23

D.2.3 ASSUMPTION 3: DISTRIBUTIONS HAVE FULL SUPPORT

  • Observational and interventional distributions must share support, so an observation alone cannot reveal whether it came from an intervention.Without shared support, encodings could depend on intervention targets already inferred from the observation.Source: §D.2.3 · p. 23

D.2.4 ASSUMPTION 4: TEMPORAL CONNECTIONS AND INTERVENTIONS BREAK ALL SYMMETRIES IN THE DISTRIBUTIONS

  • Temporal and intervention parents must break shared mechanism symmetries; otherwise, functional dependencies between representations can preserve distributions and defeat identification.The example describes an alternative encoding that retains likelihood while the true causal variable cannot be recovered from its encoded value alone.Source: §D.2.4 · p. 24

D.2.5 ASSUMPTION 5: CAUSAL GRAPH STRUCTURE REQUIREMENTS

  • Graph identification assumes an acyclic, faithful graph and causal sufficiency; cyclic instantaneous graphs and latent confounders fall outside this setting.The authors leave extension to cyclic causal graphs for future work.Source: §D.2.5 · p. 25

D.3 THEOREM 3.4 - PROOF OUTLINE

  • Theorem 3.4’s proof shows that the true representation is a likelihood optimum, all optima identify minimal causal variables, and the learned graph contains the true edges.The proof combines these steps to derive the theorem.Source: §D.3 · p. 25

D.4 THEOREM 3.4 - PROOF STEP 1: THE TRUE MODEL IS A GLOBAL OPTIMUM OF THE LIKELIHOOD OBJECTIVE

This section proves that the true causal representation is a global optimum of the likelihood objective and describes how interventions and latent assignments identify minimal causal variables and constrain the learned graph. It also states that the maximum-likelihood solution uniquely identifies causal orientation under the specified experiment sets and explains the role of additional latent variables in graph analysis.

D.4 THEOREM 3.4 - PROOF STEP 1: THE TRUE MODEL IS A GLOBAL OPTIMUM OF THE LIKELIHOOD OBJECTIVE

  • The true identification function, assignment, and causal graph attain a global maximum of p(Xt+1|Xt, It+1), but this solution need not be unique.The resulting likelihood matches the true generative model, so reparameterizations or other directed acyclic graphs cannot achieve a higher likelihood.Source: §D.4 · pp. 26–27

D.5.1 MINIMAL CAUSAL VARIABLES

  • A minimal causal variable splits each causal variable into an intervention-dependent part svar(Ci) and an invariant part sinv(Ci); interventions ensure the invariant part has no instantaneous parents.If a variable is never intervened on, its split is sinv(Ci)=Ci and svar(Ci)=∅.Source: §D.5.1 · p. 28

D.5.2 IDENTIFYING THE MINIMAL CAUSAL VARIABLES

  • Every maximum-likelihood representation has a deterministic map from zψi to svar(Ci), although zψi may also contain additional information.This result holds across representation functions in the solution class, including solutions with different learned graph structures.Source: §D.5.2 · p. 29

D.5.3 DISENTANGLING THE MINIMAL CAUSAL VARIABLES

  • For every maximum-likelihood representation, zψi contains no information about another minimal causal variable svar(Cj), because interventions remove incoming edges to intervened variables.The proof uses intervention-augmented graphs to show that each zψi must exclude the other variables’ minimal causal information.Source: §D.5.3 · p. 30

D.5.4 DISENTANGLING THE REMAINING VARIABLES

  • Maximizing the information content of zψ0 allows the remaining latent variables zψi to model exclusively their respective minimal causal variables svar(Ci).The remaining information—including invariant causal parts, non-intervened variables, and noise—may be arbitrarily entangled in zψ0, and pointwise dependencies can remain.Source: §D.5.4 · p. 31

D.6 THEOREM 3.4 - PROOF STEP 3: IDENTIFIABILITY OF THE CAUSAL GRAPH

  • The graph-identifiability analysis considers the optimal graph on identified minimal variables, instantaneous-graph identification, and temporal-graph identification.Source: §D.6 · p. 31

D.6.1 CAUSAL GRAPH ON MINIMAL CAUSAL VARIABLES

  • The learned graph on minimal causal variables can differ from the full causal graph, but its parent sets are restricted to parents’ minimal parts and zψ0.A ground-truth edge between full variables may be represented through zψ0 when the causal effect comes from an invariant part.Source: §D.6.1 · p. 31
  • Under each described experiment set, the maximum-likelihood solution for p(C1, C2|I1, I2) uniquely identifies the causal orientation.Source: §D.6.1 · p. 32
  • The additional latent variables zψ0 summarize invariable intervened-variable parts, non-intervened causal variables, and noise, so they cannot be instantaneous children of minimal causal variables.The instantaneous-graph orientations for those edges can therefore be predefined.Source: §D.6.1 · p. 32
  • For simplicity, the section uses “identifying the graph on C1, ..., CK” to mean identifying the graph on the minimal causal variables.Source: §D.6.1 · p. 32

D.6.2 OPTIMIZING THE MAXIMUM LIKELIHOOD OBJECTIVE UNIQUELY IDENTIFIES THE INSTANTANEOUS CAUSAL GRAPH UNDER INTERVENTIONS

Under the stated intervention setups, maximizing likelihood identifies instantaneous edge orientations; structural minimality then recovers the full instantaneous graph. Temporal edges are identified by pruning conditionally redundant connections.

D.6.2 OPTIMIZING THE MAXIMUM LIKELIHOOD OBJECTIVE UNIQUELY IDENTIFIES THE INSTANTANEOUS CAUSAL GRAPH UNDER INTERVENTIONS

  • Likelihood identifies instantaneous edge orientations across all valid experimental settings, including when a variable is observed only under joint intervention.For two causally related variables, the orientation is identified by optimizing p(Ci, Cj|Ii, Ij).Source: §D.6.2 · pp. 32–33
  • Structural minimality selects the full causal graph from acyclic likelihood-equivalent graphs that contain the true edges.The remaining non-identified edges connect variables that are independent conditioned on their parents and do not affect the likelihood.Source: §D.6.2 · p. 33

D.6.3 IDENTIFYING THE TEMPORAL CAUSAL RELATIONS BY PRUNING EDGES

  • Temporal edges can be identified by starting from the fully connected previous-to-next-step graph and removing pairs conditionally independent given the other previous-step variables.The true temporal graph is a subset of that graph, and faithfulness makes conditional independence the pruning criterion.Source: §D.6.3 · p. 34

D.7 THEOREM 3.4 - PROOF STEP 4: FINAL IDENTIFIABILITY RESULT

The theorem states conditions under which iTRIS identifies the causal system: likelihood maximization, maximum information content for the remaining latent variables, minimum graph edges, and non-deterministically related interventions. Together, these conditions yield the stated identifiability result.

D.7 THEOREM 3.4 - PROOF STEP 4: FINAL IDENTIFIABILITY RESULT

  • Likelihood maximization identifies the minimal causal variables, while maximizing the remaining latent-variable information and minimizing edges completes graph identification.The proof summary says the resulting graph must contain at least the ground-truth edges before the least-edge choice recovers the full graph.Source: §D.7 · p. 34
  • iTRIS identifies the causal system when likelihood is maximized, H(z_{t+1,ψ0}|z_t) is maximized through LDDP, and the graph has the fewest edges.The theorem also requires that intervention variables are not deterministically related.Source: §D.7 · p. 34

E DATASETS

This appendix describes the Voronoi, Instantaneous Temporal Causal3DIdent, and Causal Pinball datasets, including their causal variables, graph structures, and intervention setups. It also details iCITRIS’s training schedule, model architectures, and optimization choices used in the experiments.

E.1 VORONOI BENCHMARK

  • The Voronoi benchmark provides flexible synthetic data for varying variable counts and instantaneous and temporal graph structures, with 150k-step sequences and single-target interventions.Each variable is intervened on with probability 1/(K + 2), while the purely observational regime has probability 2/(K + 2).Source: §E.1 · p. 35

E.1.1 NETWORK SETUP

  • Voronoi causal mechanisms use neural networks to parameterize conditional Gaussian means, and a normalizing flow entangles the causal variables before mapping them to image colors.The conditional standard deviation is σ = 0.3; the image uses an irregular Voronoi diagram whose mapping from images to colors does not transfer across datasets.Source: §E.1.1 · pp. 35–36

E.1.2 GRAPH GENERATIONS

  • The Voronoi benchmark uses random, chain, and full instantaneous graphs, plus random temporal graphs with orientations fixed by time order.Random instantaneous graphs are acyclic and may leave variables without incoming or outgoing edges; chain graphs have K − 1 edges, while full graphs include every edge consistent with an ordering.Source: §E.1.2 · p. 36

E.2 INSTANTANEOUS TEMPORAL CAUSAL3DIDENT

  • Instantaneous Temporal Causal3DIdent adapts Temporal Causal3DIdent to include instantaneous effects while retaining its seven causal factors and underlying mechanisms.The adaptation moves acyclic cross-variable temporal relations into instantaneous relations; for example, an intervention on position can immediately affect spotlight rotation.Source: §E.2 · p. 37

E.3 CAUSAL PINBALL

  • Causal Pinball represents the game with five multidimensional causal variables, including ball state, two paddle positions, bumpers, and score, with immediate effects among them.Paddle interventions occur in 20% of frames, ball interventions in 10%, and score and bumper interventions in 5% each; the dataset consists of a 150k-image sequence rendered at 64 × 64 pixels.Source: §E.3 · pp. 38–39, Figure 16 · p. 39

F.1 ICITRIS - MODEL DETAILS

  • iCITRIS adds continuous-optimization graph learning to the CITRIS model, using NOTEARS or ENCO to learn instantaneous causal structure.NOTEARS uses differentiable acyclicity and sparsity regularizers; ENCO estimates edge existence and orientation from sampled graphs. The appendix also describes a mutual-information estimator used for perfect interventions and optimization stability.Source: §F.1 · pp. 40–41
  • iCITRIS initially trains its mutual-information estimator using only temporal parents, then progressively adds instantaneous parents to reduce instability from early random graphs.Under perfect interventions, the optimal classifier for true causal variables reaches 0.5; latent gradients move the classifier toward that value.Source: §F.1 · p. 42

F.2 HYPERPARAMETERS

  • VAE-based methods share an encoder, decoder, and normalizing-flow prior architecture, while their priors differ in whether they model instantaneous parents and how they parameterize latents.iCITRIS models instantaneous parents in its prior, CITRIS omits them, and iVAE and iVAE-AR use MLP-based priors.Source: §F.2 · p. 42
  • For iCITRIS and CITRIS, pretrained autoencoders with noisy, L2-regularized latents precede normalizing-flow training on the Causal3DIdent and Causal Pinball datasets.The latent noise standard deviation is 0.05; the L2 weights are 1e-5 for Causal3DIdent and 1e-6 for Causal Pinball.Source: §F.2 · p. 43
  • All models use Adam with a 1e-3 learning rate, a 100-step warmup, and cosine annealing to 5e-5.Source: §F.2 · p. 43

F.3 EVALUATION METRICS

The evaluation combines latent-to-causal assignment and graph prediction with a test of generated interventional distributions. The appendix also reports experimental extensions, hyperparameters, and results across multiple seeds.

F.3 EVALUATION METRICS

  • For graph evaluation, iCITRIS and CITRIS use the learned assignment function, while iVAE uses ground-truth correlations; ENCO then learns the graph, pruning redundant edges for iCITRIS.The ground-truth-based iVAE assignment gives it an advantage; iCITRIS reuses learned edge orientations and relearns edge existence for pruning.Source: §F.3 · pp. 43–45
  • Hyperparameters were searched individually; iVAE networks were kept smaller on the latter two datasets, and iCITRIS warmup and weighting choices tolerated variation.The graph-learning warmup was shared across datasets, while the graph-sparsity regularizer was held constant across graph structures of the same size.Source: Table 4 · p. 44
  • A discriminator tests whether generated samples match true interventional distributions, jointly probing causal-variable disentanglement and graph accuracy.The models sample next-step latents under novel interventions, map them to causal-variable space, and compare generated samples with test data.Source: §F.3 · p. 45, Table 5 · p. 46

G ADDITIONAL EXPERIMENTAL RESULTS AND ABLATION STUDIES

  • The additional-results section reports standard deviations, intervention-outcome metrics, Voronoi ablations, and predicted-graph visualizations for two datasets.The visualizations cover Instantaneous Temporal Causal3DIdent and Causal Pinball.Source: G ADDITIONAL EXPERIMENTAL RESULTS AND… · p. 46

G.1 VORONOI BENCHMARK

The Voronoi experiments compare disentanglement, graph prediction, and intervention-sample fit, then test noisy targets and settings without instantaneous or temporal relations. Additional experiments report on Instantaneous Temporal Causal3DIdent and the original temporal dataset.

G.1 VORONOI BENCHMARK

  • On size-9 random graphs, 90% of iVAE-AR samples were classifiable, versus 55% accuracy for iCITRIS on small graphs (50% is random); iCITRIS still outperformed baselines on larger graphs.The discriminator measures the gap between true intervention samples and generated ones; performance declined for iCITRIS on larger graphs.Source: §G.1 · p. 46
  • iCITRIS-ENCO has higher graph-prediction recall and precision across graph structures and sizes; temporal predictions tend toward excess edges, while baselines have low instantaneous-graph recall.The tables report instantaneous and temporal graph recall and precision, highlighting false negatives and false positives.Source: §G.1 · pp. 46–48, Table 6 · p. 47, Table 7 · p. 47

G.1.1 ABLATION 1: NOISY INTERVENTION TARGETS

  • With 10% false-positive intervention targets, iCITRIS variables remain almost as well disentangled as before, though noisy interventions partly add temporal variables during post-processing.The ablation uses random graphs with six variables and simulates flagged interventions that did not succeed.Source: Table 8 · p. 48, §G.1.1 · p. 48

G.1.2 ABLATION 2: EMPTY INSTANTANEOUS GRAPH

  • With an empty instantaneous graph, iCITRIS, CITRIS, and iVAE identify the causal variables and graph, supporting iCITRIS as a replacement under perfect interventions.This setting has variables conditionally independent given the previous time step.Source: §G.1.2 · p. 48

G.1.3 ABLATION 3: EMPTY TEMPORAL GRAPH

  • With no temporal relations, none of the methods identifies the causal variables; different optimization strategies are needed for this setting.The models find independent factors of variation rather than causal variables when previous-step information is unavailable.Source: §G.1.3 · p. 49

G.2 INSTANTANEOUS TEMPORAL CAUSAL3DIDENT

  • Instantaneous Temporal Causal3DIdent results include correlation, graph-prediction, and triplet metrics, with standard deviations across three seeds.Triplet distance measures whether causal-factor combinations in latent space avoid correlations among factors.Source: §G.2 · p. 49, Table 9 · p. 49

G.2.1 ABLATION STUDY 4: ORIGINAL TEMPORAL CAUSAL3DIDENT WITHOUT INSTANTANEOUS EFFECT

  • On the original Temporal Causal3DIdent dataset, iCITRIS achieves almost identical results to CITRIS, supporting its use where effects occur only across time steps.iCITRIS occasionally assigns an instantaneous shape-to-rotation edge that CITRIS incorrectly predicts over time.Source: §G.2.1 · p. 49

G.2.2 ABLATION STUDY 5: PERFECT INTERVENTIONS IN INSTANTANEOUS TEMPORAL CAUSAL3DIDENT

With perfect interventions, iCITRIS-ENCO remains strongest in the instantaneous Temporal Causal3DIdent ablation; on Causal Pinball, iCITRIS and CITRIS both work well.

G.2.2 ABLATION STUDY 5: PERFECT INTERVENTIONS IN INSTANTANEOUS TEMPORAL CAUSAL3DIDENT

  • On original Temporal Causal3DIdent, iCITRIS performs on par with CITRIS despite not assuming an empty instantaneous graph.Source: Table 10 · p. 50
  • With all-perfect-intervention Causal3DIdent data, iCITRIS-ENCO still performs best, and discriminator accuracy indicates accurate modeling of the true causal distribution.The comparison also shows clear discriminator differences from iVAE and iVAE-AR.Source: §G.2.2 · pp. 49–50, Table 11 · p. 50
  • iCITRIS-ENCO misses only sparse hue_b-to-hue_o edges and adds an object-shape-to-rotation edge, while CITRIS misorients edges and VAE graphs poorly match truth.Source: §G.2.2 · p. 50
  • Removing the mutual-information estimator increases latent–factor correlations and worsens estimates of both instantaneous and temporal graphs.Source: §G.2.2 · p. 50

G.3 CAUSAL PINBALL

  • On Causal Pinball, triplet evaluation shows that iCITRIS and CITRIS both work well.The full results are reported over three seeds.Source: §G.3 · p. 50

G.3.1 ABLATION STUDY 6: PERFECT INTERVENTIONS IN CAUSAL PINBALL

With perfect interventions in Causal Pinball, iCITRIS-ENCO identifies the causal variables well and the instantaneous graph with minor errors. Its instantaneous SHD remains better than CITRIS's, while paddle–ball relations are difficult across models.

G.3.1 ABLATION STUDY 6: PERFECT INTERVENTIONS IN CAUSAL PINBALL

  • Perfect interventions replace partially-perfect interventions in this Causal Pinball ablation, allowing use of the mutual-information estimator.Source: §G.3.1 · p. 50
  • iCITRIS-ENCO identifies the causal variables well and the instantaneous graph with minor errors; its instantaneous SHD remains better than CITRIS's.CITRIS also obtains a good correlation score, likely because instantaneous effects are sparse and perfect interventions favor independent variables.Source: §G.3.1 · p. 51, Table 13 · p. 52
  • iCITRIS-ENCO recovers the graph perfectly for one seed, but misorients a ball–paddle edge in the other two; iCITRIS-NOTEARS also often misorients these edges.Source: Figure 19 · p. 52
  • Paddle–ball relations, especially orientation, are difficult because ball position predicts hits and states with same-step paddle hits and ball interventions are rare.All models encounter this problem, though iCITRIS handles it.Source: §G.3.1 · p. 52

Related papers

Loading 2206.06169v2…