Source-linked AI summary
Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions
Sara Magliacane, Thijs van Ommen, Tom Claassen, Stephan Bongers, Philip Versteeg, Joris M. Mooij
TL;DR
Domain adaptation must predict accurately when training and test distributions differ. The paper uses causal inference to select invariant feature subsets without knowing the causal graph or interventions, and reports promising simulated-data results.
Problem
Domain adaptation lacks reliable predictions when training and test distributions differ, because ignoring the shift can produce arbitrarily large transfer bias.
Method
The approach selects source-predictive feature subsets whose conditional distribution P(Y | A) remains invariant across source and target domains, using Joint Causal Inference.
Results
The feature-selection strategy can provide an asymptotic prediction-error guarantee, whereas ignoring distribution shifts can yield unbounded prediction error in the worst case.
Takeaways & Limitations
Invariant separating feature subsets enable transferable predictions without requiring the causal graph or intervention targets to be known.
Takeaways & Limitations
The method requires an assumption about intervention targets or the target-domain mechanism, and currently handles only about seven variables on a laptop computer.
Abstract
from arXiv · showhide
An important goal common to domain adaptation and causal inference is to make accurate predictions when the distributions for the source (or training) domain(s) and target (or test) domain(s) differ. In many cases, these different distributions can be modeled as different contexts of a single underlying system, in which each distribution corresponds to a different perturbation of the system, or in causal terms, an intervention. We focus on a class of such causal domain adaptation problems, where data for one or more source domains are given, and the task is to predict the distribution of a certain target variable from measurements of other variables in one or more target domains. We propose an approach for solving these problems that exploits causal inference and does not rely on prior knowledge of the causal graph, the type of interventions or the intervention targets. We demonstrate our approach by evaluating a possible implementation on simulated and real world data.
1 Introduction
The paper addresses prediction under domain shift by exploiting causal structure to transfer knowledge across intervention-related domains. It proposes a causal domain adaptation approach that does not require prior knowledge of the causal graph, intervention types, or intervention targets and can accommodate latent confounders.
- Motivation: Domain adaptation becomes necessary when training and test data come from different distributions because their data-gathering circumstances differ.The paper frames predicting unknown values from observed data as a central statistical and machine-learning problem whose difficulty increases under distribution shift.
- Motivation: A function that predicts well across domains for one variable set may fail for another because domain changes can alter relations among variables.Causal graphs provide a principled way to reason about such changes when domains correspond to external interventions or more general shifts.
- Contributions: Causal structure enables knowledge transfer across domains to address domain adaptation.The paper situates its approach within work using causal structure of the data-generating process to transfer information between domains.
- Related work: Existing causal domain adaptation methods rely on assumptions such as a known causal graph, perfect interventions, and known intervention targets.The cited transportability theory assumes all three conditions, while related work makes different assumptions.
- Contributions: The proposed setting allows the causal graph and intervention types and targets to be partially unknown.The authors formulate relatively weak assumptions that make the problem well-posed and propose an approach that handles latent confounders.
2 Theory
The theory frames domain adaptation as predicting across context-induced distribution shifts, where causal structure motivates selecting feature sets whose conditional relationship with the target remains invariant. It characterizes the tradeoff between eliminating transfer bias and retaining predictive information, while requiring assumptions and methods to identify suitable separating sets and handle covariate shift.
- Motivating example: The motivating example shows that source-only feature selection may prefer {X3} or {X1, X3}, although only X1 supports safe transfer because P(Y | X1) is invariant across domains.Under intervention, P(Y | X3) and P(Y | X1, X3) change, potentially causing extreme target-domain bias.
- Assumptions: The separating-set strategy requires assumptions about causal structure and intervention mechanisms, including invariance of relevant causal structure and sufficient information about intervention targets or mechanisms.Without information about intervention targets or the target-domain mechanism determining Y, prediction is generally impossible.
- Transfer bias: Ignoring source–target distribution differences can produce arbitrarily large transfer bias and arbitrarily bad infinite-sample predictions.This motivates restricting prediction to feature sets satisfying the separating set property.
- Bias tradeoff: A separating feature set yields zero transfer bias, but using only that subset can introduce incomplete information bias by omitting predictive variables.Selecting among separating sets by source-domain risk can reduce this information loss while preserving the transfer guarantee.
- Identification and prediction: Because target-domain Y values are unavailable, separating sets cannot be tested directly from data; when the graph is known, d-separation verifies the property.After selecting a set, prediction methods should account for covariate shift because its distribution may change between source and target domains.
3 Evaluation
The evaluation compares a Random Forest feature-selection baseline with the proposed method on synthetic causal models and an IMPC-based real-world dataset. On the real-world task, the method abstains in difficult cases and performs similarly to the baseline when it predicts.
- Evaluation data: The study evaluates the approach on both synthetic data and hematology-related measurements from mice with single-gene knockouts.The real-world data come from the International Mouse Phenotyping Consortium and a causal inference challenge.
- Methods: Both evaluations compare standard Random Forest feature selection with the proposed method, which selects the best separating set from the Random Forest output.The baseline selects the best-performing subset and predicts Y, whereas the proposed method further searches among the Random Forest output.
- Synthetic evaluation: The synthetic evaluation generates 200 linear acyclic models with latent variables, Gaussian noise, three system variables, and soft interventions across observational and experimental domains.Each model includes N samples per domain, randomly selected intervention targets, and varying perturbation sizes.
- Results: 170 of 1000 cases trigger abstention, while the method performs similarly to the baseline on the remaining cases.The comparison uses the L2 loss between predicted and real Y values; the small datasets make the task challenging.
4 Discussion and Conclusion
The paper defines causal domain adaptation problems and proposes a method for identifying feature sets that yield transferable predictions without requiring prior knowledge of the causal graph or intervention targets.
- Discussion and Conclusion: The proposed method identifies feature sets that lead to transferable predictions under general causal domain adaptation assumptions.It does not require the causal graph or intervention targets to be known.
- Discussion and Conclusion: The method produces promising results on simulated data and can be extended to cyclic settings using results by Forré and Mooij (2018).The passage also notes that more work remains to be done.
A Supplementary material · A.1 Stronger assumption
The supplementary section shows that Assumption 2(ii) follows from standard causal Markov and faithfulness assumptions applied separately to pooled source and target distributions. These stronger assumptions exclude perfect interventions in the target domain, although Assumption 2(ii) permits them.
- A Supplementary material: Using causal Markov and faithfulness separately in source and target domains prevents perfect interventions in the target domain.Assumption 2(ii) is weaker because it otherwise allows perfect target interventions.
- A.1 Stronger assumption: Assumption 2(ii) is implied by a stronger assumption combining Markovness and faithfulness in the pooled source and target domains.This is stated as Proposition 2.
- A.1 Stronger assumption: The pooled source distribution P(V | C1 = 0) must be Markov and faithful to G\C1.G\C1 is the graph after removing C1 and all incident edges.
- A.1 Stronger assumption: The pooled target distribution P(V | C1 = 1) must also be Markov and faithful to G\C1.The same induced subgraph is used for both pooled distributions.
- A.1 Stronger assumption: Here, G\C1 denotes the induced subgraph on V \ {C1}, obtained by removing C1 and every edge involving it.This defines the graph appearing in the stronger assumption.
- A.1 Stronger assumption: For arbitrary A, B, S ⊆ V \ {C1}, the proof applies the stated conditional-independence relation under the stronger assumption.The proof introduces A, B, and S as subsets excluding C1.
- A.1 Stronger assumption: Because the relation holds for both c = 0 and c = 1, it directly establishes Assumption 2(ii).The proof concludes the implication by covering both source and target contexts.
A.2 Other proofs
The proofs establish that conditional independences and JCI assumptions identify {X1} as a valid separating set for predicting X2. They also show why including X3 can cause possibly unbounded target-domain prediction error.
- Proof of Proposition 1: Under the Markov and faithfulness assumption, the conditional-independence argument in Proposition 1 is equivalent to A ⊥B | S ∪{C1} [G].The proof combines dependence under C1 = 0 with independence under both contexts.
- Proof of Example 2: In Example 2, JCI assumptions and the observed independences imply that {X1} is a valid separating set for X2.Across the possible ADMG structures, the proof obtains C1 ⊥X2 | X1 [G].
- Proof of Example 2: A standard feature-selection method may prefer {X1, X3} over {X1}, because {X1, X3} is the Markov blanket of X2 in context [C1 = 0].This preference arises when selecting features using source-domain data.
- Proof of Example 2: As a result, prediction methods trained on all source-domain features may incur possibly unbounded error when predicting X2 in target domain [C1 = 1].The proof gives an almost deterministic copy of X2 by X3 as an example mechanism.
A.3 Additional results on synthetic data
Additional synthetic experiments varied intervention-effect scales and sample sizes across randomly generated acyclic models. The method offered little advantage when domains were similar, but performance improved as intervention effects and sample sizes increased.
- Experimental setup: The experiments used 200 randomly generated acyclic models with three system variables, two context variables, and up to two randomly selected latent variables.Each latent variable had two system-variable children.
- Intervention effects: Scaling all intervention effects by 0.1 produced very similar distributions across domains, where the method offered no advantage.Figure 5 varied the scale of intervention effects.
- Sample size: Performance improved with more samples per regime, especially for the method, because larger samples improved conditional-independence test quality.The baseline also improved as sample size increased.
- Sample size: Accuracy was low with N = 100 samples but improved substantially with N = 1000 samples.The passage identifies these settings as comparisons across sample sizes per regime.