Source-linked AI summary
Private Causal Inference
Matt J. Kusner, Yu Sun, Karthik Sridharan, Kilian Q. Weinberger
TL;DR
Causal inference can involve sensitive personal data, while existing methods lacked formal privacy guarantees. This paper applies differential privacy to ANM quantities and reports that small noise can preserve confident causal decisions. Its experiments demonstrate practical application on real-world datasets.
Problem
Causal inference often uses sensitive personal data, but existing methods lacked formal guarantees protecting individuals.
Source: §1 · p. 1Method
Results
With high probability, privately released quantities do not change the causal inference result when that result is sufficiently confident.
Source: §1 · p. 1Takeaways & Limitations
Experiments on real-world causal-inference datasets demonstrate that the privacy-preserving methods can be applied practically.
Source: §1 · p. 1, §7 · p. 8The stated two-variable setting assumes no confounder and no simultaneous causation between X and Y.
Source: §3 · p. 2
Abstract
from arXiv · showhide
Causal inference deals with identifying which random variables "cause" or control other random variables. Recent advances on the topic of causal inference based on tools from statistical estimation and machine learning have resulted in practical algorithms for causal inference. Causal inference has the potential to have significant impact on medical research, prevention and control of diseases, and identifying factors that impact economic changes to name just a few. However, these promising applications for causal inference are often ones that involve sensitive or personal data of users that need to be kept private (e.g., medical records, personal finances, etc). Therefore, there is a need for the development of causal inference methods that preserve data privacy. We study the problem of inferring causality using the current, popular causal inference framework, the additive noise model (ANM) while simultaneously ensuring privacy of the users. Our framework provides differential privacy guarantees for a variety of ANM variants. We run extensive experiments, and demonstrate that our techniques are practical and easy to implement.
1 Introduction
Causal inference findings often rely on sensitive personal data, yet existing methods lacked formal guarantees protecting individuals. This paper applies differential privacy to additive noise models (ANMs), reporting that small amounts of noise can preserve confident causal decisions.
1 Introduction
- Causal findings may expose personal data, and existing causal inference methods lacked formal guarantees protecting individuals.The cited risks include medical diagnosis, fraud detection, and risk analysis.Source: §1 · p. 1
- Differential privacy is presented as protecting individuals by limiting releases to aggregate information about a dataset.The paper describes reassurance that personal information cannot be recovered with extremely high probability.Source: §1 · p. 1
- The paper is the first, to the authors’ knowledge, to investigate private causal inference.It proposes privately releasing ANM quantities with differential-privacy noise.Source: §1 · p. 1
2 Related Work
The section situates ANMs among causal-inference approaches and contrasts them with conditional-independence testing. It notes that conditional-independence methods can leave causal direction ambiguous.
2 Related Work
- Causal structures are framed as relevant to policy and health-care practice, with smoking and lung cancer offered as an example.The cited example describes their correlation as arising from a direct causal link.Source: §2 · p. 2
- ANMs model an effect as a nonlinear function of its cause plus independent noise, and are a popular alternative to conditional-independence testing.The approach was developed by Hoyer et al. and used in later work [18].Source: §2 · p. 2
- Conditional-independence approaches cannot distinguish Markov-equivalent graphs, and may not determine direction when only two variables are considered.In the two-variable case, there is no conditional-independence test to determine whether X →Y or Y →X.Source: §2 · p. 2
3 Background
The background sets out the two-variable ANM setting and its assumptions, then describes how ANM inference compares residual dependence across directions. It reviews dependence scores and the differential-privacy mechanisms used to release them.
3 Background
- The setting uses i.i.d. samples for two variables and assumes no confounder and no simultaneous causation between them.The target is to protect the samples while inferring whether X causes Y or Y causes X.Source: §3 · p. 2
3.1 Additive Noise Model
- An ANM X →Y represents Y as a nonlinear function of X plus noise independent of X; causal direction is identifiable when the induced joint distributions differ.ANMs are generically identifiable from i.i.d. samples, except for some nonlinear functions and noise distributions.Source: §3.1 · p. 2
3.2 Inferring Causality
- ANM inference trains regressions in both directions, computes test-set residuals, and compares each residual’s dependence on its input [23].The lower dependence score determines the declared direction.Source: §3.2 · pp. 2–3
3.3 Dependence Scores
- Dependence scores include rank-based measures and HSIC; the paper introduces an IQR-based score, while variance can change arbitrarily from one input-value change.IQR is the difference between the third and first quartiles; the variance score is ill suited to differential privacy.Source: §3.3 · p. 3
3.4 Differential Privacy
- Differential privacy uses sensitivity to calibrate Laplace noise, and the paper reviews privatization methods for scores with meaningful global or local sensitivity bounds.The Laplace mechanism adds noise with scale ΔA/ϵ; local sensitivity can be bounded for particular datasets even when global sensitivity is unbounded.Source: §3.4 · pp. 3–4, Table 1 · p. 4
4 Test Set Privacy
Section 4 develops test-set privacy by bounding global sensitivity for rank-correlation and HSIC scores and using a local-sensitivity method for IQR. The guarantees preserve causal direction with high probability when score margins are sufficiently large; IQR recovery also depends on dataset size.
4 Test Set Privacy
- The framework privatizes test-set dependence scores, enabling private direction decisions while also releasing confidence information and leaving score choice to practitioners.Training-set privacy is treated separately, and the two mechanisms can be used jointly.Source: §4 · p. 4
4.1 Rank Correlation Coefficients
- Bounded global sensitivity lets Laplace-noised Spearman’s ρ and Kendall’s τ scores achieve (2ε, 0)-differential privacy.Functions of the private scores, including comparing them to choose a direction, remain private when they do not access the data.Source: §4.1 · p. 4
- With correctly ordered scores and a sufficiently large margin, Laplace noise preserves the causal direction with high probability for rank-correlation methods.Incorrect-inference probability decreases nearly exponentially as margin γ increases, and also decreases as m and ε grow.Source: §4.1 · pp. 4–5
4.2 HSIC Score
- For HSIC, a global-sensitivity bound supports Laplace release that preserves the direction of causal influence with high probability under certain conditions.The sensitivity argument uses bounded kernels and a single-element change in the data.Source: §4.2 · p. 5
4.3 IQR Score
- With released private log-IQR scores and correctly ordered true scores, the method recovers causal direction with high probability, depending on dataset size.IQR lacks bounded global sensitivity; its modified local-sensitivity release is (3ε, δ)-differentially private, and a failed stability check can return null.Source: §4.3 · pp. 5–6
5 Training Set Privacy
Training-set privacy is achieved by bounding how changes to training data affect ANM residuals and dependence scores. The methods use sensitivity bounds or stability to privately release ranking, HSIC, and IQR scores.
5 Training Set Privacy
- For classifiers from eq. (5) trained on (x, y) and (˜x, ˜y), Theorem 5 bounds residuals across the test set, with λ ≤ 1 and test-set size m.The paper uses kernel ridge regression.Source: §5 · p. 6
5.1 Rank Correlation Coefficients
- Ranking scores have global sensitivity 1, so the Laplace mechanism cannot provide useful privacy; a stability-based Propose-Test-Release method instead privatizes them.A lower bound on distance to instability uses the minimum gap between residual ranks, and the method releases the same ranking score while those ranks remain unchanged.Source: §5.1 · pp. 6–7
5.2 HSIC Score
- For kernels bounded by 1 and an L_l-Lipschitz kernel l, the HSIC score has a training-set sensitivity bound, enabling Laplace privacy and a utility guarantee.The theorem specifies m ≥ 2; the supplied passage does not include the sensitivity expression.Source: §5.2 · p. 7
5.3 IQR Score
- Adding noise to IQR scores on residuals yields (6ε, 2δ)-differential privacy, and the authors state that Theorem 4’s utility results can be tightened.The approach adapts Propose-Test-Release by defining A_j through the number of training points needed to move the IQR out of an interval.Source: §5.3 · pp. 7–8
6 Results
On selected Cause-Effect Pairs datasets, privatization can reduce correct causal inference at low ε, while higher ε improves it. For training-set-private HSIC, tuning λ balances sensitivity against regression quality.
6 Results
- At low ε, noise lowers the probability of correct causal inference for HSIC and IQR; that probability rises as ε increases, sometimes drastically.For the IQR dataset in Figure 1, none of 10,000 sampled points crossed the null threshold across the tested δ ∈ [10^-5, 10^-2], so the null probability was essentially 0.Source: §6 · p. 8
- HSIC accuracy has a hump-shaped relationship with λ: small λ yields excessive sensitivity, while large λ over-regularizes; the best λ supports high-quality training-set-private inference.The range of λ that balances sensitivity and regularization grows as ε increases.Source: §6 · p. 8
7 Conclusion
The paper presents what it describes as the first work toward differentially private causal inference. It identifies further work on other causal frameworks, ANM without train/test splits, and additional dependence scores.
7 Conclusion
- The authors present this as the first work toward differentially private causal inference and identify several directions for extending it.Future work includes privatizing frameworks such as IGCI, analyzing ANM without train/test splits, and studying other dependence scores.Source: §7 · p. 8