PaperlayerView exact revision on arXiv

Source-linked AI summary

DECAF: Generating Fair Synthetic Data Using Causally-Aware Generative Networks

Boris van Breugel, Trent Kyono, Jeroen Berrevoets, Mihaela van der Schaar

arXiv:2110.12884v2cs.LGstat.ML

TL;DR

Fair synthetic data is difficult to generate from biased data while preserving the underlying data-generating process and downstream utility. DECAF addresses this with a causal GAN that supports inference-time edge removal, and the paper reports fair, useful generation with theoretical guarantees, subject to causal-graph and fairness-definition limits.

  • Problem

    DECAF addresses the challenge of generating fair synthetic data from biased data while preserving utility and ensuring downstream fairness on real-data distributions.

  • Method

    DECAF uses a causal GAN whose variable-specific generators learn causal conditionals and whose graph can be modified at inference time to remove biased edges.

  • Results

    DECAF is compatible with several fairness definitions, maintains high downstream utility, and provides theoretical guarantees for generator convergence and downstream fairness.

  • Takeaways & Limitations

    Publishing debiased synthetic data can shift responsibility for satisfying fairness requirements from downstream users to data-generating entities.

  • Takeaways & Limitations

    DECAF is not compatible with separation- or sufficiency-based fairness definitions and may require correct causal directionality for some fairness criteria.

Abstract

from arXiv · show

Machine learning models have been criticized for reflecting unfair biases in the training data. Instead of solving for this by introducing fair learning algorithms directly, we focus on generating fair synthetic data, such that any downstream learner is fair. Generating fair synthetic data from unfair data - while remaining truthful to the underlying data-generating process (DGP) - is non-trivial. In this paper, we introduce DECAF: a GAN-based fair synthetic data generator for tabular data. With DECAF we embed the DGP explicitly as a structural causal model in the input layers of the generator, allowing each variable to be reconstructed conditioned on its causal parents. This procedure enables inference time debiasing, where biased edges can be strategically removed for satisfying user-defined fairness requirements. The DECAF framework is versatile and compatible with several popular definitions of fairness. In our experiments, we show that DECAF successfully removes undesired bias and - in contrast to existing methods - is capable of generating high-quality synthetic data. Furthermore, we provide theoretical guarantees on the generator's convergence and the fairness of downstream models.

1 Introduction

The paper addresses the underexplored problem of generating fair synthetic data from biased data while preserving utility and supporting fairness in downstream models. DECAF uses causal structure within a GAN to enable flexible fairness interventions and reports theoretical and empirical support.

  • Motivation: Synthetic data fairness is less studied than fidelity, diversity, and privacy in generative modeling.Fidelity concerns resemblance to real samples, diversity concerns distributional coverage, and privacy concerns avoiding reproduced samples.
  • Motivation: Debiasing data before public release assigns responsibility for fairness to data-generating entities and can protect downstream users who may unknowingly include bias.The paper motivates this approach because end-users may lack the expertise to remove bias from their models.
  • Problem: DECAF targets synthetic data that removes bias from a dataset while minimizing utility loss and supporting fair predictions on real-life evaluation data.The downstream model must be fair both on synthetic data and on real-life datasets.
  • Method: DECAF is a causal GAN with one generator per variable that learns causal conditionals and synthesizes variables sequentially in topological order.The approach uses causal structure to support targeted interventions during generation.
  • Contributions: The framework supports flexible fairness modifications, theoretical guarantees for downstream fairness, and compatibility with several fairness definitions while maintaining high downstream utility.The contributions include DECAF, causal modifications for fair generation, and guarantees on downstream predictions.

2 Related Works

Prior synthetic-data methods often lack causal fairness mechanisms, post-hoc distribution changes, or broad flexibility. The paper positions DECAF as combining these capabilities with direct concern for downstream-model fairness.

  • Non-parametric generative modeling: Standard VAE- and GAN-based generators produce realistic data but generally lack causal structure and targeted fairness interventions.The related-work discussion notes exceptions but characterizes non-causal generators as unable to alter distributions specifically for fairness.
  • Fair data generation: Existing fair-data methods often do not support post-hoc distribution changes, although different settings may require different fairness perspectives and protected variables.The paper also distinguishes methods by causal fairness, inference-time fairness, flexibility, and assumptions.
  • Fair data generation: Only some prior methods provide causal fairness, flexible frameworks, or multiple protected attributes, while DECAF combines the listed key areas of interest.The comparison specifically contrasts DECAF with methods that are binary, discrete, less flexible, or not directly focused on downstream fairness.

3 Preliminaries

The preliminaries model tabular data with a causally sufficient DAG and a structural equation for each feature. Each feature is generated from its causal parents and an independent noise variable.

  • Structural causal model: The data-generating process is represented by a causally sufficient directed acyclic graph and a structural equation model.Each feature has a set of causal parents among the other features.
  • Structural causal model: Each feature Xi is generated as a deterministic function of its causal parents and a noise variable Zi.The noise variables carry the randomness of the conditional distributions P(Xi|Pa(Xi)).
  • Structural causal model: The noise variables are mutually independent and have no parents in the causal graph.This isolates the stochastic component of each conditional mechanism in its corresponding noise variable.

4 Fairness of Synthetic Data

The paper defines fairness for synthetic distributions by evaluating optimal predictors trained on synthetic data against a potentially different real-data distribution. Causal graph conditions then identify edge removals that can guarantee CF, FTU, or DP fairness.

  • Distributional fairness: Fair synthetic data is defined by whether an optimal predictor trained on P′(X) satisfies the chosen fairness criterion when evaluated on P(X).The evaluation distribution P(X) need not equal the synthetic training distribution P′(X).
  • Distributional fairness: Using P′(X)=P(X) can be uninformative because random sampling or removing the protected attribute may appear fair despite correlations in real deployment data.The paper emphasizes that downstream users may apply models to real-world data where other variables remain correlated with the protected attribute.
  • Graphical perspective: If every variable in Y’s Markov boundary is d-separated from A given R in G, then P′(X) is conditionally fair with respect to P(X).This graphical condition connects the synthetic graph G′, evaluation graph G, protected attributes, and explanatory features.
  • Graphical perspective: CF fairness is achieved by removing edges between Y and any Markov-boundary feature B that is not conditionally d-separated from A given R.The specified removable edges include B → Y and Y → B.
  • Graphical perspective: FTU debiasing removes the protected–target edge and protected- or target-to-shared-child edges, while DP removes edges between Y and Markov-boundary variables connected to A.These are the paper’s specialized corollaries for R = X\A and R = ∅.
  • Graphical perspective: Under faithfulness, the graphical condition becomes necessary and sufficient for CF, FTU, and DP fairness, identifying the required edge removals.The theorem states that the corresponding corollaries characterize necessary and sufficient edge sets.

5 Method: DECAF

DECAF learns causal conditionals with separate generators and intervenes on parental information at inference time to remove biased edges. This supports multiple fairness definitions without retraining and preserves theoretical convergence under stated assumptions.

  • Architecture: DECAF separates training from inference: it learns causal conditionals, then intervenes on them to generate fair synthetic data.The underlying graph must be known or approximated through causal discovery.
  • Architecture: Each variable has a generator that conditions on its causal parents, with variables generated sequentially in topological order.Root nodes are generated first, followed by descendants using already generated parents and independent noise.
  • Guarantees: The sequential generator has standard GAN-like convergence guarantees when the data distribution is compatible with the DAG and the networks have sufficient capacity.The stated theorem also assumes discriminator optimization conditions; finite data and limited generator capacity weaken practical guarantees.
  • Assumptions: Causal compatibility is weaker than perfect causal knowledge, but some fairness definitions still require correct edge directionality.The framework can use a graph from a Markov equivalence class for generation, while debiasing may need more precise directions.
  • Inference-time debiasing: Removing graph edges implements surrogate do-operations that replace removed dependencies while leaving the protected variable itself unchanged in the synthetic sample.Surrogate assignments can be sampled, fixed, or optimized, depending on the fairness task.
  • Flexibility: Because debiasing occurs at inference time, DECAF can produce multiple fairness-specific datasets without retraining the learned generator.Different datasets can target different fairness definitions or downstream prediction targets.

6 Experiments

Experiments evaluate DECAF on real and synthetically biased data using utility, downstream prediction, and fairness metrics. Across bias strengths, DECAF removes bias while retaining stronger data quality and AUROC than the fairness benchmark.

  • Experimental design: The experiments use real bias and synthetically injected bias to assess debiasing and, where available, fidelity to an unbiased target distribution.The authors also evaluate performance when the underlying DAG is unknown and discovered from data.
  • Metrics: Evaluation combines precision, recall, downstream AUROC, FTU, and demographic parity.FTU measures direct protected-attribute influence on predictions, while DP uses total variation.
  • Adult bias removal: On Adult, DECAF-FTU, DECAF-CF, and DECAF-DP trade lower utility against DECAF-ND because debiasing distorts the original biased distribution.The experiment trains separate downstream MLPs on synthetic datasets and averages results over 10 repetitions.
  • Synthetic bias injection: DECAF removes injected bias and yields higher data utility than benchmark methods trained on the biased distribution.This setting evaluates whether models trained on synthetic data perform well on the real-world distribution rather than the biased training distribution.
  • Fair credit approval: DECAF-FTU and DECAF-DP match FairGAN on debiasing while achieving significantly better precision, recall, and AUROC.DECAF-DP is among the strongest methods across all five metrics and performs better on DP at higher bias strengths.

7 Discussion

DECAF’s fairness guarantees depend on the fairness definition and causal structure supplied, with important boundaries for definitions, graph knowledge, confounding, and data domains.

  • Definitions: DECAF supports FTU, DP, and other independence-based fairness definitions, but not separation- or sufficiency-based definitions such as Equality of Opportunity.Its debiasing mechanism removes feature edges, while separation and sufficiency depend more explicitly on the downstream model.
  • Causal structure: The method relies on a supplied or discovered DAG to order feature generation and determine which edges to remove for fairness.A graph compatible with the data may suffice, although some fairness definitions require correct edge directionality.
  • Causal sufficiency: For counterfactual bias, directionality is essential and correcting hidden confounders is not generally possible.The framework focuses on causally sufficient directed graphs, though other graph types may be usable when they support a valid factorization.
  • Scope: DECAF is developed for tabular data, with possible extension to time-series but non-trivial application to images when protected attributes are not single observed features.The paper suggests disentangled latent-space graphs as one future direction for image data.
  • Social implications: Fairness is task- and context-dependent, so human judgment and public debate remain necessary when selecting fairness requirements.The paper frames DECAF as enabling data issuers to take responsibility for downstream model fairness.

A.2 Experiment

The Adult experiment compares bias removal methods with protected-attribute removal variants, showing that naive removal can trade away data quality while only directly ensuring FTU fairness.

  • Protected removal: Protected-attribute removal variants ensure FTU fairness, but WGAN-GP-PR and DECAF-PR reduce precision, recall, and AUROC.The experiment denotes protected-attribute removal with the *-PR suffix.
  • Protected removal: GAN-PR shows a slight data-quality improvement over GAN, but the improvement is minimal compared with DECAF.The comparison is reported across the protected-removal experiment’s data-quality evaluation.
  • Experiment design: Table 3 reports the Adult bias-removal experiment, including protected-removal metrics; *-PR methods omit the protected attribute before synthesis.Their FTU values are zero because the protected attribute is removed from the generation method.

B Convergence guarantees DECAF GAN

DECAF inherits GAN convergence guarantees under stated assumptions about the causal DAG, model capacity, and optimization. Its generator can represent and converge to the original data distribution, although finite data and limited capacity weaken practical guarantees.

  • Under a fixed optimal discriminator, the generator loss reaches its global minimum if and only if PG = PX.
  • The convergence theorem assumes Markov compatibility with a known DAG, sufficient generator and discriminator capacity, and optimal discriminator updates.
  • Given these assumptions, the generator distribution PG converges to the true data distribution PX.
  • The proof follows from DECAF’s causal generator construction and the capacity of its component generators to express the full distribution.
  • The graph need not equal the true DAG if PG disentangles according to the graph’s causal factors; any graph in the true DAG’s Markov equivalence class can suffice.
  • In practice, finite data and limited generator capacity mean convergence to the true distribution is not guaranteed.

C Compatibility different fairness definitions

DECAF accommodates multiple fairness definitions by translating their requirements into causal or independence constraints and corresponding edge removals. However, definitions based on final predictions and continuous path-specific thresholds are outside or beyond its current scope.

  • DECAF supports independence-based definitions such as FTU, DP, and CF, as well as definitions that prohibit directed paths from protected attributes to outcomes.
  • Equality of opportunity and calibration are incompatible with fair synthetic-data generation because they depend on the final prediction.
  • The framework uses binary fairness judgments, while continuous thresholds such as the 80% rule require path-specific effect quantification and remain limited by linearity assumptions.
  • DECAF is compatible with the fairness definitions summarized in Table 4, which specifies their required edge removals under the original evaluation distribution.
  • Different fairness definitions require different edge removals, and the strictness of CF, ¬UD, and ¬PD depends on explanatory variables or proxies.

D.1 Implementation details.

The experiments instantiate DECAF with shared-layer subnetworks, standard GAN training, and benchmark-specific causal graphs. On Credit Approval, the method removes bias while retaining comparable synthetic-data metrics and high downstream AUROC.

  • Implementation: DECAF uses d subnetworks with shared hidden layers; generator and discriminator networks each have two hidden layers with 2d neurons.
  • Implementation: Training uses standardized continuous variables, Adam with learning rate 0.001, up to 50 epochs, and one generator update per 10 discriminator updates.
  • Implementation: Runtime scales linearly with the number of variables, with the larger Communities and Crime dataset taking about 35 seconds per epoch on the reported CPU.
  • Credit Approval: For Credit Approval, DECAF supports FTU by removing the sex-to-income edge and DP by removing target-incoming edges whose protected attribute is an ancestor.
  • Credit Approval: The Credit Approval experiment injects bias by probabilistically denying approval to samples with ethnicity value 413.
  • Results: Across 10 runs, DECAF generated examples with the highest AUROC, while synthetic-data metrics showed no significant differences among the top methods.

E Surrogate variables

DECAF removes bias by replacing removed causal influences with surrogate variables, preserving fairness across mechanisms while making utility sensitive to how surrogates are chosen. Its robustness varies with causal-graph perturbations and hidden confounding.

  • Surrogate mechanisms: DECAF requires surrogate variables to replace the influence of each removed edge after debiasing.
  • Surrogate mechanisms: Mechanism 1 samples a surrogate from the parent’s marginal distribution, whereas Mechanism 2 assigns a fixed surrogate value.
  • Surrogate mechanisms: Both mechanisms remove injected bias, but Mechanism 1 can randomly disadvantage individuals and reduce quality for the discriminated group.
  • Surrogate mechanisms: Prior knowledge about the bias can improve utility by guiding surrogate selection, while fairness remains guaranteed across the tested mechanisms.
  • Imperfect causal knowledge: Increasing edge removals degrades precision, recall, and AUROC, whereas edge addition and reversal preserve stable synthetic-data quality.
  • Imperfect causal knowledge: DECAF-FTU and DECAF-ND continue to debias consistently across the tested DAG perturbations.
  • Hidden confounding: Under hidden confounding, DECAF-FTU and DECAF-DP retain better data quality and downstream AUROC than FairGAN at comparable DP and FTU.
Loading 2110.12884v2…