PaperlayerView exact revision on arXiv

Source-linked AI summary

Synthetic Data -- Anonymisation Groundhog Day

Theresa Stadler, Bristena Oprisanu, Carmela Troncoso

arXiv:2011.07018v6cs.LGcs.CR

TL;DR

Synthetic data is presented as a way to preserve utility while protecting privacy, but evidence for a better tradeoff than traditional anonymisation is lacking. This paper quantitatively evaluates privacy gain across generative models and finds inference vulnerabilities, utility costs, and unpredictable protection.

  • Problem

    Synthetic data is advertised as preserving statistical properties while protecting individuals against privacy attacks, motivating quantitative scrutiny of its privacy–utility claims.

  • Method

    The paper evaluates synthetic-data publishing with a framework that quantitatively measures adversarial privacy risk and compares synthetic data with traditional anonymisation techniques.

  • Results

    Across the evaluated models, synthetic data either remains vulnerable to inference attacks or incurs significant utility loss, without demonstrating a better privacy–utility tradeoff than traditional anonymisation.

  • Takeaways & Limitations

    Synthetic data is not a reliable privacy-preserving solution because its privacy gain and utility loss are difficult to predict from the dataset and training algorithm alone.

  • Takeaways & Limitations

    In practice, data holders may lack suitable disjoint or public data for metadata, forcing weaker privacy guarantees or larger utility losses when constructing synthetic data.

Abstract

from arXiv · show

Synthetic data has been advertised as a silver-bullet solution to privacy-preserving data publishing that addresses the shortcomings of traditional anonymisation techniques. The promise is that synthetic data drawn from generative models preserves the statistical properties of the original dataset but, at the same time, provides perfect protection against privacy attacks. In this work, we present the first quantitative evaluation of the privacy gain of synthetic data publishing and compare it to that of previous anonymisation techniques. Our evaluation of a wide range of state-of-the-art generative models demonstrates that synthetic data either does not prevent inference attacks or does not retain data utility. In other words, we empirically show that synthetic data does not provide a better tradeoff between privacy and utility than traditional anonymisation techniques. Furthermore, in contrast to traditional anonymisation, the privacy-utility tradeoff of synthetic data publishing is hard to predict. Because it is impossible to predict what signals a synthetic dataset will preserve and what information will be lost, synthetic data leads to a highly variable privacy gain and unpredictable utility loss. In summary, we find that synthetic data is far from the holy grail of privacy-preserving data publishing.

1 Introduction

Synthetic data is promoted as a privacy-preserving alternative to traditional anonymisation, but this paper evaluates whether it actually improves the privacy–utility tradeoff. The authors report inference vulnerabilities, utility costs, implementation flaws, and unpredictable tradeoffs.

  • High-dimensional, sparse microdata remain difficult to publish because information that supports statistical analysis can also enable privacy attacks.
  • Synthetic data is promoted as preserving original statistical properties while containing no personal data and protecting personally identifiable information.
  • The paper introduces a quantitative framework and open-source library for comparing privacy gain and utility across synthetic and traditional anonymisation mechanisms.
  • Synthetic data without explicit privacy protection leaves outlier records vulnerable to linkage attacks and enables high-confidence record-presence inference.
  • Differentially private synthetic data protects targets from inference attacks but incurs significant utility costs and offers little transparency about the resulting tradeoff.
  • Existing implementations of two differentially private training algorithms can violate formal privacy guarantees and leave records vulnerable to inference attacks.

2 Synthetic data and generative models

The paper formalises synthetic data generation as learning an approximation of a raw dataset’s joint distribution and sampling new records from the trained model. Because the approximation is lower-dimensional, it preserves some characteristics while losing others unpredictably.

  • A raw dataset R consists of n records sampled independently from an unknown population distribution DR.
  • A generative training algorithm takes raw data R, learns a representation Dg(R) of its joint multivariate distribution, and outputs a stochastic model that samples synthetic datasets.
  • Generative models provide lower-dimensional approximations of the true data distribution and therefore cannot preserve every statistical characteristic, pattern, or correlation.
  • The model’s selected features determine which statistical properties are replicated, while other characteristics may also be reproduced unintentionally.
  • The evaluation covers five existing training algorithms, including three without explicit privacy protection and two with differential privacy, spanning multiple architectures for tabular data.

3 Quantifying the privacy gain of synthetic data publishing

The paper develops a quantitative framework to test whether synthetic data reduces linkability and inference risks relative to raw or traditionally sanitised data. It evaluates privacy gain at both population and record levels using black-box adversaries.

  • Synthetic data is promoted as an anonymisation solution that addresses shortcomings of generalisation and perturbation while preserving useful statistical information.
  • The paper addresses the lack of evidence that synthetic data protects against linkability and inference attacks at lower utility cost than traditional sanitisation.
  • The framework supports black-box attacks on tabular data and direct comparison of synthetic-data leakage with traditional anonymisation techniques.
  • The framework models each privacy concern as an adversary inferring a secret about a target record and measures advantage from including that record in published data.
  • Privacy gain is defined by the reduction in adversarial advantage when the adversary receives synthetic data instead of raw data.
  • A high privacy gain indicates substantially reduced modelled privacy risk, whereas a low gain means publishing synthetic rather than raw data leaves the adversary’s advantage unchanged.
  • Privacy risks are assessed both across the population and per record, exposing disparate protection for outliers and minority members.

4 Does synthetic data mitigate the risk of linkability?

The paper evaluates whether synthetic data protects against linkability attacks using a membership-inference framework. Across datasets, targets, models, and attack strategies, protection is uneven, and some outliers remain highly vulnerable.

  • 4.1 Formalizing linkability as membership inference: The linkability game tests whether an adversary can infer a target record’s presence in the sensitive dataset from raw or synthetic data and prior knowledge.The challenger publishes either the raw dataset or a synthetic dataset generated from it, while the adversary guesses the target’s membership.
  • 4.3 Empirical evaluation: The evaluation compares expected privacy gain for random records and manually selected outliers across Adult and Texas datasets.The experiments use five randomly chosen targets and five outliers selected for rare categorical values or numerical values beyond the 95% quantile.
  • 4.3 Empirical evaluation: Synthetic data does not consistently protect outliers: one tested target received privacy gain below PG < 0.005 from CTGAN-generated data.The results contradict claims that replacing raw data with synthetic data reliably protects outlier targets from linkage attacks.
  • 4.3 Empirical evaluation: Privacy gain varies with generative model and attacker feature set, with the same targets receiving different protection levels under different attacks.For Texas outliers, CTGAN with FNaive produced PG < 0.3 for three of five targets, whereas IndHist reached a maximum PG = 0.77 for the same group.
  • 4.3 Empirical evaluation: The minimum privacy gain is difficult to predict because adversaries may target different features and synthetic models may preserve unexpected characteristics.The paper states that prediction would require knowing both the attacker’s target features and which features the synthetic data preserves.
  • 4.3 Empirical evaluation: Synthetic data does not provide uniform protection against strategic adversaries, leaving some outliers highly vulnerable to linkage attacks.The paper notes that assessments based on aggregate population measurements underestimate these risks.

5 Does differentially private synthetic data mitigate the risk of linkability?

The paper tests whether differentially private generative models improve protection against linkage attacks. The original implementations still leak information, whereas patched implementations generally bring privacy gain closer to the formal guarantee, subject to remaining uncertainty and practical metadata constraints.

  • 5 Does differentially private synthetic data mitigate the risk of linkability?: The evaluation extends the analysis to PrivBay and PATEGAN, two generative-model training algorithms designed to protect their training data with differential privacy.PrivBay uses a differentially private Bayesian network, while PATEGAN is a differentially private GAN.
  • 5.1 Empirical evaluation: With ε = 0.1, neither original implementation reliably prevents linkage attacks, and two of five Texas outliers achieve PG < 0.1.The low gain violates the theoretical lower bound on privacy provided by the models’ formal guarantees.
  • 5.1 Empirical evaluation: The implementations violate differential-privacy assumptions by learning required metadata directly from the raw training data.This metadata includes numerical ranges and possible categorical values, allowing rare attributes or range shifts to reveal target presence.
  • 5.1 Empirical evaluation: After patching the implementations to obtain metadata independently, privacy gain for most outliers was bounded by the differential-privacy guarantee.Remaining gaps may reflect undiscovered implementation issues or correlations between metadata sources and training data.
  • 5.1 Empirical evaluation: Achieving the intended protection requires both theoretically sound models and implementations and operational environments that preserve the theory’s assumptions.Otherwise, the evaluation provides no guarantee that outliers are protected from linkage attacks.
  • 5.1 Empirical evaluation: In practice, suitable independent or public metadata may be unavailable, forcing data holders to risk weaker guarantees or substantial utility loss.The paper identifies this as a practical boundary for applying differentially private synthetic data to small sensitive datasets.

6 Does synthetic data improve the privacy-utility tradeoff of sanitisation?

Synthetic data does not reliably improve sanitisation’s privacy-utility tradeoff: stronger privacy can coincide with substantial, unpredictable utility loss, while useful signals may be suppressed.

  • Privacy gain: Synthetic data improves protection against linkage for some outliers, but privacy gain varies substantially across records and can be lower than traditional sanitisation.BayNet raises minimum gain for vulnerable targets, while one target loses protection relative to sanitisation; differential privacy increases minimum gain as ε decreases.
  • Privacy gain: Low privacy gain can mean either that anonymised data retains target-specific information or that publishing the raw data already caused little privacy loss.In both cases, synthetic or sanitised publishing provides no improvement over publishing the raw data.
  • Utility loss: For RiskMortality prediction with n = 1000, raw-data accuracy is 72.2%, sanitised-data accuracy is 70.5%, BayNet accuracy is 68.0%, and PrivBay accuracy is 62.0% at ε = 10.The PrivBay result remains 10 percentage points below raw-data accuracy despite the relatively weak privacy setting.
  • Utility loss: Synthetic datasets can substantially distort raw-data statistics, with deviations increasing as differential-privacy ε decreases.TotalChargesAccomm means range from 9K–11K in raw data but 218K–232K for BayNet synthetic data; PrivBay overestimates RiskMortality category 0 by 3 to 15.3 percentage points.
  • Privacy-utility tradeoff: Synthetic data can suppress outlier signals, reducing utility for similar test records and potentially preventing models from learning patterns about protected groups.Differential privacy provides more robust protection but may increase bias and unfairness in data-driven decisions.
  • Privacy-utility tradeoff: Higher privacy gain generally comes with higher utility loss, and metadata choices can trade utility improvements for reduced linkage protection.Raw-data-derived metadata yields PG ≤0.35 for three of five targets, whereas independently derived metadata can produce high privacy gain but insufficient utility.

7 Key takeaways

Synthetic data does not resolve the privacy–utility tradeoff in high-dimensional data sharing and may make that tradeoff unpredictable. The paper finds that preserving useful individual-level signals can also preserve sensitive information.

  • Synthetic data does not magically resolve the basic privacy–utility tension in high-dimensional data sharing.
  • Preserving original-data characteristics with high accuracy can simultaneously enable adversaries to extract sensitive information about individuals.
  • Privacy gain from synthetic data is highly unpredictable because generative models may preserve different data features across datasets.
  • Unlike deterministic sanitisation, synthetic data cannot transparently indicate which information will be omitted or retained.
  • Synthetic data does not provide a better privacy–utility tradeoff than traditional row-level sanitisation, especially for outlier analysis.The paper highlights applications such as financial fraud and medical anomaly detection.

8 Appendix

The appendix documents notation, datasets, library components, privacy attacks, feature sets, and experimental procedures used to evaluate synthetic-data privacy and utility.

  • Notation: Table 1 summarises the paper’s frequently used notation for adversaries, target records, and membership secrets.
  • Framework: The evaluation framework is implemented as a Python library with generative-model and privacy-attack classes.Generative models fit raw data and sample synthetic datasets; privacy attacks train on target records and produce guesses from published data.
  • Models: Five existing generative models were integrated, with model hyperparameters listed for each model and dataset.
  • Feature sets: FeatureSet objects convert synthetic datasets into vectors using naive statistics, histograms, or pairwise correlations.The naive set uses numerical summaries and categorical-frequency information; histogram and correlation sets use configurable preprocessing.
  • Datasets: The experiments use two tabular datasets covering financial and health data, including the Adult dataset with 45,222 individuals and 15 attributes.
  • Attribute inference: Attribute inference is formalised as predicting an unknown sensitive attribute from known attributes using raw, sanitised, or synthetic data.For continuous targets, the attack uses linear regression over a feature matrix and corresponding sensitive values.
  • Outlier evaluation: Figure 15 reports privacy gain for five Texas outlier targets across varying dataset sizes.
Loading 2011.07018v6…