PaperlayerView exact revision on arXiv

Source-linked AI summary

Neighborhood Watch: Privacy Risks in Seeded Local Combination Synthetic Data

Hadrien Lautraite, Tristan Allard, Anne-Sophie Charest, Jean-François Rajotte, Sébastien Gambs

arXiv:2608.27037v1cs.CR

TL;DR

Synthetic data may enable sharing sensitive healthcare data, but privacy attacks expose residual risks in methods without formal guarantees. This paper evaluates three Seeded Local Combination methods using membership inference, linkage, and reconstruction attacks, finding substantial leakage and questioning whether their outputs are anonymous. It also argues that Simulant’s claimed differential privacy guarantee does not account for all pipeline steps and uses unsuitable noise calibration.

  • Problem

    Synthetic data sharing has residual privacy risks, but the privacy guarantees of Seeded Local Combination methods have received limited attack-based evaluation.

  • Method

    The paper empirically evaluates SMOTE, Simulant, and Avatar using enhanced membership inference, SLC-adapted linkage, and reconstruction attacks.

  • Results

    All three studied SLC methods show substantial privacy leakage, with attacks often achieving success rates an order of magnitude higher than random guessing.

  • Takeaways & Limitations

    The findings raise serious doubts about whether synthetic data produced by these SLC techniques should be considered anonymous in practice.

  • Takeaways & Limitations

    Simulant’s claimed differential privacy guarantee does not cover seed selection and neighbor computation, and its multiplicative Gaussian noise is generally unsuitable for differential privacy.

Abstract

from arXiv · show

Synthetic data is seen as a promising solution for sharing data in sensitive contexts. However, recent work on privacy attacks have shown that there are still significant residual risks, especially for synthetic data generations methods that are not based on formal approaches such as differential privacy. In this paper, we investigate the privacy risks associated with local combination approaches for generating synthetic data in which synthetic profiles are built by combining real neighbouring profiles. More precisely, we focus on three methods from this family, namely SMOTE, Simulant and Avatar, which have been recently used as a way to share 'anonymised data' in the healthcare domain. In particular, we conduct an extensive privacy analysis through a diverse set of attacks: membership inference, linkage and reconstruction attacks. Our results demonstrate substantial privacy leakage for all three methods, raising serious doubts about whether their outputs should be regarded as anonymous in practice.

1 Introduction

The paper examines privacy risks in Seeded Local Combination methods, which generate synthetic profiles by combining real neighboring profiles. Across attacks targeting these methods, the study finds substantial privacy leakage and questions whether their outputs are anonymous.

  • Methods studied: The study evaluates SMOTE, Avatar, and Simulant, methods used or proposed for synthetic data generation in healthcare.SMOTE is used for healthcare data augmentation and synthetic generation; Avatar and Simulant have also been proposed for healthcare applications.
  • Privacy findings: All studied SLC methods are highly vulnerable to membership inference and linkage attacks, while a reconstruction attack can recover a significant part of the WBCD dataset.The findings raise doubts about treating data produced by these techniques as anonymous.
  • Privacy attacks: The enhanced ensemble membership inference attack achieves TPR@10%FPR significantly better than random guessing and consistently outperforms single baseline attacks.The attack combines multiple scores with original features and uses a Gradient Boosting model trained with shadow modeling.
  • Privacy attacks: The novel linkage attack models the synthetic distribution generated from each real profile and uses likelihood to infer a synthetic profile’s source.This differs from approaches based only on distances between real and synthetic profiles.
  • Privacy attacks: The extended ReconSMOTE attack targets the non-oversampling setting and is effective against Avatar-generated data with privacy parameter k = 2.This parameter setting was used in practice for releasing medical data.
  • Seeded Local Combination: Seeded Local Combination methods select a seed profile, identify neighboring real profiles, and generate a synthetic profile through a noisy combination.The framework combines synthetic-data generation with profile modification while retaining a relationship to the selected seed.

9 end

The paper describes how Avatar and Simulant construct synthetic profiles and reviews privacy metrics and attacks relevant to SLC methods. It emphasizes that distance-based post-hoc metrics can understate privacy risks compared with direct attacks.

  • SLC methods: Avatar searches for neighbors in a PCA-based latent space and creates a noisy weighted barycenter before projecting it back to the original data space.Neighbor weights depend on seed-to-neighbor distances and additional randomization.
  • SLC methods: Simulant selects feature values from neighbors in the original data space and adds multiplicative Gaussian noise, while correlated features may be sampled together.The paper challenges the claim that this noise addition ensures differential privacy.
  • Privacy metrics: Avatar-related privacy evaluation includes distance-to-closest-record, nearest-neighbor distance ratio, local cloaking, and hidden rate metrics.Local cloaking counts synthetic profiles closer to a seed than its own avatar, while hidden rate measures how often a profile’s closest avatar is not its own.
  • Privacy metrics: Prior work reports that distance-based metrics such as DCR can be poor indicators of true privacy risk because attacks may succeed even when those metrics appear reassuring.The paper therefore distinguishes interpretable post-hoc measures from attack-based evaluation.
  • Privacy attacks: Membership inference attacks guess whether a specific profile belonged to a model’s training set, which can reveal sensitive attributes in the AIDS and WBCD datasets.Membership in AIDS indicates HIV infection, while membership in WBCD indicates a benign or malignant breast tumor.
  • Privacy attacks: ReconSMOTE reconstructs original points from aligned synthetic profiles and achieved 100% precision with an average of 85% recall on eight datasets.The reported attack targeted SMOTE in a minority-class oversampling context.

4 Novel privacy attacks against SLC

The paper evaluates SLC privacy through membership inference, linkage, and reconstruction-oriented settings. Its ensemble MIA combines attack scores with original features and shadow modeling, while linkage is framed as identifying a synthetic profile’s source.

  • Privacy attack settings: The evaluation considers membership inference, reconstruction, and linkability as privacy goals for SLC data.The linkage setting gives the adversary both original and synthetic datasets for benchmarking and alignment with GDPR anonymisation criteria.
  • Membership inference attack: The MIA samples members and non-members, then tests whether an attacker can distinguish profiles used to generate synthetic data from excluded profiles.Synthetic data are generated from a training subset, and the attacker predicts membership for a balanced guessing set.
  • Membership inference attack: The ensemble MIA feeds data-space and latent-space attack scores plus original features into a Gradient Boosting model trained with 50 shadow models.Including original features accounts for attack scores behaving differently across dense regions and distribution edges.
  • Linkage attack: The linkage attack attempts to identify the real profile corresponding to each synthetic profile when the adversary knows the original dataset.This setting is described as unrealistic for many real-world contexts but useful for benchmarking linkability.

3 Shadow modeling (i.e., attack dataset construction).

The linkage attack models how synthetic outputs arise from candidate seed profiles rather than linking each output solely to its nearest real observation. It estimates seed-conditional distributions, scores candidates by likelihood, and compares against random and distance-based baselines.

  • Shadow modeling: The attack simulates synthetic profiles for each original profile and estimates a distribution from those outputs.For Avatar, the distribution uses estimated µ_seed and Σ_seed; for SMOTE and Simulant, it uses a Gaussian KDE.
  • Linkage comparison: A nearest-profile linkage rule can fail because the most likely generating seed may differ from the closest real profile.In the illustrated example, distance favors C3 while the estimated densities favor C2.
  • Seed-conditional modeling: For each candidate seed, the attack models its avatars as s ∼ N(µ_seed, Σ_seed) and computes the target avatar’s log-likelihood.Covariance singularity can be addressed with ridge correction, PCA, or a global covariance estimate.
  • Seed-conditional modeling: The attack uses the top-1 versus top-2 log-likelihood gap to quantify confidence in the predicted source seed.The highest-scoring candidate is compared with the runner-up rather than evaluated only by absolute likelihood.
  • Linkage comparison: The evaluation compares the attack with random guessing, closest-real-profile linkage, and a PCA-latent-space linkage baseline.Random guessing treats success at chance level as consistent with anonymity.

6 end

The experiments use classical datasets after duplicate removal and report practical attack runtimes. The Avatar reconstruction procedure combines colinear triplets and close pairs to produce candidate real points, then filters repeatedly reconstructed points.

  • Reconstruction attack: Repeated reconstruction of a point indicates multiple supporting segments, so the procedure retains points reconstructed most often.This filtering rule is intended to improve precision after relaxing the requirement for three aligned synthetic points.
  • Computational setting: All attacks run in under one hour on a laptop with 16 CPU cores and 32 GB RAM.The paper does not report exact computational times for individual attacks.
  • Experimental setting: The experiments evaluate attacks on three classical datasets, including WBCD and AIDS.WBCD has 449 profiles, nine ordinal categorical columns, and a binary tumor-classification target; AIDS combines categorical and continuous attributes.
  • Experimental setting: Duplicates are removed before evaluation to avoid direct synthetic-real duplicate privacy violations.The Avatar generation retains half the original number of dimensions as principal components, with additional component settings reported in an appendix.
  • Reconstruction attack: The Avatar reconstruction attack with k = 2 takes synthetic data, colinearity and neighborhood thresholds, and a distance function as input.It outputs a set of reconstructed candidate real points.
  • Reconstruction attack: The reconstruction procedure finds colinear triplets, close synthetic pairs, supporting lines, and intersections near previously identified synthetic points.The close-pair and nearby-point tests are controlled by separate thresholds.

19 end

SLC-generated data retains useful predictive performance but exhibits substantial privacy leakage across membership inference, linkage, and reconstruction attacks. Increasing neighborhood size often reduces attack success, yet important vulnerabilities remain, especially for SMOTE and Avatar.

  • Utility: SMOTE slightly outperformed Simulant and Avatar on AIDS and WDBC classification tasks, while Simulant and Avatar utility decreased as k increased.SMOTE did not show the same utility decline as k increased.
  • Post-hoc privacy metrics: Simulant showed the strongest selected post-hoc privacy metrics, whereas SMOTE was highly vulnerable to distance-based linkage, with the closest real profile being its counterpart for half of synthetic data.The hidden rate for Simulant was mostly above 95% except at extremely low k values.
  • Membership inference: At k = 10, ensemble MIA found Simulant less vulnerable than SMOTE and Avatar, but still achieved approximately 20% TPR10%FPR on WBCD and AIDS.SMOTE and Avatar reached TPR10%FPR values nearly five to seven times better than random guessing.
  • Membership inference: The ensemble MIA outperformed individual attacks across all datasets, whose relative effectiveness varied by dataset and generative method.No single individual attack consistently outperformed the others.
  • Membership inference: Increasing k generally reduced MIA success, but SMOTE and Avatar still produced attack rates four to eight times larger than random guessing.The residual leakage remained non-trivial despite larger neighborhoods.
  • Linkage attacks: At k = 10, likelihood-based linkage correctly linked Avatar profiles to seeds in 10–30% of cases and SMOTE profiles in one third to one half, while Simulant remained less vulnerable.For Avatar, the likelihood attack outperformed both baselines; for SMOTE, baselines were especially strong on California Housing.
  • Reconstruction attack: The reconstruction attack recovered one quarter of the WBCD dataset from Avatar outputs at k = 2 with 80% precision after filtering.The baseline ReconSMOTE attack reconstructed 19 WBCD points with 95% precision but none from AIDS or California Housing.

6 Differential privacy and SLC

The study finds substantial privacy risks across the three SLC methods and questions whether their outputs should be considered anonymous. It argues that differential privacy would require accounting for every generation step, not only the final combination.

  • The authors propose modifying SLC methods to satisfy differential privacy as a potential solution to the observed leakage.The paper notes that differential privacy bounds the maximum contribution of one observation to a randomized algorithm's output.
  • Simulant's claimed ε-DP guarantee is disputed because seed selection and neighbor-distance computation access the data without noise.The paper states that all pipeline steps must be included in the privacy analysis, with relevant noise added at each step.
  • Multiplicative Gaussian noise with fixed scale is described as unsuitable for differential privacy, especially when the protected quantity can equal zero.The paper contrasts this with the Gaussian mechanism, whose noise scale is calibrated to sensitivity.
  • All three studied SLC methods are susceptible to multiple privacy attacks, with success rates often an order of magnitude above random guessing.The attacks include ensemble membership inference, SLC-tailored linkage, and reconstruction against Avatar with k = 2.
  • The study focuses on attacks against privacy techniques intended for real, sensitive use cases and reports responsible disclosure for affected products.The authors withhold reconstruction-attack code because vulnerable health-data copies may remain publicly available.

A Attribute Inference Attack

The attribute-inference attack tests whether synthetic data helps infer unknown attributes more accurately than auxiliary data from the same distribution. Results indicate strong vulnerability for SMOTE and Avatar, while Simulant is more resilient.

  • Attack setup: The attack loops over attributes, treating each in turn as the value missing to the adversary.Its output is a predicted value for the unknown feature.
  • Attack setup: Two k-NN classifiers with k = 5 compare inference from synthetic data against inference from an auxiliary dataset.The dataset is split between guessing and auxiliary sets, and the measured quantity is the percentage of attributes where synthetic data performs better.
  • Interpretation: More than 50% of columns favoring synthetic data indicates privacy leakage because synthetic data improves attribute inference over auxiliary data.Table 5 reports this percentage by dataset and SLC method.
  • Results: SMOTE is extremely vulnerable, with synthetic data providing more inference power for almost all features across datasets.Avatar is also vulnerable, though with slightly less inference power.
  • Results: Simulant appears more resilient, with most features better predicted using auxiliary data than synthetic data.

B.2 Data restricted linkage attack results

The data-restricted linkage attack models synthetic-output distributions using only synthetic data and evaluates links for randomly selected targets. Its high-confidence predictions generally outperform baselines, though distribution estimation limits performance.

  • Evaluation: The experiment evaluates linkage independently on 100 randomly selected points, repeating fresh synthetic-data generation five times.Attack success is assessed through link-prediction scores.
  • Attack pipeline: The attack takes one real seed, released synthetic data, an SLC generator, simulations, a candidate-set size, and a distance function as inputs.It returns a link score and predicted synthetic record.
  • Attack pipeline: The method fits a seed-conditional distribution from simulated synthetic outputs and scores candidates by their likelihood under that model.Avatar uses estimated µ_seed and Σ_seed, while SMOTE and Simulant use Gaussian KDE.
  • Attack pipeline: The procedure first builds the seed-conditional synthetic-output distribution, then identifies the linked synthetic record.

14 end

The linkage attack generally exceeds baselines among high-confidence predictions and often beats random guessing by an order of magnitude. Reconstruction also succeeds against SMOTE, while Avatar's neighbor-selection representation affects membership-inference success only for California Housing.

  • Data-restricted linkage results: At k = 10, the likelihood attack outperforms both baselines among the 10% highest-confidence links, except for Simulant on California Housing.Results are often at least an order of magnitude better than random guessing, especially for SMOTE and Avatar on AIDS and California Housing.
  • Data-restricted linkage results: The likelihood attack performs worse in the data-restricted setup because synthetic-only simulations estimate the output distribution less accurately.
  • Reconstruction results: Against SMOTE with k=10, the proposed reconstruction attack reconstructs twice as many WBCD observations as ReconSMOTE after filtering.ReconSMOTE reconstructs 66 observations with 75% precision, while the proposed method also reconstructs observations for AIDS and California Housing.
  • Membership-inference results: For Avatar at k = 10, the number of latent-space principal components does not affect membership-inference success on WBCD or AIDS.
  • Membership-inference results: For California Housing, latent-space dimensionality significantly affects membership-inference success, for reasons the authors leave for future work.
Loading 2608.27037v1…