Source-linked AI summary
Membership Inference Attack on Private Evolution
Daniil Filienko, Sikha Pentyala, Martine De Cock
TL;DR
Private Evolution enables inference-only synthetic data generation under differential privacy, but practical privacy still needs empirical auditing for susceptibility to membership inference. This paper introduces EVO-MIA, which uses per-round voting evidence to test whether a target instance participated in PE. Its results indicate membership leakage across modalities, with performance varying by dataset and strong text results even when utility is low.
Problem
Empirical evidence is needed to audit whether PE’s released synthetic data reveals membership, complementing its theoretical privacy guarantees.
Source: §1 · p. 2Method
Results
Takeaways & Limitations
The results indicate that low synthetic-text utility does not necessarily mean low membership-inference leakage.
Source: §3 · p. 4The evaluation uses datasets from the original works, so its results may not perfectly generalize to other settings.
Source: Limitation · p. 4
Abstract
from arXiv · showhide
Synthetic Data Generation (SDG) is an actively studied way to enable safe sharing of private data more broadly. Inference-only SDG methods, such as Private Evolution (PE), are particularly appealing, as they can leverage black-box access to pre-trained foundation models to generate high quality data without the need for model fine-tuning on the private data. PE is explicitly designed to produce synthetic data under Differential Privacy (DP). Yet formal DP guarantees do not eliminate the need for empirical privacy auditing of the practical susceptibility of released synthetic data to adversarial attacks. In this work, we introduce the first white-box Membership Inference Attack (MIA) tailored to the voting-based algorithm underlying the PE framework. By exploiting how the iterative filtering process biases the survival and proximity of synthetic candidates toward private instances, our attack achieves empirical performance indicating private data leakage in the PE applications across image, text, and tabular modalities.
1 Introduction
Private Evolution (PE) offers inference-only synthetic data generation using pre-trained models and voting, but its practical privacy still warrants empirical auditing. The paper introduces EVO-MIA, a modality-agnostic attack using released voting patterns to test whether a private instance shaped the synthetic distribution.
1 Introduction
- PE uses API-only access to pre-trained foundation models and iteratively retains candidates that private instances vote most similar to, avoiding foundation-model fine-tuning on private data.PE has been used across text, image, and tabular data, in centralized and federated settings.Source: §1 · p. 1
- Empirical membership inference attacks provide a complementary audit of whether specific private instances were used to shape released synthetic data.The paper frames empirical auditing as important alongside PE’s theoretical privacy guarantees, given industry deployments.Source: §1 · p. 2
- EVO-MIA uses private samples’ voting patterns and DP-protected per-round noisy vote histograms to test whether a target instance shaped PE’s synthetic distribution.The attack compares how well the observed votes fit membership versus non-membership.Source: §1 · p. 2, Figure 1 · p. 2
- The authors describe EVO-MIA as efficient, generalizable across modalities, applicable to other voting-based SDG algorithms, and requiring no shadow-model training.They present it as a practical way to verify privacy protection before deployment.Source: §1 · p. 2
2 Methodology
The methodology describes PE’s noisy, clipped voting process and an adversary that uses per-round candidate releases and vote histograms to score whether a target instance participated. EVO-MIA estimates expected votes under membership and non-membership, then accumulates per-round log-likelihood ratios; its threat model also covers a candidate-survival-only variant for centralized settings.
2 Methodology
- PE releases the final-round candidates as synthetic data; each private instance votes for its nearest candidate, and Gaussian-noised, threshold-clipped counts determine which candidates survive for mutation.A candidate’s released count is max(κ + ξ − θ, 0), with ξ drawn from a Gaussian distribution with standard deviation σ.Source: §2 · p. 2
- EVO-MIA assumes access to per-round candidates and noisy vote counts, plus a public embedding model, mechanism parameters, dataset size, and an in-distribution reference set.Its score estimates whether the target belongs to the private dataset.Source: §2 · p. 3
- The threat model covers federated settings where participants or the aggregation server can log vote counts, and centralized settings where the model provider can observe candidate survival but not counts.For the weaker centralized adversary, EVO-MIA-BB uses candidate survival without vote counts.Source: §2 · p. 3
- Evaluation measures ROC-AUC on members used for PE voting versus held-out non-members drawn from the same distribution.The target set combines these two groups.Source: §2 · p. 3
- The adversary’s score sums per-round log-likelihood ratios comparing observed noisy votes under membership and non-membership hypotheses.For each round, it selects the candidate closest to the target and uses an auxiliary reference set to estimate its expected vote distribution.Source: §2 · p. 3
- The evaluation treats mechanism parameters as public, following Kerckhoffs’s principle.The holder of the proprietary model is a natural adversary in the centralized setting.Source: §2 · p. 3
- The exact non-member true-count distribution is binomial and can be close to Poisson for small vote probabilities, but experiments use a negative binomial with mean µ and variance 1.8 µ.Observed counts varied 1.5–3.8 times more than a Poisson distribution allows; independent noise makes the per-round independence assumption exact for noise but approximate for correlated underlying counts.Source: §2 · p. 4
3 Experiments and Results
Across modalities, EVO-MIA’s reported performance varies: it is strongest on text, stands out on Cat images, and is near the baselines on much of the tabular evaluation. The results also show that low synthetic-text utility can coexist with a membership-inference AUC above chance.
3 Experiments and Results
- The reported average AUCs summarize three runs across the text, image, and tabular evaluations.The results are presented in Tables 1, 2, and 3.Source: §3 · p. 4
- Text results were strongest: OpenReview AUC was 0.873 at ϵ=∞ and 0.685 at ϵ=10, while MIMIC-III AUC was 0.758 and 0.605 respectively, exceeding the n-gram baseline [15] across ϵ values.For images, Cat achieved AUC 0.846 at ε=∞ and 0.692 at ε=10, substantially above LOGAN; tabular performance was on par with baselines overall, with a slight high-epsilon advantage on breast cancer, while adult and person-activity attacks stayed near chance.Source: §3 · p. 4
- At ϵ=10, synthetic text fell below the majority baseline in utility while retaining an AUC of 0.685.The authors note that low utility does not imply low leakage.Source: §3 · p. 4
4 Conclusion and Future Work
The paper presents EVO-MIA as the first membership-inference attack for PE-based frameworks that works across modalities. Future work will examine tighter privacy auditing and LLM-based attack improvements.
4 Conclusion and Future Work
- The authors characterize EVO-MIA as the first cross-modal membership-inference attack for PE-based frameworks, addressing a gap in attacks for inference-only synthetic data generators.Source: §4 · p. 4
- Future work will examine tighter privacy auditing using closed-form evaluation of PE’s voting mechanism’s privacy loss for known datasets, and explore LLM-based methods to improve the attack.The authors contrast the prospective auditing approach with DP-SGD, for which they say tight auditing is less easy.Source: §4 · p. 4
Limitation
The evaluation may not generalize perfectly beyond datasets used in the original works. The attack also assumes access to per-round releases, including noisy vote histograms in its main form.
Limitation
- The evaluation uses datasets from the original works, so its results may not perfectly generalize to other settings.The reported MIA tables cover tabular, image, and text PE results.Source: Limitation · p. 4, Table 1 · p. 5, Table 2 · p. 5
- Attack AUC is computed after subsampling to at most 2,000 members and 2,000 non-members, pooling predictions across classes.For datasets with fewer non-members, all available non-members are used.Source: Limitation · p. 4
- The main attack assumes per-round candidates and noisy vote histograms are observable; keeping intermediate rounds private would reduce this exposure.EVO-MIA-BB relaxes the need for vote counts but still requires per-round candidates.Source: Limitation · p. 5
A Datasets
The evaluation defines members as records used by PE and compares them with disjoint, in-distribution non-members, using a separate reference set. It covers image, text, and tabular datasets.
A Datasets
- Members are records used by PE, non-members are disjoint records from the same dataset, and a third disjoint in-distribution reference set is reserved for the attack.Figure 2 depicts the target dataset and reference pool.Source: A Datasets · p. 8, Figure 2 · p. 8
- The evaluation includes OpenReview peer reviews and MIMIC-III discharge notes, with the text datasets conditioned on review area and rating or diagnosis classes.OpenReview uses joint area × rating classes; MIMIC-III uses five diagnosis classes.Source: A Datasets · p. 8
- Image datasets are CIFAR-10, Camelyon17, and Cat, while tabular benchmarks are adult, artificial-characters, breast-cancer, and person-activity.The image datasets span natural images, histopathology patches, and cat faces; tabular utility labels range from two to eleven classes.Source: A Datasets · p. 8
B Baseline Attacks
The paper compares its PE-specific attack with black-box MIA baselines tailored to text, tabular data, and images.
B Baseline Attacks
- Text comparisons use an n-gram MIA [15], tabular comparisons use GAN-leaks [1] and DOMIAS [19], and image comparisons use LOGAN.The n-gram baseline trains on synthetic and reference data; LOGAN trains a shadow GAN on synthetic data.Source: B Baseline Attacks · p. 9
- The n-gram attack uses differences in output probability as its membership signal, while LOGAN uses a shadow GAN as a membership classifier.GAN-leaks and DOMIAS are the tabular baselines named in the comparison.Source: B Baseline Attacks · p. 9
C Distribution Effect on MIA Performance
The distribution-shift experiment constructs member and auxiliary pools from opposite PCA tails or from an iid split, then evaluates utility, separation, and attack AUC. Near-perfect AUCs under opposite-tail sampling reflect distribution differences; only the iid setting tests leakage with exchangeable members and non-members.
C Distribution Effect on MIA Performance
- For qX splits, each class contributes its lowest X% of PCA-1 rows as members and highest X% as auxiliary data; smaller X creates stronger shift.The pools are subsampled to 1,500 rows each; q18 leaves the middle 64% between the tails, while iid uses a class-stratified random disjoint split.Source: C Distribution Effect on MIA Performance · p. 9
- Utility is held-out 10-class accuracy, while embedding_gap and pca1_wasserstein measure separation between member and auxiliary pools.embedding_gap uses mean embeddings in the full seven-feature tabular embedding space; iid has gap 0.015 and q18 has 0.775.Source: C Distribution Effect on MIA Performance · p. 9
- Each distribution experiment scores 1,000 targets—500 members and 500 challenge non-members—and reserves 750 rows for the reference set.Members are drawn from a 1,501-row member pool.Source: C Distribution Effect on MIA Performance · p. 9
- Near-perfect AUCs under opposite-tail sampling reflect distribution differences, so only the iid setting measures leakage with exchangeable members and non-members.Table 5 reports distribution-shift and privacy metrics across splits.Source: C Distribution Effect on MIA Performance · p. 9, Table 5 · p. 9
D Black-Box EVO-MIA
EVO-MIA-BB infers membership from whether the synthetic candidate nearest the target survives each PE round, using survival probabilities under competing membership hypotheses.
D Black-Box EVO-MIA
- The attack estimates the cutoff τ by matching the expected number of H0 survivors to the observed survivor count V.It computes expected votes from the auxiliary set, estimates survival under H0, then searches for τ by bisection.Source: D Black-Box EVO-MIA · p. 10
- EVO-MIA-BB adds a likelihood-ratio contribution each round according to whether the target’s nearest synthetic candidate survived.It uses the per-round candidate pool and survival set rather than noisy histogram counts; Table 6 compares performance with and without those counts at ε = 10.Source: D Black-Box EVO-MIA · p. 10, Table 6 · p. 11
- For image PE, EVO-MIA-BB substitutes the probability that a candidate is drawn at least once for the threshold-based survival probability.Text and tabular PE keep candidates with the highest released counts; image PE samples candidates with replacement in proportion to noisy counts.Source: D Black-Box EVO-MIA · p. 10
- Table 6 compares EVO-MIA with noisy histogram counts against EVO-MIA-BB without them on datasets at ε = 10.The supplied caption identifies the comparison but gives no performance values.Source: Table 6 · p. 11
E Additional utility and MIA Results
This section presents results for PE-Tabular.
E Additional utility and MIA Results
- The section reports results for PE-Tabular.Table 7 is identified as the results table for PE-Tabular.Source: Table 7 · p. 11
F Ablation Study Results
The ablations show that leakage varies with the number of evolution rounds and privacy budget, while generator choice has a smaller effect; reference-assisted attack AUC is largely stable across private-dataset sizes.
F Ablation Study Results
- Leakage AUC grows with evolution rounds and saturates around T ≈ 6 at fixed privacy budget, while utility keeps improving; at fixed noise, AUC grows for text and images but stays flat for tabular data.The rerun experiment recomputes noise for each T, whereas the single-run analysis attacks only its first T rounds.Source: F Ablation Study Results · p. 11, Table 11 · p. 13, Table 12 · p. 13
- On artificial-characters, AUC falls from 0.618 without noise to 0.515 at ε = 0.25; on adult and person-activity, it stays near 0.5 across budgets, including ε = ∞.Baseline attacks perform similarly on adult and person-activity, indicating low leakage there even without DP noise.Source: F Ablation Study Results · p. 11, Table 13 · p. 13
- The listed utility-and-fidelity tables cover PE-Tabular and PE-Images at ε = 10.Their captions identify MIA leakage, utility, and fidelity as the comparison dimensions but provide no values.Source: Table 8 · p. 12, Table 9 · p. 12
- At ε = 10 after 9 rounds, generator-model AUCs range from 0.578 to 0.599, with no trend by model size.The paper reports that generator choice affects leakage less than ε and T.Source: F Ablation Study Results · pp. 11–12, Table 14 · p. 13
- With a reference dataset, EVO-MIA is largely unaffected by private-set size; without it, AUC drops from 0.636 to 0.564.The no-reference variant assumes every candidate is equally likely to receive a vote.Source: F Ablation Study Results · p. 12, Table 15 · p. 14