Source-linked AI summary
Subgroup Membership Inference Audits of Differentially Private Synthetic Text
Yidan Sun, Viktor Schlegel, Srinivasan Nandakumar, Siew Kei Lam, Anil Anthony Bharath
TL;DR
Synthetic-text membership audits may miss vulnerable subgroups by measuring only random-record risk. The paper introduces a pool-parameterized, release-based audit across proxies, attacker scenarios, datasets, generators, and privacy budgets, finding persistent and uneven leakage under DP. The results show that residual risk is concentrated, protection is weaker for high-risk records than random controls, and vulnerable records depend on the release mechanism.
Problem
Existing audits measure average-case membership risk for random records, leaving subgroup-targeted leakage insufficiently characterized.
Method
The paper defines a pool-parameterized membership game and audits synthetic releases with 32 proxies under three attacker scenarios across four datasets, three generators, and five privacy budgets.
Results
DP substantially reduces average leakage at every tested budget, but leakage persists above chance, concentrates in a small record subset, and is reduced less for high-risk than random records.
Takeaways & Limitations
Record-level risk cannot be assessed independently of the release mechanism, so average-case attacks and attribute-based screening can miss vulnerable records.
Takeaways & Limitations
The corpora and target pools are small, pool-drawing variance is not captured, and the adversary is strictly release-based across three mechanism families.
Abstract
from arXiv · showhide
Synthetic data releases are increasingly proposed in the literature as a means of sharing realistic data replicas in lieu of sensitive private datasets. Even when the worst-case privacy leakage of such releases is bounded by means of differential privacy (DP), in practice a residual risk remains. Membership inference attack (MIA) audits are conducted to empirically quantify this risk. However, existing methods only measure average-case risk for randomly drawn records, which might conceal the risk to vulnerable subgroups. To highlight this issue, we define a subgroup-targeted membership inference game in which the target pool is an explicit parameter, and instantiate it with an audit of 32 proxies under three scenarios with different levels of attacker knowledge, across four datasets, three generators (DP-SGD fine-tuning, API-based prompting, and activation steering), and five privacy budgets. The audit shows that synthetic releases leak subgroup membership and that prior attacks systematically underestimate this leakage. DP is effective at the aggregate level: it substantially reduces average leakage at every budget we test. Three observations temper this picture. First, the remaining leakage is concentrated rather than spread out: under DP, a tenth of the records carries roughly 40% of it. Second, the protection DP delivers in practice is uneven: within its worst-case guarantee, the noise removes more of the measured leakage from random records than from high-risk ones---and a merged-pool audit that scores both record types against shared negatives confirms this at the record level. Third, \emph{which} records leak proves to be a property of the release mechanism rather than of the record alone, so record-level risk cannot be assessed independently of the release.
1 Introduction
Synthetic-text membership audits can miss concentrated subgroup risk when they sample only random records. The paper introduces a pool-parameterized audit and finds persistent, uneven leakage despite DP.
- Motivation: Existing synthetic-text audits often omit membership inference or estimate only average-case risk from random targets.Text releases require extracting evidence from whole datasets without obvious aggregate statistics, while stronger model-access attacks do not fit release-based or training-free settings.
- Approach: The pool-parameterized membership game makes the target pool explicit, supporting high-risk, random-control, and composite audits.The formulation separates targeting effects from leakage that would appear regardless of targeting.
- Findings: On high-risk records, the best attacker reaches AUC 0.79–0.83 and identifies 34–61% of true members below a 5% false-positive rate.With only the release, 39–52% are exposed; under DP, leakage remains above chance with AUC 0.55–0.65 and TPR up to approximately 18%.
- Findings: DP reduces average leakage, but a tenth of records carries roughly 40% of residual vulnerability and high-risk records lose less leakage than random controls.Which records leak depends on the release mechanism rather than intrinsic record attributes, requiring per-release auditing.
- Approach: The audit combines 32 proxies, three attacker scenarios, four datasets, three generators, and five privacy budgets.It evaluates release-only, raw-reference, and generator-assisted attacker knowledge across a broad release-based audit.
2 Related Work
Prior work established that memorization and privacy risk can vary across records, but synthetic-text audits have not made concentrated subgroup risk the explicit object of a release-based audit. This paper addresses that gap with pool targeting, matched controls, heterogeneous proxies, and attacker-capability scenarios.
- Membership inference: Model-based MIA emphasizes per-example calibration and low false-positive rates, whereas this work reasons from released synthetic data without model access.The release-based adversary can apply to training-free synthesis methods and does not query likelihoods.
- Synthetic-data auditing: Synthetic-data privacy studies and release-based attacks show that releasing synthetic records does not guarantee privacy, but much prior work focuses on tabular data or aggregate density.Text-specific work commonly uses crafted canaries to measure worst-case memorization.
- Synthetic-text auditing: Canary audits provide valuable upper bounds on worst-case memorization but do not measure naturally occurring subgroup risk.The paper instead targets naturally occurring records and evaluates vulnerability at the individual-record level.
- Novelty: The paper makes concentrated risk the audit object through subgroup-targeted games, matched random controls, heterogeneous proxies, and explicit attacker-capability scenarios.This release-based formulation had not been studied for synthetic text across an explicit range of attacker capabilities.
- Motivation: Prior work shows that outliers and atypical examples are memorized more readily and that average-case metrics can understate risk to vulnerable records.The paper extends these observations to subgroup-targeted auditing of synthetic text.
3 Method
The method treats the target pool as an audit parameter and tests membership using release-derived proxy evidence under progressively stronger attacker information. Repeated attack episodes then quantify per-record vulnerability and its concentration while preserving comparability through composite pools.
- Target pools: A selection rule defines a target pool, from which the target is sampled while private training records come from its complement.Pools may represent outliers, rare labels, random controls, or composites.
- Membership game: Each game samples a target and membership bit, constructs a member or non-member training set, releases synthetic data, and thresholds an attack score.The adversary receives the target, release, and scenario-dependent auxiliary information.
- Evaluation: Attack advantage is reported as ROC AUC and, where relevant, true-positive rate at low false-positive rates.Using RANDOM recovers the average-case game, while comparing against it measures what targeting adds.
- Comparability: Composite pools score constituent record types against one reference distribution, making their advantages and attribute associations directly comparable.Separate within-pool games normalize against different complements and cannot support the same comparison.
- Proxy evidence: The ideal release likelihood-ratio test is intractable, so the audit substitutes proxy metrics such as n-gram likelihood, similarity, overlap, and distance.Reference objects vary by scenario, while the proxy set remains fixed across setups.
- Attacker scenarios: Three scenarios provide release-only, raw-reference, and generator-assisted attacker capabilities.Reference-based scenarios calibrate target difficulty using IN and OUT reference sets; generator-assisted calibration operates in release space.
- Per-record vulnerability: Repeated episodes estimate sample-level vulnerability from member and non-member scores within a fixed proxy and scenario.Cross-proxy and cross-scenario analyses use ranks because raw vulnerability scores are not comparable.
4 Experiment Setup
The experiments test subgroup-targeted membership leakage across diverse datasets, pools, generators, attacker settings, and privacy mechanisms. The protocol keeps pool comparisons consistent while measuring both aggregate and record-level vulnerability.
- Five research questions assess subgroup leakage, attacker knowledge, existing attacks, uneven DP protection, and predictability of record-level risk.
- Datasets and Domains: Four datasets span healthcare, finance, and legal text with differing lengths, label densities, and linguistic structures.The corpora include N2C2’08, PSYTAR, DMSAFN, and EURLEX.
- Target Pools: Three target-pool types comprise geometric outliers, rare-label records, and size-matched random controls.Rare-label and outlier pools use distinct high-risk criteria, while random pools provide dataset-specific controls.
- Generators: Three generators represent API-based prompting, DP-SGD fine-tuning, and activation steering, while sharing the Llama-3.2-1B backbone family and scale.The privacy mechanisms differ across Aug-PE, DP-Gen, and EPSVec, with the backbone held broadly constant.
- Attacks and Proxies: The audit adapts release-based attacks using calibrated lexical and embedding proxies, including retrieve-then-compare scores over attacker-accessible data.Reference attacks are re-instantiated under identical pools, releases, and trial plans where applicable.
- Evaluation Protocol: Each setting uses 100 releases and 50 negative-sampling rounds, with positives and negatives drawn from the same target pool to measure within-pool membership signal.Reported confidence intervals capture negative-sampling variability only, while the highest mean-AUC proxy is an empirical worst-case audit rather than a deployable attack.
5 Experimental Results
Across subgroup-targeted audits, synthetic releases leak membership above chance, including under DP, while leakage varies with generator, privacy budget, attacker scenario, and target-pool definition. DP reduces aggregate leakage but leaves residual risk concentrated in a small, mechanism-dependent subset that prior attacks and record attributes do not reliably identify.
- 5.1 Subgroup Membership Leakage Across Generators and Budgets: Best-proxy AUC reaches 0.79–0.83 on high-risk records for non-private DP-SGD fine-tuning, identifying 34–61% of true members at false-positive rates below 5%.A release-only attacker is already within 0.01–0.04 AUC of the best calibrated attacker in these combinations.
- 5.1 Subgroup Membership Leakage Across Generators and Budgets: At every finite DP budget tested, subgroup membership remains inferable above chance, with best-proxy AUC 0.52–0.65 and TPR up to 18%.Under DP-SGD fine-tuning, average AUC falls from 0.80 without privacy to 0.59 under DP.
- 5.1 Subgroup Membership Leakage Across Generators and Budgets: Rare-label pools reveal leakage missed by outlier audits, reaching best AUC 0.86 on PSYTAR with TPR 59% under non-private fine-tuning.Under DP, rare-label pools remain on a 0.53–0.66 AUC floor across generators and budgets.
- 5.2 Scaling with Attacker Side Information: Additional attacker knowledge changes which proxy wins more than how much leaks: raw-reference calibration shifts AUC by +0.009, while generator access shifts it by −0.004 on average.The raw-reference gain is statistically detectable but practically negligible; generator access is indistinguishable from zero.
- 5.3 Comparison with Prior Release-Based Attacks: Prior release-based attacks underestimate subgroup leakage because DOMIAS captures embedding evidence and CANARY uses only one fixed signal of each type.The audit evaluates multiple lexical and embedding proxies and selects the best proxy for each setting.
- 5.4 Aggregate vs. Individual Protection: Leakage Concentration under DP: Under DP, the median top-decile leakage share S10 rises to 40% while the median share of records with positive vulnerability falls from 82% to 61%.S10 ranges from 28–69% across 16 DP fine-tuning combinations and exceeds even-spread leakage in 15 of 16 combinations.
- 5.4 Aggregate vs. Individual Protection: Leakage Concentration under DP: Average AUC does not identify the most vulnerable records: none of 210 attribute–pool tests predicts vulnerability after Holm correction.Weak pooled correlations under DP-SGD fine-tuning are not considered useful for predicting membership-inference risk from records.
- 5.6 High-Risk vs. Random Pools: DP changes high-risk versus random-pool distinguishability: all six DP-Gen outlier gaps at ε = 4 are positive, whereas non-private outlier pools are often no easier to attack.The training-free EPSVEC generator shows positive outlier gaps in 11 of 12 comparisons regardless of noise.
6 Conclusion and Limitations
The paper concludes that subgroup-targeted audits reveal leakage that aggregate or prior attacks can miss, while DP protects high-risk groups less than random controls. It recommends per-release auditing, but limits confidence because the corpora and pools are small and several sources of uncertainty and scope remain.
- Subgroup-targeted auditing shows that DP protects membership inference signal from high-risk groups less than from random controls.
- Limitations: The corpora contain 604–5,102 records and the pools 16–71, so correlations below |ρ| ≈0.33–0.50 cannot reach significance.
- Limitations: Uncertainty intervals capture negative-sampling variance but not variance from drawing the pool itself, and the release-based adversary and three mechanism families limit scope.
7 Appendix
The appendix specifies how target pools, proxy families, reference sets, privacy budgets, and mechanisms are constructed for the audit. These choices distinguish subgroup atypicality from random controls and make attacker capability and privacy settings explicit.
- Three target pools capture geometric outliers, statistical rarity, and uniform random controls used as the null condition.The outlier and rare pools represent different notions of atypicality, while controls are replicated over three seeds.
- Rarity is dataset-specific: N2C2 uses co-morbidity count, while DMSAFN contributes no rare-combination subgroup.
- The 32 proxies comprise 13 families, with retrieval and neighborhood features instantiated across multiple aggregation or neighborhood definitions.
- Reference sets are shared across targets, so each target is IN for half of Mref = 4 references and the baseline omits dataset-level sampling variance.
- The audit evaluates ε ∈ {∞, 4, 2, 1, 0.5}, with ∞ denoting the non-private setting, across DP-Gen, Aug-PE, and activation steering.
- Proxy scores are computed against a release, raw reference dataset, or synthetic reference release depending on attacker scenario.
7.5 More Results of Our MIA Auditing
Additional analyses test uncertainty, proxy fusion, and performance across privacy budgets. They show that deployable fusion remains strong when leakage is strong, but correlated proxies do not improve on the per-cell oracle under DP.
- Tables 9–12 report per-cell 95% intervals that capture negative-sampling variability only.
- Fusion performance: The deployable fusion reaches AUC 0.76–0.79 on strong non-private outlier cells and up to 0.83 on rare pools, within 0.03–0.05 of the per-cell oracle.
- Fusion performance: Across 234 populated cells, fusion wins in 4 and loses in 200, with a mean deficit of ∆ = −0.077 AUC.
- Fusion performance: Under DP, where the best single proxy reaches 0.55–0.65, fusion typically falls to 0.44–0.57.
- Interpretation: Highly rank-correlated proxies and small training sets leave little complementary signal, while cross-validated model variance harms fusion.
7.7 Extended Baseline Comparisons
Extended baseline comparisons show that the audit’s proxy selection remains competitive across datasets, generators, privacy budgets, pools, and attacker scenarios. The appendix also reports EPSVEC results under the same overall evaluation framework.
- Across 78 populated dataset, generator, privacy, and pool cells, the audit has the highest AUC in 69, ties in 2, and is beaten in 7.
- The comparisons maximize each method over its own variants independently in every cell, applying the same worst-case treatment to all methods.
- Table 10 reports EPSVEC performance across privacy budgets and attacker-capability scenarios, with lexical or embedding proxies identified for highlighted cells.
7.8 Per-Cell Vulnerability Distribution
Per-cell vulnerability is unevenly distributed: under DP, a small subset of qualifying records accounts for a disproportionate share of leakage, while non-private fine-tuning concentrates leakage differently.
- Under DP, the top tenth of qualifying records carries roughly 40% of leakage, indicating concentrated rather than evenly distributed vulnerability.
- The four non-private fine-tuning cells are the only cells showing the contrasting concentration pattern identified in the table.
7.9 Attribute Analysis: Computation and Robustness
The attribute analysis computes within-pool rank correlations between record attributes and vulnerability, using pooled inference and explicit selection-rule safeguards. Single-pool nulls have limited sensitivity, so combined pools are needed to detect smaller effects.
- Results: Figure 5 finds no nominally significant attribute association in outlier or rare-label pools, consistent with the random-pool null result.
- Computation: Spearman correlations relate five release-independent attributes to per-record vulnerability ranks within each qualifying pool.Qualifying records must appear at least once as inserted members and once as non-member candidates; vulnerability is pooled over privacy budgets.
- Computation: Fisher-z pooling combines random pools with weights n −3, while Holm correction is applied across the five attribute tests.
- Inference limits: At n = 36 and n = 19, the per-pool detection floors are |ρ|min = 0.33 and 0.46, respectively.A nonsignificant single-pool result therefore does not exclude smaller correlations.
- Selection robustness: The conservative selection rule fixes a proxy using 80% of instances, whereas the worst-case rule selects by AUC over all tested episodes and is treated as sensitivity analysis.The worst-case rule is more sensitive but yields optimistic p-values because selection includes the tested episodes.
7.10 Robustness to the proxy-selection rule
Attribute conclusions are broadly robust to proxy-selection choices: no corrected attribute–pool test is significant under the paper’s rule, although significance can appear under more favorable selection rules.
- Overall robustness: No attribute–pool test is significant after Holm correction under the paper’s proxy-selection rule.
- Selected pools: Figure 5 reports no nominal significance across 70 per-pool tests in the selected outlier and rare-label pools.
- Selection dependence: Outlierness changes from marginal under the paper’s rule to significant under all-episode and fixed-embedding selection, with Holm p = 0.09 versus p = 0.002.The fixed-embedding result reflects selecting and scoring in the same embedding geometry, not evidence of a stronger effect.
- Effect size: Even under the most favorable rule, the strongest attribute reaches only |ρ| = 0.32, below the level needed for record-level triage.
7.11 Cross-mechanism consistency of per-record vulnerability
Per-record vulnerability is not consistent across release mechanisms, even when identical records are scored with the same proxy. The accompanying robustness analyses show that proxy fusion generally does not improve over the best individual proxy.
- Cross-mechanism consistency: On identical records with the identical proxy, vulnerability ranks correlate near zero across mechanisms.Mean Spearman correlation is −0.09 between DP-SGD fine-tuning and activation steering, and +0.07 between fine-tuning and API-based prompting in the reported overlapping pools.
- Fusion robustness: Across another 60-cell comparison, fusion wins in 2 and loses in 58, with mean Δ = −0.071.
- Fusion robustness: Across 60 populated cells, fusion wins in 0 and loses in 60, with mean Δ = −0.084.
- Fusion robustness: Across 60 populated cells, proxy fusion wins in 2 and loses in 58, with mean Δ = −0.085.
- Fusion robustness: Across 54 populated cells, fusion wins in 1 and loses in 53, with mean Δ = −0.082.
- Scope: Table 23 compares vulnerability correlations on the same records and proxy, while noting that AUG-PE has no random-pool releases.