PaperlayerView exact revision on arXiv

Source-linked AI summary

Enhanced Membership Inference Attacks against Machine Learning Models

Jiayuan Ye, Aadyaa Maddi, Sasi Kumar Murakonda, Vincent Bindschaedler, Reza Shokri

arXiv:2111.09679v4cs.LGcs.CRstat.ML

TL;DR

Membership-inference auditing needs a consistent way to measure and explain privacy leakage from machine-learning training data. The paper formulates attacks as hypothesis tests across indistinguishability games, designs stronger attacks that reduce attacker uncertainty, and reports higher power for a given error while explaining differential vulnerability. Its framework also identifies a limitation of prior work: attack results can conflate inherent model leakage with uncertainty about the target.

  • Problem

    Prior membership-inference attacks lack consistent, comparable interpretations and formal explanations of what leakage they measure and why strategies succeed.

  • Method

    The paper introduces indistinguishability games, binary hypothesis testing, likelihood-ratio-based attacks, and target-dependent reference-model strategies to compare and strengthen audits.

  • Results

    Reference-model and self-distillation attacks improve power for a given error, while differential analysis explains why attacks detect different vulnerable records.

  • Takeaways & Limitations

    Meaningful privacy audits should report power-versus-error behavior and distinguish model leakage from attacker uncertainty across explicitly defined inference games.

  • Takeaways & Limitations

    Prior attack success rates can reflect uncertainty about the target in addition to inherent model leakage, limiting direct comparability.

Abstract

from arXiv · show

How much does a machine learning algorithm leak about its training data, and why? Membership inference attacks are used as an auditing tool to quantify this leakage. In this paper, we present a comprehensive \textit{hypothesis testing framework} that enables us not only to formally express the prior work in a consistent way, but also to design new membership inference attacks that use reference models to achieve a significantly higher power (true positive rate) for any (false positive rate) error. More importantly, we explain \textit{why} different attacks perform differently. We present a template for indistinguishability games, and provide an interpretation of attack success rate across different instances of the game. We discuss various uncertainties of attackers that arise from the formulation of the problem, and show how our approach tries to minimize the attack uncertainty to the one bit secret about the presence or absence of a data point in the training set. We perform a \textit{differential analysis} between all types of attacks, explain the gap between them, and show what causes data points to be vulnerable to an attack (as the reasons vary due to different granularities of memorization, from overfitting to conditional memorization). Our auditing framework is openly accessible as part of the \textit{Privacy Meter} software tool.

1 INTRODUCTION

The paper argues that membership-inference auditing lacks consistent risk measurement and explanation, then develops a hypothesis-testing framework for comparing and strengthening attacks. It uses differential analysis to explain attack differences and identify vulnerable records more precisely.

  • Membership-inference attacks can audit privacy risk, but prior attacks may measure different leakage notions because their success depends on adversarial uncertainty and game assumptions.
  • The framework models membership inference as binary hypothesis testing over in and out worlds, quantifying attack power with TPR and error with FPR.
  • It formalizes prior attacks and compares them consistently across the full power-versus-error curve, emphasizing low-FPR and high-TPR regions.
  • The paper introduces population-based, reference-model, and self-distillation attacks to reduce attacker uncertainty and improve membership-inference performance.
  • Differential analysis identifies why records differ in vulnerability, including distinct loss-distribution patterns among highly vulnerable points.
  • 0.984 AUC is achieved by the strongest attack when analysis focuses on differentially vulnerable points.

2 RELATED WORK

Prior work established membership-inference risks across aggregate data, machine-learning models, and differential privacy. Later studies broadened attack settings and examined average-case and worst-case leakage, motivating a more unified framework.

  • Early studies analyzed membership risk in genome mixtures, aggregate statistics, and Bayesian networks learned from dependent data.
  • Differential privacy formally bounds membership-inference success for distinguishing neighboring datasets, linking privacy definitions to attack performance.
  • Shadow-model attacks demonstrated membership vulnerability in black-box machine-learning models and inspired extensions to white-box, label-only, and federated-learning settings.
  • Research on worst-case privacy restricted targets to heuristically selected or poisoned records and observed significantly higher attack success rates.

3 ATTACK FRAMEWORK

The framework defines privacy loss through indistinguishability games and hypothesis tests, varying which model, record, dataset, and randomness are fixed. It reports attack power against error while separating model leakage from attacker uncertainty.

  • The auditing framework has three elements: an inference game, an indistinguishability metric, and membership-inference attacks constructed as hypothesis tests.
  • The average-model-and-record game samples datasets, models, and target records randomly, so measured performance averages leakage across models and records.
  • Fixing dataset and model seeds yields a fixed-model game that quantifies the privacy loss of a specific released model.
  • Fixing the record while sampling the remaining dataset distinguishes models trained with and without that particular record, measuring record-specific privacy loss.
  • Fixing both record and dataset produces a game for specific record-dataset privacy loss that resembles worst-case leakage when both are crafted.
  • The indistinguishability metric treats privacy loss as an adversary’s success in distinguishing models trained with and without a target record.
  • Audits should report true-positive power versus false-positive error rather than accuracy alone, because attack success depends on both model leakage and attacker uncertainty.

4 CONSTRUCTING BLACK-BOX MEMBERSHIP INFERENCE ATTACKS

The paper constructs black-box membership inference attacks as hypothesis tests whose thresholds progressively depend on the target model, target record, or both, reducing adversarial uncertainty. This framework enables approximate false-positive control and explains why more specific attacks can approach the impractical leave-one-out attack.

  • Attack framework: The attacks use binary hypothesis tests on a target model’s output and a target record to decide whether the record belongs to the training dataset.The adversary has black-box access to the model and outputs a membership guess.
  • Attack framework: Threshold dependency progressively shrinks the out world: Attack S uses the broadest uncertainty, while later attacks condition on the model, record, or both.The paper identifies S as the shadow-model attack, P as model-dependent, R as sample-dependent, and D as model- and sample-dependent.
  • Attack comparison: The framework estimates thresholds that approximately guarantee a chosen FPR α before evaluation, unlike prior attacks that do not provide a generic threshold-selection rule for a specified FPR.This supports comparisons across the full TPR-versus-FPR curve and distinguishes attacks by their uncertainty assumptions.
  • Attack S: Attack S uses a class-dependent threshold independent of target features and model, so it reduces label uncertainty but retains uncertainty about the target data and model.Its threshold and loss-histogram shapes remain the same across the four targets in Figure 1.
  • Attack P: Attack P computes a model-dependent threshold from population data, reducing uncertainty about the target model without reducing uncertainty about the target record.It avoids shadow-model training while exploiting statistics specific to the target model.
  • Attack R: Attack R uses reference models to set a sample-dependent threshold, reducing uncertainty about the target record but not the target model.Its threshold is the α-percentile of the target-data loss distribution on reference models.
  • Attack D: Attack D conditions thresholds on both target model and record, leaving only training-algorithm randomness under its fixed-pair inference game.It approximates leave-one-out performance without requiring knowledge of n−1 other training records.

5 EMPIRICAL EVALUATION

The evaluation compares attacks under matched confidence requirements to explain performance differences, vulnerability detection, and the role of reduced uncertainty. Reference- and distillation-based attacks generally identify more and more vulnerable records than shadow- and population-based attacks, while Attack D performs best overall.

  • Evaluation goals: The evaluation tests how reduced uncertainty affects attack performance, how predictions differ, and which records remain vulnerable or are missed.Experiments use matched confidence requirements and include Purchase100 results, with broader evaluations reported elsewhere.
  • Attack performance: Attack D achieves the highest AUC on Purchase100, and this AUC ranking also holds across additional training setups and datasets.The authors attribute the performance improvement to reducing uncertainty in Attacks R and D compared with the overly general targets of S and P.
  • Attack performance: At FPR< 0.2, Attacks R and D achieve above 10x higher TPR than Attacks S and P under fixed low FPR.Attack R performs slightly better than D in TPR at small FPR, while D performs better at fixed high TPR by attaining lower FPR.
  • Prediction comparisons: Attack S and P make similar membership predictions, whereas Attack R differs substantially and correctly identifies 19.84% of inputs that S misses.The high agreement between S and P contrasts with R’s stronger predictions and greater confidence on jointly correct regions.
  • Prediction comparisons: Attack D agrees most with ground-truth membership on training points and agrees with the ideal leave-one-out Attack L at rate 0.804.Its stronger agreement is consistent with D’s larger threshold and its approximation of retraining through distillation.
  • Vulnerable records: R-correct records are more vulnerable than SP-correct records, showing that Attack R identifies more vulnerable records and estimates individual vulnerability more credibly.R-correct records are detected by R but missed by S and P under FPR 0.3.
  • Vulnerable records: R-correct records generally have higher loss than SP-correct and all-correct records, suggesting S and P miss vulnerable training data with intrinsically high loss.The comparison links differential vulnerability to the loss distributions of record types.
  • Comparison with concurrent work: Attack R achieves higher AUC than LiRA with similarly high TPR at small FPR, while Attack D additionally obtains lower FPR at fixed high TPR.For attacking k points, the authors report LiRA’s amortized reference-model cost as (k + 1)/2 times their cost; D also captures unknown target-dataset influence.

6 CONCLUSIONS

The framework audits privacy risk for individual records using membership inference, guarantees false-positive-rate confidence over fine-grained non-member worlds, and derives progressively stronger attacks by exposing attacker uncertainties.

  • The framework audits machine-learning privacy risk about individual data records through membership inference attacks.
  • Its attacks provide FPR confidence over fine-grained non-member out worlds.
  • The framework derives increasingly strong attacks by identifying uncertainties that limit attacker performance.

A DETAILED DERIVATION OF APPROXIMATED LRT FOR MEMBERSHIP INFERENCE

The appendix derives an approximated likelihood-ratio test for membership inference, using posterior approximations and assumptions about large datasets and i.i.d. samples.

  • For large datasets of i.i.d. samples, the expected posterior with n samples is approximated by the expected posterior with n−1 samples.
  • This posterior approximation is justified by convergence of the trained-model posterior as the number of training samples tends to infinity.
  • The derivation begins with two membership hypotheses over jointly sampled target models and data points.
  • The approximation treats the likelihood-ratio statistic as a decision quantity with a threshold c shared across models and data points.
  • The likelihood-ratio test is approximated using a Bayesian posterior distribution for the trained model.The derivation explicitly invokes a posterior approximation for models trained on private datasets.
  • The null hypothesis is rejected when the likelihood-ratio statistic is sufficiently small.

B EXPERIMENTAL SETUP

The experiments train target models across image, purchase, and handwritten-digit datasets using specified architectures, optimizers, dataset sizes, and regularization settings.

  • All models use categorical cross-entropy loss, with target training datasets sampled without replacement except for CIFAR10 setup IV.CIFAR10 setup IV uses Poisson sampling and trains on 25000 data points in expectation.
  • Purchase100 uses a four-layer MLP with units [512, 256, 128, 64], SGD, and 2500 or 10000 training data points.Setup IV additionally uses gradient clipping with ℓ2 norm 2.0.
  • CIFAR10 uses AlexNet with Adam in setups I and II, a three-block VGGNet with SGD, momentum, and L2 regularization in setup III, and a Wide ResNet with Adam in setup IV.Setup III uses λ=0.001; setup IV has depth 28 and width 2.
  • CIFAR100 and MNIST use two-layer CNNs with [32, 64] filters and max pooling, trained with SGD on 2500 or 5000 data points.

B.2 Details about training shadow, reference or distilled model and population records

The experimental setup trains shadow, reference, and distilled models alongside population records, then constructs attack thresholds from smoothed loss distributions and reports attack AUC across datasets and configurations.

  • Each target configuration uses shadow, reference, and distilled models trained with the target’s structure, algorithm, and dataset size on random i.i.d. sub-splits.
  • Purchase100 attacks use 999 auxiliary models in setups I–III and 29 in setup IV, with 1000 population points sampled per class for Attack P.
  • CIFAR10 uses 399 auxiliary models in setups I–II, 29 in setup III, and 15 in setup IV; Attack P is not launched for setup IV.
  • CIFAR100 and MNIST sample 400 and 1000 population points per class, respectively, for Attack P.
  • Attack thresholds use smoothed loss-distribution percentiles to support small-FPR requirements below 1/(N+1).The setup evaluates linear interpolation, logit rescaling, their minimum, and averaged confidence methods.
  • The averaged-confidence method combines linear-interpolation and logit-rescaling CDF estimates, whose inverse determines thresholds for a chosen α.The two smoothing methods receive equal weight.

C.1 Illustrations of Attack Thresholds on Different Targets

The experiments examine attack results across different target models and data records, showing a general trend toward reduced uncertainty and increased performance from Attacks S and P to R and D.

  • C.1 Illustrations of Attack Thresholds on Different Targets: Attack thresholds are reported across randomly chosen target models and target data records to expose their variation.The analysis covers Purchase100 setup II, using 10 target models for most attacks and 3 for Attack D.
  • C.1 Illustrations of Attack Thresholds on Different Targets: Attacks R and D generally reduce attacker uncertainty and improve performance relative to Attacks S and P across datasets and training algorithms.This trend is reported across different training setups and datasets.

C.2 Additional Empirical Results for Evaluating Attack Performance

Additional experiments show that the relative strength of Attacks S, P, R, and D is broadly consistent across datasets, while absolute performance and attack gaps vary substantially.

  • C.2 Additional Empirical Results for Evaluating Attack Performance: Attack R and D generally outperform Attack S and P across Purchase100 configurations, even as privacy risk decreases with larger training datasets.The comparison covers multiple training configurations and varies training-dataset size.
  • C.2 Additional Empirical Results for Evaluating Attack Performance: On CIFAR100 and MNIST, attack-performance gaps are smaller: all attacks succeed on CIFAR100, while all perform poorly on MNIST.Attack R and D still slightly outperform S and P in 3 of 4 setups.
  • C.2 Additional Empirical Results for Evaluating Attack Performance: The CIFAR10 comparisons evaluate FPR-TPR tradeoffs and AUC scores across multiple attacks and target models, with fewer target models for Attack D.Attacks S, R, and D use 29 models of their respective types, while Attack P uses 400 population points per class.
  • C.2 Additional Empirical Results for Evaluating Attack Performance: Figure 10 compares Attacks R and D with three concurrent-attack variants using matched counts of reference, distilled, and shadow models.The evaluated FPR and TPR are averaged over member and non-member records of one target model.
  • C.2 Additional Empirical Results for Evaluating Attack Performance: MNIST AUC results cover configurations trained on 2,500 and 5,000 data points.

C.3 Additional Experiment Results for Worst-case Vulnerable Record

Additional results examine another randomly selected worst-case vulnerable record and relate its vulnerability to the behavior of latent neighboring records.

  • C.3 Additional Experiment Results for Worst-case Vulnerable Record: A second randomly chosen worst-case vulnerable record is analyzed to complement the earlier latent-neighbor vulnerability results.The analysis uses a record for which Attack R and D predictions were correct.

C.4 Details about comparison with concurrent work [2]

The comparison and cost studies show that reference- and distilled-model attacks can outperform concurrent attacks, while their performance responds differently to computational resources and remains stronger under gradient clipping.

  • C.4 Details about comparison with concurrent work [2]: Attacks R and D achieve higher AUC scores than all three concurrent-attack variants under matched counts of 29 models.Attack R also has similar TPR at small FPR, while Attack D achieves small-FPR performance advantages described in the comparison.
  • C.4 Details about comparison with concurrent work [2]: Attack-threshold distributions vary across target models and records, reflecting different threshold dependencies among Attacks S, P, R, and D.The threshold analysis complements the attack-performance comparisons.
  • C.4 Details about comparison with concurrent work [2]: Increasing reference or distilled models substantially improves AUC for Attacks R and D, whereas more shadow models or population points yield only slight gains for S and P.The computation-cost study varies model counts from 10 to 200 and population points per class from 100 to 1,000.
  • C.4 Details about comparison with concurrent work [2]: Gradient clipping lowers attack performance overall, but the attacks retain their increasing-strength ordering under this heuristic defense.The authors attribute this persistence to deriving the attacks without assumptions about training or defense algorithms.
Loading 2111.09679v4…