PaperlayerView exact revision on arXiv

Source-linked AI summary

Low-Cost High-Power Membership Inference Attacks

Sajjad Zarifzadeh, Philippe Liu, Reza Shokri

arXiv:2312.03262v3stat.MLcs.CRcs.LG

TL;DR

Membership inference attacks measure whether a particular point was used to train a model, but prior attacks can be costly and unstable across settings. RMIA composes likelihood-ratio tests over fine-grained replacement worlds; the paper reports that it outperforms prior attacks across settings, including with few reference models.

  • Problem

    Prior attacks can be impractical under limited computation and show performance instability across reference-model budgets, data distributions, and out-of-distribution evaluation.

    Source: §1 · p. 1

  • Method

    RMIA composes pairwise likelihood-ratio tests over null-hypothesis worlds in which the target point is replaced by population samples.

    Source: §1 · p. 1, §3 · p. 3

  • Results

    With one reference model, RMIA achieves at least 24% higher AUC than LiRA across datasets, and it consistently outperforms other attacks in tested settings.

    Source: §1 · p. 2, §5.1 · p. 6

  • Takeaways & Limitations

    The authors conclude RMIA can be reliably used to audit privacy risks in machine learning under realistic practical assumptions.

    Source: §6 · p. 9

Abstract

from arXiv · show

Membership inference attacks aim to detect if a particular data point was used in training a model. We design a novel statistical test to perform robust membership inference attacks (RMIA) with low computational overhead. We achieve this by a fine-grained modeling of the null hypothesis in our likelihood ratio tests, and effectively leveraging both reference models and reference population data samples. RMIA has superior test power compared with prior methods, throughout the TPR-FPR curve (even at extremely low FPR, as low as 0). Under computational constraints, where only a limited number of pre-trained reference models (as few as 1) are available, and also when we vary other elements of the attack (e.g., data distribution), our method performs exceptionally well, unlike prior attacks that approach random guessing. RMIA lays the groundwork for practical yet accurate data privacy risk assessment in machine learning.

1. Introduction

Prior membership inference attacks can be costly and unstable across evaluation settings, limiting their practicality for privacy auditing. RMIA uses a fine-grained statistical test and outperforms prior attacks across benchmark datasets and varied settings, including with few reference models.

1. Introduction

  • Prior attacks can be impractical under limited computation and vary in performance across reference-model budgets, data distributions, model similarity, and out-of-distribution data.These limitations motivate attacks that remain efficient and robust across settings.Source: §1 · p. 1
  • RMIA composes likelihood-ratio tests over fine-grained null-hypothesis worlds, where the target point is replaced by population samples.The paper also frames many prior attacks as simplifications of this approach.Source: §1 · p. 1
  • RMIA outperforms prior attacks throughout the TPR-FPR curve on CIFAR-100 with one offline reference model, including at zero FPR versus LiRA.The figure caption reports at least 25% higher AUC and an order of magnitude better TPR at zero FPR than LiRA.Source: Figure 1 · p. 2
  • Across benchmark datasets, RMIA achieves significantly higher AUC and 2× to 4× higher TPR at low FPRs with only 1 or 2 reference models.The advantage is particularly evident when using independently pre-trained reference models.Source: §1 · p. 2
  • The paper reports that RMIA consistently outperforms other attacks even in worst-case robustness scenarios.These evaluations vary data distributions and model architectures, among other settings.Source: §1 · p. 2

2. Performing Membership Inference Attacks

Membership inference is framed as a hypothesis test: determine whether a data point was used to train a model. An attack scores the model-point pair and thresholds that score to make a membership prediction, with performance assessed by the TPR-FPR trade-off.

2. Performing Membership Inference Attacks

  • The attack distinguishes H_in, where the target model was trained on x, from H_out, where x was not in its training set.The adversary receives x and the target model, along with background knowledge about the training algorithm and population distribution.Source: §2 · p. 2
  • The adversary computes a membership score from the target point, model, and population distribution, then predicts membership by comparing the score with threshold β.The game randomly selects whether the target point is drawn from the training set or from the population outside that set.Source: §2 · p. 2
  • Attack performance is measured by TPR-FPR trade-off curves, alongside computational efficiency and stability under changes to data distribution and computation budget.The threshold β controls the false-positive error the adversary is willing to tolerate.Source: §2 · p. 2

3. Designing RMIA

RMIA models non-membership as a set of possible worlds in which the target point is replaced by a random population point, then composes pairwise likelihood-ratio tests. Its score measures how often a population sample passes the test, and thresholds determine the membership decision.

3. Designing RMIA

  • RMIA models the null as replacement worlds in which x is swapped for population samples z, then composes pairwise likelihood-ratio tests to distinguish membership from non-membership.This provides a finer-grained analysis than relying on an average likelihood for the null hypothesis.Source: §3 · p. 3
  • The likelihood Pr(θ|x) is computed over training-algorithm randomness, while Pr(x|θ) is the model likelihood evaluated on x.For classification in the black-box setting, Pr(x|θ) is the model's prediction score for x's class.Source: §3 · p. 3
  • In offline attacks, RMIA estimates Pr(x) by averaging predictions over OUT reference models and scales this estimate to approximate the IN-model contribution.The offline approach avoids training customized reference models for each query; the same computation applies to z.Source: §3 · p. 3
  • RMIA's score is the fraction of population samples z that pass the pairwise likelihood-ratio test at threshold γ, and the attack labels x a member when the score reaches β.For γ ≥ 1, γ sets how much larger the probability under x must be than under a random alternative z to pass.Source: §3 · p. 3
  • At γ = 1, RMIA's expected FPR is 1 − β, and small changes to γ leave performance relatively stable except at considerably high γ.The paper notes that adjusting β can yield roughly the same result across different γ values.Source: §3 · pp. 3–4

4. Why is RMIA a More Powerful Test Compared with Prior Attacks?

RMIA combines population data and reference models in a calibrated likelihood-ratio test, capturing signals that prior attacks can overlook. Its Bayesian computation strictly dominates LiRA across the TPR-FPR curve, especially with few reference models.

4. Why is RMIA a More Powerful Test Compared with Prior Attacks?

  • RMIA leverages both population data and reference models, contrasting probability shifts for x and z to capture signals neglected by prior attacks.The test compares Pr(x|θ)/Pr(x) with Pr(z|θ)/Pr(z), rather than assuming Pr(x) = Pr(z).Source: §4 · p. 4
  • A reference point z votes for x only when the target model enlarges Pr(x)/Pr(z), indicating that it fits x better.Source: §4 · p. 4
  • Reference-model attacks can miss population calibration: Ye et al. (2022) neglect Pr(z|θ)/Pr(z), while LiRA’s score is an average-case version of RMIA (Carlini et al., 2022).The passages attribute LiRA’s limitations both to missing population-data calibration and to its average-case computation.Source: §4 · p. 4
  • In the black-box setting, evaluating model outputs on x and z provides an approximation to the likelihood ratio.The passage motivates this approximation by noting that output-distribution divergence is greatest on the point distinguishing a model from its leave-one-out counterpart.Source: §4 · p. 4
  • RMIA strictly dominates LiRA throughout the TPR-FPR curve, with a larger gap under tight reference-model budgets (Carlini et al., 2022).The Bayesian construction dominates direct likelihood-ratio computation with few reference models and matches it when many are available.Source: §4 · p. 4

5. Empirical Evaluation

The evaluation compares RMIA with prior attacks across benchmark datasets, reference-model budgets, OOD queries, distribution and architecture shifts, parameter settings, and GBDT models.

5.1. Inference Attack under Low Computation Budget

  • Using a single reference model, RMIA achieves at least 24% higher AUC than LiRA across all datasets, and it also outperforms the Quantile-Regression attack in both AUC and TPR.In this setting, RMIA also has better TPR at low FPRs than LiRA.Source: §5.1 · p. 6

5.2. Ultimate Power of (Online) Inference Attacks

  • Online RMIA beats LiRA Online by up to 50% in zero-FPR TPR, while offline RMIA achieves performance comparable to online attacks.With 18 augmented queries, RMIA also achieves 4× zero-FPR TPR and 4.6% higher AUC than LiRA.Source: §5.2 · p. 7, Table 3 · p. 8

5.3. Dependency on Availability of Reference Models

  • RMIA stays close to its maximum AUC with few reference models; LiRA Online has an approximately 8% AUC gap between 2 and 254 CIFAR-10 models.RMIA Offline is stronger than LiRA Online unless hundreds of reference models are used.Source: Figure 3 · p. 7, §5.3 · p. 8

5.4. Robustness of Attacks against OOD Non-members

  • With CINIC-10 OOD nonmembers, RMIA exceeds Attack-R by at least 21% AUC; Attack-P’s high AUC does not yield good low-FPR TPR.LiRA performs little better than random guessing in this setting.Source: §5.4 · p. 8, Figure 4 · p. 9

5.5. Robustness to Model and Data Distribution Shift

  • With two reference models, RMIA gains up to 25% AUC over LiRA under dataset shift; architecture shifts widen its AUC gap over other attacks by up to 3%.The dataset-shift comparison uses CIFAR-10 target models and CINIC-10 reference models.Source: §5.5 · p. 9

5.6. Analyzing RMIA Parameters

  • Using 2,500 z samples (10% of training-set size) gives results comparable to a 10× larger population; low-FPR TPR remains high with 250 samples.AUC and the FPR-TPR curve remain relatively stable under small changes to γ, except at considerably high values.Source: §5.6 · p. 9, Table 4 · p. 9

5.7. Applying RMIA to Other ML Algorithms

  • Against Gradient Boosting Decision Trees, RMIA consistently outperforms other attacks, with TPR at zero FPR higher by an order of magnitude.The comparison is reported in Figure 15 in Appendix C.6.Source: §5.7 · p. 9

6. Conclusions

The paper argues that membership-inference tests should be stress-tested under limited computation, few reference models, and shifts in data or models. It presents RMIA and evaluation scenarios intended to assess efficiency and robustness under practical assumptions.

6. Conclusions

  • The authors argue that MIA tests should be evaluated under low computation budgets, few reference models, and changes to data distributions and models.They present RMIA as a privacy-auditing method for realistic practical assumptions.Source: §6 · p. 9

A. Experimental Setup

The experimental setup evaluates attacks across several datasets using ROC-based metrics, while RMIA compares samples through likelihood ratios and can operate online or offline. For offline RMIA, the paper approximates the normalizing probability using existing OUT reference models.

A. Experimental Setup

  • Models are evaluated on CIFAR-10, CIFAR-100, CINIC-10, ImageNet, and Purchase-100, with dataset-specific training setups.The setup includes Wide ResNets, ResNet-50, and a four-layer MLP; Table 5 reports average model accuracy by dataset.Source: A. Experimental Setup · p. 15, Table 5 · p. 15
  • Attack performance is measured by TPR-FPR curves and AUC, with particular attention to TPR at 0% FPR.The evaluation samples members and non-members from the population data for random target models.Source: A. Experimental Setup · p. 15

B.1. Pseudo-code of RMIA

  • RMIA compares target-to-reference prediction-probability ratios for a test sample and population samples, then scores the fraction of samples γ-dominated by the test sample.Online RMIA trains IN reference models per query; offline RMIA uses pre-trained OUT models and approximates Pr(x).Source: §B.1 · pp. 15–16

B.2. Likelihood Ratio Computation in equation 3

  • SM-Taylor-Softmax has a slightly higher AUC and better low-FPR TPR than the other tested confidence functions, so it is used for CIFAR and CINIC-10.The selected settings are m = 0.6, n = 4, and T = 2 for CIFAR-10; AUC is robust to parameter variations, with consistent results for n ≥ 3.Source: §B.2 · p. 17, Table 6 · p. 17, Figure 5 · p. 18

B.2.2. COMPUTING Pr(x)

  • Offline RMIA approximates Pr(x)IN from Pr(x)OUT with a linear function; on CIFAR-10, a = 0 avoids very low Pr(x)OUT for atypical or hard samples and still improves results.As a approaches 1, AUC degrades by up to 5.5%; the paper selects a per dataset using existing models without training new ones.Source: §B.2.2 · p. 19

B.3. Direct Computation of Likelihood Ratio in equation 2

The paper compares Bayesian and direct likelihood-ratio computation, finding the Bayesian approach more suitable when reference models are scarce. It also reports that RMIA is relatively stable across γ choices when β is adjusted, while very high γ can make detection difficult.

B.3. Direct Computation of Likelihood Ratio in equation 2

  • RMIA-direct models logits with Gaussians across two reference-model distributions, but requires online reference-model training to compute probabilities.The paper notes this computation is significantly more costly than the Bayesian approach.Source: §B.3 · p. 19
  • With four reference models, RMIA-Bayes performs better across datasets, including 6.6% higher AUC on CIFAR-10, while RMIA-direct incurs substantially higher processing cost.With 64 models, RMIA-direct slightly outperforms on AUC and low-FPR TPR, while RMIA-Bayes closely matches it.Source: §B.3 · pp. 19–20

B.4. Analyzing γ and β Parameters and their Relation

  • RMIA performance remains relatively stable as γ increases when β is adjusted, but detecting γ-dominated samples becomes difficult at very high γ.The experiments use γ = 2, which yields slightly higher TPR@FPR.Source: §B.4 · p. 20, Figure 7 · p. 20
  • As β increases toward 1, both TPR and FPR approach zero, but FPR declines faster, especially at γ = 2; at γ = 1, FPR decreases proportionally to 1 − β.β is the threshold used to generate the ROC curve.Source: §B.4 · p. 20

B.5. Boosting RMIA with Augmented Queries

Augmented queries strengthen RMIA, especially in the offline setting; online results also favor RMIA over LiRA as query count increases.

B.5. Boosting RMIA with Augmented Queries

  • Majority voting defines the multi-query test: x dominates z when more than half of x’s generated transformations dominate z.The transformations can be simple shifts or rotations for image data.Source: §B.5 · p. 20
  • The comparison increases augmented queries from 1 to 50 and evaluates online and offline attacks, with results averaged over 10 random target models.Offline attacks use 127 OUT models; online attacks use 254 models, half IN and half OUT.Source: §B.5 · p. 20, Table 7 · p. 21
  • Offline, RMIA leads Attack-R by 6.7% in AUC and 116% in TPR at zero FPR without augmentations; from 1 to 50 queries, its TPR improves 4× and AUC about 4.6%.The offline comparison uses 127 OUT reference models; Attack-P and Attack-R do not support multiple augmented queries.Source: Table 7 · p. 21, §B.5 · p. 21
  • Online, RMIA has slightly higher AUC than LiRA and at least 48% better TPR at zero FPR; both improve TPR about 2× from 1 to 50 queries.The online setting uses all 254 IN and OUT reference models; the paper uses 18 augmented queries by default.Source: Table 7 · p. 21, §B.5 · p. 21

C.1. Robustness of Attacks against Noise and OOD Non-member Queries

RMIA maintains an advantage when non-member queries are OOD or noise, and its FPR is substantially lower for OOD than ID non-members.

C.1. Robustness of Attacks against Noise and OOD Non-member Queries

  • With CINIC-10 OOD queries, RMIA’s AUC is at least 21% higher than other attacks; with noise queries, it is at least 61% higher.LiRA is near random guessing with OOD queries and falls significantly below random guessing with noise queries.Source: Figure 9 · p. 22, §C.1 · p. 22
  • As the OOD share of non-member queries rises, RMIA and Attack-P AUC increase while the other two attacks’ AUC decreases.At zero FPR, the population attack has nearly zero TPR, while RMIA outperforms all other attacks; zero-FPR TPR declines for all attacks as OOD share rises.Source: §C.1 · p. 22, Figure 10 · p. 23
  • At the same β, ID-query FPR can exceed 0.8 while OOD-query FPR is almost zero, as RMIA more readily identifies OOD non-members.As β increases from zero, OOD FPR falls faster than ID FPR and ID TPR.Source: §C.1 · p. 22, Figure 11 · p. 23

C.2. Data Distribution Shift

The section tests offline attacks when target and reference training data differ, and reports RMIA’s advantage under that distribution shift.

C.2. Data Distribution Shift

  • The comparison uses CIFAR-10 target models, CINIC-10 reference models, common class labels, and 1, 2, or 4 reference models.All test queries are from CIFAR-10.Source: §C.2 · p. 23
  • The training-data distribution shift between target and reference models affects the performance of all attacks.Source: §C.2 · p. 23
  • RMIA always achieves higher AUC, by up to 25% versus LiRA, and better TPR at low FPR than the other two attacks.Source: §C.2 · p. 23

C.3. Variations in Neural Network Architectures

RMIA outperforms other attacks across architecture settings, a larger CINIC-10 training set, and a GBDT experiment; the supplied passages also describe a few-reference-model evaluation.

C.3. Variations in Neural Network Architectures

  • With matching target and reference architectures on CIFAR-10, RMIA consistently outperforms other attacks, with 7.5%-16.8% higher AUC than LiRA.The architectures include CNN and Wide ResNet variants.Source: §C.3 · p. 23, Figure 12 · p. 24
  • When reference architectures shift while the target remains WRN28-2, similar target and reference architectures yield optimal attack performance, but RMIA still wins and its gap widens under shifts.Source: §C.3 · p. 24, Figure 13 · p. 25

C.4. Performance of Attacks on Models Trained with a Larger Subset of CINIC-10

  • On CINIC-10 models trained with 90k samples and one OUT reference model, RMIA achieves about 15% higher AUC than LiRA and an order of magnitude higher TPR at 0.01% FPR than Quantile Regression.Quantile Regression has higher AUC than LiRA with one reference model but no true positives at zero FPR.Source: Figure 14 · p. 25, §C.4 · p. 25

C.5. Performance of Attacks using Fewer Reference Models

  • Table 9 reports average results and standard deviations across CIFAR-10, CIFAR-100, and CINIC-10 under a few-reference-model budget.All attacks except LiRA (Online) are low-cost and offline, without per-query reference-model training.Source: §C.5 · p. 25, Table 9 · p. 26

C.6. Performance of Attacks on Models Trained with other ML Algorithms

  • On Purchase-100 models trained with GBDT at three max-depth values and about 53% test accuracy, RMIA consistently exceeds other attacks in TPR, especially by an order of magnitude at zero FPR.The experiment uses two reference models, one IN and one OUT.Source: Figure 15 · p. 26, §C.6 · p. 26

C.7. Performance of Attacks Obtained on Different Datasets

Across datasets and reference-model budgets, RMIA outperforms the compared attacks, especially with few reference models. On Purchase-100, it also exceeds LiRA under offline and online comparisons.

C.7. Performance of Attacks Obtained on Different Datasets

  • With 1 reference model, RMIA works better than the other three attacks across four datasets; LiRA has comparable low-FPR TPR on CIFAR-10 and Purchase-100 but 22% lower AUC on CIFAR-10.On CIFAR-100 and CINIC-10, RMIA achieves around 3 times better TPR at zero FPR than its closest rival.Source: §C.7 · p. 26
  • At 254 reference models, RMIA’s TPR at zero FPR is at least 50% higher than LiRA’s on CIFAR-10, CINIC-10, and Purchase-100, although their AUCs are close.With 127 OUT models, offline RMIA also achieves at least 20% higher AUC than LiRA across datasets.Source: §C.7 · p. 27
  • As reference-model counts rise from 1 to 254, AUC improves for all attacks, while RMIA remains strongest across offline and online settings, particularly with few models.Online settings use half OUT and half IN reference models.Source: §C.7 · p. 27

C.8. Performance of Attacks on Purchase-100 Models

  • With 1 OUT reference model on Purchase-100, RMIA achieves over 30% higher AUC than LiRA; offline RMIA with 2 or 4 references can surpass online LiRA at the same count.Table 10 results average 10 random target models and use no augmented queries; attacks are offline by default, with online LiRA and RMIA also reported.Source: §C.8 · p. 27, Table 10 · p. 27

C.9. Performance of RMIA with Different Number of z Samples

The section reports how RMIA behaves as population-sample counts change, compares its scores with other attacks, and situates the method among membership-inference research and defenses.

C.9. Performance of RMIA with Different Number of z Samples

  • With 1 OUT reference model, RMIA’s AUC increases with more population samples, while 2,500 samples—10% of training-set size—gives results comparable to a population set 10 times larger.The reported comparison is for offline RMIA on CIFAR-10 without augmented queries, averaged over 10 random target models.Source: §C.9 · p. 27, Table 11 · p. 28

C.10. MIA Score Comparison between Attacks

  • With 1 reference model, RMIA separates member and non-member scores more clearly than LiRA and Attack-R; with more models, RMIA and LiRA scores become more correlated.The score comparisons include RMIA versus LiRA with 1 or 4 reference models, and RMIA versus Attack-R with 1 or 4 models.Source: §C.10 · p. 28, Figure 16 · p. 28, Figure 17 · p. 29
  • With 1 OUT reference model, RMIA separates members from non-members more clearly than Attack-P; score distributions across 254 target models also show RMIA members tending toward higher scores.Figure 26 uses 127 OUT reference models trained on CIFAR-10, with each sample evaluated across 254 targets, half IN and half OUT.Source: §C.10 · pp. 28–29, Figure 18 · p. 29, Figure 26 · p. 37

D. Related Work

  • Prior membership-inference work spans genome statistics, shadow-model attacks, low-FPR likelihood-ratio tests, and defenses including regularization and differential privacy.Shadow-model approaches can require many models, while likelihood-ratio attacks can require many references for high performance; the paper positions RMIA as a different test that maintains strong performance with few references and offline use.Source: D. Related Work · pp. 38–39
  • Related work also includes f-MIP bounds and gradient-access auditing, per-point threshold calibration, and low-cost auditing for differentially private learning.The cited low-cost DP-auditing approach uses a single training run and parallel addition or removal of multiple examples.Source: D. Related Work · p. 39
Loading 2312.03262v3…