PaperlayerView exact revision on arXiv

Source-linked AI summary

Label-Only Membership Inference Attacks

Christopher A. Choquette-Choo, Florian Tramer, Nicholas Carlini, Nicolas Papernot

arXiv:2007.14321v3cs.CRcs.LGstat.ML

TL;DR

Membership inference threatens privacy by exposing whether sensitive data was used to train a model, but existing attacks depend on confidence scores unavailable in label-only settings. The paper replaces confidence with robustness to perturbations, showing that label-only attacks match confidence-based attacks and defeat confidence-masking defenses. It identifies differential privacy and strong L2 regularization as effective defenses within the reported scope.

  • Problem

    Existing membership inference attacks rely on abnormal training-data confidence, so they do not apply when an adversary receives only predicted labels.

  • Method

    The paper infers membership from predicted-label robustness under data augmentations and adversarial perturbations, using decision-boundary distance as a confidence proxy.

  • Results

    Label-only attacks perform on par with confidence-based attacks, break confidence-masking defenses, and also match confidence-based attacks for worst-case outliers.

  • Takeaways & Limitations

    Confidence-masking is not a viable defense against membership inference; differential privacy and strong L2 regularization are the only known strategies reported to prevent all attacks.

Abstract

from arXiv · show

Membership inference attacks are one of the simplest forms of privacy leakage for machine learning models: given a data point and model, determine whether the point was used to train the model. Existing membership inference attacks exploit models' abnormal confidence when queried on their training data. These attacks do not apply if the adversary only gets access to models' predicted labels, without a confidence measure. In this paper, we introduce label-only membership inference attacks. Instead of relying on confidence scores, our attacks evaluate the robustness of a model's predicted labels under perturbations to obtain a fine-grained membership signal. These perturbations include common data augmentations or adversarial examples. We empirically show that our label-only membership inference attacks perform on par with prior attacks that required access to model confidences. We further demonstrate that label-only attacks break multiple defenses against membership inference attacks that (implicitly or explicitly) rely on a phenomenon we call confidence masking. These defenses modify a model's confidence scores in order to thwart attacks, but leave the model's predicted labels unchanged. Our label-only attacks demonstrate that confidence-masking is not a viable defense strategy against membership inference. Finally, we investigate worst-case label-only attacks, that infer membership for a small number of outlier data points. We show that label-only attacks also match confidence-based attacks in this setting. We find that training models with differential privacy and (strong) L2 regularization are the only known defense strategies that successfully prevents all attacks. This remains true even when the differential privacy budget is too high to offer meaningful provable guarantees.

1. Introduction

Membership inference exposes whether sensitive data contributed to model training, while prior attacks rely on confidence scores unavailable through label-only interfaces. The paper introduces label-only attacks that use perturbation robustness and challenges confidence-masking defenses.

  • Membership inference can reveal sensitive facts about individuals by identifying whether their data appeared in training.The paper illustrates this with medical data: inferring training-set membership can reveal that a victim has cancer.
  • Existing attacks infer membership from higher model confidence on training points, a signal largely attributed to overfitting.
  • Confidence-masking defenses alter confidence scores to thwart existing attacks but leave predicted labels unchanged.
  • Label-only attacks infer membership from robustness to perturbations, requiring only predicted labels rather than confidence scores.They query augmentations and adversarial examples to extract a finer-grained membership signal.
  • The paper reports that label-only attacks match confidence-vector attacks, break two confidence-masking defenses, and identify effective defenses as differential privacy and strong L2 regularization.It also reports that data augmentation can worsen leakage, whereas transfer learning can mitigate it.

2. Background and Related Works

Prior membership inference work commonly assumes black-box access to confidence vectors and uses shadow models or thresholds to classify membership. Related studies examine querying related inputs, adversarial robustness, and approximating white-box attacks, but use confidence scores or different methodologies.

  • Most prior black-box attacks query part or all of a model’s confidence vector and train a membership classifier under assumptions about training data.
  • Shadow models provide labeled member and non-member examples for training attack classifiers, though later work showed similar-task data can suffice without them.
  • Related-input queries and adversarial robustness studies improve or analyze membership inference but assume access to confidence scores.
  • Black-box attacks can approximate white-box attacks by effectively estimating model loss for a data point.

3. Attack Model Design

The proposed attacks operate with label-only black-box access and estimate confidence-like membership signals from robustness to strategic perturbations. They use augmentations, adversarial examples, or random noise to assess distance from the decision boundary.

  • The attacks combine multiple strategically perturbed queries while operating in a label-only regime, threatening any queryable machine-learning service.
  • Because a single query reveals only the predicted class, the gap attack classifies misclassified points as non-members and additional queries are needed for finer membership inference.
  • 3.2. Attack Intuition: The method treats robustness to synthetic or adversarial perturbations as a label-only proxy for model confidence and decision-boundary distance.
  • Augmentation attacks train a classifier on shadow models, create perturbed versions of a target point, and transfer the classifier to the black-box target model.
  • 3.4. Decision Boundary Distance: Membership is predicted when estimated boundary distance exceeds a threshold, with misclassified points assigned distance zero.
  • Label-only boundary estimates use black-box adversarial attacks such as HopSkipJump or robustness under random noise.

4. Evaluation Setup

The evaluation reproduces prior experimental settings across eight datasets and measures membership inference on balanced member and non-member samples. It tests parity with confidence-based attacks, defense robustness, query complexity, and defenses that block both attack types.

  • The evaluation uses an identical setup to prior work and examines whether label-only attacks match confidence-vector attacks and defeat their defenses.
  • The study also evaluates query complexity and asks which defenses prevent both label-only and full confidence-vector attacks.
  • Attack success is measured by overall membership-inference accuracy on balanced samples, with F1 scores approximately matching and recall near 100%.
  • The paper’s goal is to show that label-only access is sufficient where membership inference already succeeds, while acknowledging that it generally uses less query information than confidence-based attacks.
  • The experiments cover 8 datasets, including 3 computer-vision tasks and 4 non-computer-vision tasks, using target networks trained on subsets of the original data.

5. Evaluation of Label-Only Attacks

Label-only attacks consistently outperform the gap baseline and match or exceed confidence-vector attacks across image and non-image settings. They also remain effective against confidence-masking defenses, while query-efficient perturbation choices materially affect attack performance.

  • 5.1. Label-Only Attacks Match Confidence-Vector Attacks: Label-only attacks consistently outperform the gap attack and perform on-par with prior confidence-vector attacks, while combining attacks can surpass them.The boundary-distance attack reaches at least confidence-vector performance; combining boundary and data-augmentation attacks consistently outperforms every other attack.
  • Beyond Images: Using Gaussian or Bernoulli perturbations, label-only attacks on Texas, Purchase, Location, and Adult closely match confidence-based attacks, with at most 4 percentage-point degradation.The noise-robustness attack uses 10,000 queries where suitable augmentations may not exist.
  • 5.2. Breaking Confidence Masking Defenses: Confidence masking protects against existing confidence-vector attacks but has little to no effect on label-only attacks because predicted labels remain unchanged.MemGuard and prediction purification explicitly preserve the model’s predicted labels while modifying confidence information.
  • 5.3. Breaking MemGuard: MemGuard successfully defends against prior confidence-vector attacks but offers no protection against label-only attacks, which significantly outperform the nonadaptive confidence-vector and gap attacks.The evaluation uses a strong MemGuard variant that can arbitrarily modify confidence vectors without changing predicted labels.
  • 5.3. Breaking Adversarial Regularization: Adversarial regularization can bring the confidence-vector attack within 3 percentage points of chance, yet label-only attacks still significantly outperform it because the train-test gap is only marginally reduced.The defense is therefore not entirely ineffective, but it does not eliminate label-only leakage.
  • 5.5. The Query Complexity of Label-Only Attacks: Optimal rotation and translation magnitudes exceed the gap baseline by 3-4 percentage-points, while boundary-distance attacks match a white-box upper bound with approximately 2,500 queries.Too-small or too-large perturbations affect train and test samples similarly; with approximately 12,500 queries, the combined attack can outperform all others.

6. Defending with Better Generalization

The section evaluates data augmentation, transfer learning, and regularization as defenses against membership inference. Better generalization does not necessarily reduce leakage, while strong L2 regularization and differential privacy reduce MI at substantial utility cost.

  • Training with Data Augmentation Exacerbates MI: Data augmentation can increase membership leakage despite improving generalization.With d = 5 pixel translations, test accuracy rises from 49.7% to 58.7%, while the data augmentation attack becomes stronger.
  • Training with Data Augmentation Exacerbates MI: Augmentation-based attacks match or outperform confidence-vector attacks when training uses augmentations.The attacks mimic the training augmentations; the label-only distance attack remains on par with the confidence-vector attack.
  • Other Techniques to Prevent Overfitting: Transfer learning reduces leakage under last-layer fine-tuning but leaks more under full fine-tuning.Last-layer fine-tuning reduces attacks to the generalization gap, whereas full fine-tuning produces more leakage alongside higher test accuracy.
  • Other Techniques to Prevent Overfitting: Most regularization methods fail to prevent baseline gap attacks from reaching 60% accuracy or more.Only strong L2 regularization with λ ≥1 and differential privacy consistently reduce MI, and both impose a high test-accuracy cost through underfitting.
  • Other Techniques to Prevent Overfitting: Combining differential privacy with transfer learning provides the best privacy-utility tradeoff.The combination mitigates privacy leakage at minimal test-accuracy cost; without transfer learning, dropout performs better.

7. Worst-Case (Outlier) MI

The paper evaluates worst-case membership inference by targeting unusual outlier points rather than representative data. Targeting outliers improves attacks, while strong L2 regularization and differential privacy prevent membership inference; the label-only boundary-distance attack matches the confidence-vector attack.

  • Worst-case attacks target unusual data points that may be overfit despite good average generalization.Potential outliers are identified using a modified process based on prior work.
  • The adversary maps target points into a source model’s penultimate-layer feature space and identifies outliers by local feature density.Points are neighbors when their cosine distance is at most δ; an outlier has fewer than γ neighbors, with δ and γ tuned to select fraction β.
  • Outlier selection consistently improves membership-inference attacks, with precision used as the success metric.The attack evaluation uses the same models as Figure 6 and targets a chosen fraction of potential targets.
  • Strong L2 regularization and differential-privacy training prevent membership inference in the outlier setting.This result is reported alongside the finding that label-only boundary-distance attacks match confidence-vector attack performance.
  • The label-only boundary-distance attack matches the performance of the confidence-vector attack on outliers.The comparison is made under the same setup as the defense experiments.

8. Conclusion

The paper develops label-only membership inference attacks that match confidence-based attacks and defeat confidence-masking defenses. It finds that differential privacy, often with transfer learning, provides the strongest evaluated defense, although it may reduce test accuracy.

  • Three label-only attacks match or exceed prior confidence-vector attacks despite using a more restrictive adversarial model.Their different attack strategies cannot be trivially prevented by obfuscating confidence scores.
  • Confidence-masking defenses cannot prevent label-only attacks, so effective defenses must reduce the model’s train-test gap.The paper specifically reports breaking two state-of-the-art defenses and concludes that confidence masking is not viable.
  • Differential privacy with transfer learning provides the strongest defense in both average-case and worst-case evaluations, but may reduce test accuracy.
  • The study centers on the prior threat model and leaves label-only attacks under reduced adversarial knowledge for future work.The assumed setting includes knowledge of the task, model architecture, training setup, and related data information.

A.1.1. DATA AUGMENTATION

Data augmentation creates transformed samples that preserve class semantics and can improve classifier generalization without acquiring additional labeled data. For image classifiers, common transformations include geometric changes, contrast adjustments, cutout, and blurring; transfer learning similarly improves generalization in low-data settings.

  • Augmentations transform existing data while preserving class semantics, increasing training-set diversity without acquiring more labeled data.
  • Image-classifier augmentations include rotations, reflections, scaling, shifts, contrast adjustments, cutout, and blurring.
  • Data augmentation can teach a model to become invariant to the applied transformations.
  • Softmax outputs are treated as scores rather than rigorously interpreted probabilities, despite lying in [0, 1] and summing to 1.
  • Transfer learning improves generalization in low-data regimes by transferring knowledge from a source task to a target task and then fine-tuning.

A.2. Membership Inference

Membership inference determines whether a sample was used to train a model, traditionally using confidence signals or shadow models. Label-only approaches instead use predicted labels and related inputs, while defenses either reduce overfitting or mask confidence information.

  • Membership Inference: Membership inference attacks classify whether a sample belonged to a model’s training dataset.
  • Prior Attacks: Prior black-box attacks commonly query confidence vectors, with shadow models or tuned confidence thresholds used to construct membership decisions.
  • Label-only Approaches: The gap attack predicts membership from correct classification, with success tied directly to the train-test accuracy gap.
  • Label-only Approaches: The paper’s label-only attacks query predicted labels on perturbed versions of a target point, extending beyond the prior predicted-label baseline.
  • Defenses: Defenses address membership inference either by reducing overfitting or by modifying confidence information during training or inference.Heavy regularization can limit leakage but may degrade accuracy; confidence masking changes scores without necessarily changing labels.
  • Outliers in Membership Inference: Outlier membership inference targets a small number of unusual points, which prior work reports identifying with precision up to 90%.

B. Evaluation Setup

The evaluation uses a threat model aligned with prior work while restricting the adversary to label-only queries. The adversary retains broad task, model, training-setup, data-distribution, and target-label knowledge.

  • The evaluation matches prior threat models except that the adversary is restricted to label-only queries.
  • The adversary is assumed to know the task, model architecture, training setup, a disjoint same-distribution data partition, and target-point labels.

B.1. Our Threat Model

The paper studies membership inference under black-box, label-only access, where adversaries observe predicted labels but not confidence scores. It specifies assumptions about task, training, data, and label knowledge, and uses perturbation-based queries to extract additional membership signal.

  • Threat-model assumptions: The evaluation assumes balanced priors and uses disjoint training and test subsets for target and source models.The target and source models use separate subsets of the task data pool, with subsets matched in size under balanced priors.
  • Adversary knowledge: The adversary may know task information such as class labels and input format, while training knowledge concerns architecture and training procedures.Training details may be publicly available or inferable through model extraction attacks.
  • Attack interface: Beyond the naive label-only baseline, attacks query labels on augmented inputs or adversarial examples to analyze robustness and decision-boundary geometry.Augmentations include transformations such as image translations, while the table also identifies decision-boundary distance as an attack signal.
  • Adversary knowledge: Data knowledge ranges from access to the training data, which makes membership inference trivial, to samples from the same or a related distribution.The threat model assumes candidate samples have already been found and leaves efficient discovery through label-only queries for future work.
  • Query interface: Label-only access returns only the predicted label, making it the most restrictive query interface for a queryable model.The interface is also realistic when deployed systems expose only model-driven actions rather than confidence scores.

F. Additional Figures

The additional figures examine how label-only attack performance changes with shadow-model count, augmentation scale, defense settings, transfer learning, and outlier detection. They report stable accuracy across shadow-model counts, augmentation-matched optima, and several defense and training configurations.

  • Shadow models: The number of shadow models does not significantly affect label-only attack accuracy on CIFAR-10 models trained with 1000 points.The figure varies the number of shadow models for both target and source models.
  • Data augmentation: Translation-attack accuracy is maximized when the evaluated translation size d matches the translation-augmentation size used during training.The target models are trained on 2500 CIFAR-10 points with varied augmentation sizes.
  • Adversarial Regularization: Setting k = 1 enables the target model to converge to a defended state under Adversarial Regularization.The figure varies λ, which balances low training error against low membership leakage, for target models trained on 2500 images.
  • Transfer learning: Transfer learning is evaluated by training the source model on all CIFAR-100 data and tuning target models on CIFAR-10 subsets.The figure reports membership-inference accuracy for these transfer-learning models.
  • Outlier attacks: Outlier attacks identify β = 2% of outliers using fewer than γ = 10 neighbors and report precision improvement over the undefended model.The comparison uses the label-only boundary-distance attack on models trained with 2500 CIFAR-10 points.
Loading 2007.14321v3…