PaperlayerView exact revision on arXiv

Source-linked AI summary

Membership Leakage in Label-Only Exposures

Zheng Li, Yang Zhang

arXiv:2007.15528v3cs.LGcs.CRstat.ML

TL;DR

Membership inference asks whether a candidate sample was used to train an ML model, but conventional score-based attacks are easily mitigated when only labels are exposed. The paper develops transfer and boundary attacks using label-only access and finds strong performance, including cases where boundary attack outperforms prior score-based attacks. It further links membership status to distance from the model’s decision boundary and evaluates defenses against these attacks.

  • Problem

    Existing membership inference attacks rely on confidence scores, leaving label-only exposure comparatively underexamined despite its relevance to real-world black-box systems.

  • Method

    The paper proposes transfer and boundary membership-inference attacks that use the target model’s predicted labels, with boundary attack requiring no shadow dataset or model.

  • Results

    The attacks achieve strong performance, sometimes outperform prior score-based attacks, and show that decision-boundary distance is strongly correlated with membership status.

  • Takeaways & Limitations

    Label-only model outputs can still leak membership information, and decision-based attacks can bypass most evaluated defenses unless heavy regularization is applied.

  • Takeaways & Limitations

    Boundary attack assumes only final decisions, while transfer attack assumes a shadow dataset drawn from the same distribution as the target dataset; shadow-model architecture knowledge is limited to high-level task knowledge.

Abstract

from arXiv · show

Machine learning (ML) has been widely adopted in various privacy-critical applications, e.g., face recognition and medical image analysis. However, recent research has shown that ML models are vulnerable to attacks against their training data. Membership inference is one major attack in this domain: Given a data sample and model, an adversary aims to determine whether the sample is part of the model's training set. Existing membership inference attacks leverage the confidence scores returned by the model as their inputs (score-based attacks). However, these attacks can be easily mitigated if the model only exposes the predicted label, i.e., the final model decision. In this paper, we propose decision-based membership inference attacks and demonstrate that label-only exposures are also vulnerable to membership leakage. In particular, we develop two types of decision-based attacks, namely transfer attack, and boundary attack. Empirical evaluation shows that our decision-based attacks can achieve remarkable performance, and even outperform the previous score-based attacks in some cases. We further present new insights on the success of membership inference based on quantitative and qualitative analysis, i.e., member samples of a model are more distant to the model's decision boundary than non-member samples. Finally, we evaluate multiple defense mechanisms against our decision-based attacks and show that our two types of attacks can bypass most of these defenses.

1 Introduction

The paper investigates membership leakage when ML models expose only predicted labels, introducing decision-based attacks that remain effective despite restricted access. It proposes transfer and boundary attacks, explains leakage through decision-boundary distance, and evaluates their robustness to defenses.

  • Motivation: Decision-based membership inference uses only a target model’s final predicted label, unlike score-based attacks that require confidence scores.The label-only setting is more realistic where confidence scores are unavailable, but a naive correctness-based baseline cannot distinguish correctly classified members from non-members.
  • Approach: The paper introduces transfer and boundary attacks for membership inference under different adversarial scenarios.Transfer attack uses a relabeled shadow dataset to train a local shadow model, whereas boundary attack operates without shadow data by perturbing candidates until their predicted labels change.
  • Results: Both decision-based attacks achieve strong performance, with boundary attack sometimes outperforming previous score-based attacks.The experiments also report better performance than the baseline attack.
  • Defenses: The attacks bypass most evaluated defenses unless heavy regularization is applied, but heavy regularization can significantly reduce model accuracy.The evaluated defenses include generalization enhancement, privacy enhancement, and confidence-score perturbation.
  • Membership Leakage Analysis: Members are farther from the model’s decision boundary than non-members, providing a quantitative and qualitative perspective on membership-inference success.The boundary attack exploits perturbation magnitude as a proxy for the distance required to cross that boundary.

2 Preliminaries

This section defines membership inference and distinguishes score-based from decision-based threat models. It also specifies the adversarial knowledge, datasets, and model architecture used in the study.

  • Membership Inference: Membership inference determines whether a candidate sample belongs to a trained model’s training set using the candidate, model, and adversary knowledge.The attack outputs a binary member or non-member status.
  • Threat Models: Score-based attacks exploit detailed confidence scores, whereas decision-based attacks rely only on the target model’s final predictions.The label-only baseline predicts membership when the model classifies a sample correctly.
  • Adversarial Knowledge: Transfer attacks require final decisions, a shadow dataset from the target distribution, and a local shadow model; boundary attacks require only final decisions.The adversary has black-box access and cannot obtain membership status from confidence scores.
  • Datasets: Experiments use CIFAR-10, CIFAR-100, GTSRB, and Face benchmark datasets with different sizes and complexities.GTSRB images are resized to 64×64 pixels, and the Face dataset contains 19 classes after filtering.
  • Model Architecture: The target model uses four convolutional layers, four pooling layers, and two final hidden layers containing 256 units each.Models are trained for 200 epochs with Adam, batch size 128, and learning rate 0.001.

3 Transfer Attack

The transfer attack uses a relabeled shadow dataset and local shadow model to infer membership from cross-entropy loss, requiring only label-only target-model access. Its performance is generally strong and improves with larger shadow datasets, while remaining robust to shadow-model differences.

  • Methodology: The transfer attack relabels a same-distribution shadow dataset through the target model, trains a local shadow model, and uses candidate loss for membership inference.The methodology comprises shadow dataset relabeling, shadow model architecture selection, shadow model training, and membership inference.
  • Methodology: Membership is inferred when a candidate’s cross-entropy loss on the shadow model falls below a threshold; evaluation primarily uses threshold-independent AUC.The loss uses the ground-truth one-hot label, class probabilities, and number of classes.
  • Attack AUC Performance: 0.94 AUC versus 0.815 for the baseline appears for the transfer attack on M -5, CIFAR-10, while the transfer attack is at least on-par overall.The transfer attack also outperforms the baseline on CIFAR10 and GTSRB, with larger relative shadow datasets associated with better performance.
  • Effects of the Shadow Dataset and Model: Larger shadow datasets improve attack performance, while even simpler shadow models and fewer shadow samples can still achieve strong performance.The experiments vary shadow-dataset size from 5,000 to 42,000 and shadow-model complexity across parameter and FLOPs ranges.
  • Effects of the Shadow Dataset and Model: The transfer attack remains robust when the shadow model differs substantially from the target model.Target-model complexity in the experiment is 3.84M parameters and 153.78M FLOPs.
  • Effects of Statistical Metrics: Among the evaluated statistical measures, loss achieves the highest AUC, while maximum confidence and entropy have very similar AUC scores.The comparison covers CIFAR-10 and CIFAR-100 target models.

4 Boundary-Attack

The boundary attack performs membership inference using only label changes: it perturbs candidate samples, measures the required perturbation, and classifies membership by a threshold. Larger perturbations for members support strong attack performance across metrics, query budgets, and settings.

  • Attack Setting: The boundary attack operates without a shadow dataset or model, relying only on predicted labels from the target model.It changes candidate decisions through perturbation and infers membership from the perturbation magnitude.
  • Attack Method: The attack pipeline comprises decision change, perturbation measurement, and threshold-based membership inference.HopSkipJump and QEBA change the label; Lp distance measures the perturbation; larger values are classified as members.
  • Perturbation Distribution: 1.0755 versus 0.1102 average L2 distance separates members from non-members in Figure 7 for M -5 on CIFAR-10.The reported averages are for member and non-member samples, respectively.
  • Attack Performance: 0.9033 AUC for L∞ compared with 0.7405 for L0 in Figure 8 (M -1, CIFAR-10), while L1, L2, and L∞ perform best across datasets.The corresponding L1 and L2 AUC scores are 0.8969 and 0.8963.
  • Query Cost: After 2,500 queries, attack performance stabilizes; with 131 queries, AUC reaches 0.8228 on CIFAR-10 and 0.9266 on CIFAR-100.The results indicate that query limiting would likely not be a suitable defense.
  • Comparison: The boundary attack achieves similar or better performance than previous score-based attacks in some cases.This result supports membership leakage under label-only exposure.

5 Membership Leakage Analysis

The analysis explains boundary-attack success through decision-boundary geometry. Members generally lie farther from the boundary than non-members, and this separation transfers between target and shadow models.

  • 5. Membership Leakage Analysis: The attack’s decision-change status contains information about the target model’s decision boundary.Perturbation search stops when the model changes its final decision, linking perturbation magnitude to boundary distance.
  • 5.1 Quantitative Analysis: Certified radius provides a lower bound on the robustness radius, defined as the largest Lp ball around a sample preserving the model’s prediction.The analysis estimates L2 robustness using certified radius and compares members with non-members.
  • 5.1 Quantitative Analysis: The ACR of member samples is generally larger than that of non-members, placing members farther from the decision boundary in output space.The certified radius is an estimated lower bound rather than the exact robustness radius.
  • 5.1 Quantitative Analysis: As overfitting increases, the gap between member and non-member ACR also increases, matching the trend in attack AUC.This connects stronger overfitting with greater boundary-distance separation.
  • 5. Membership Leakage Analysis: The same member–non-member ACR pattern appears in shadow models, supporting transferability of membership information between shadow and target models.The evaluated samples were not used to train the shadow model.
  • 5.2 Qualitative Analysis: Member and non-member samples from the same class form separate clusters, while members lie farther from the decision boundary.The visualization uses transformed output-space representations and t-SNE to show class regions and boundary relationships.
  • 5.2 Qualitative Analysis: Perturbation magnitude reflects distance from the original sample to the decision boundary for HopSkipJump and QEBA searches.Both methods end at tangent samples between the neighboring Lp-radius ball and the decision boundary.

6 Defenses Evaluation

The evaluation tests generalization, privacy, and confidence-score defenses against decision-based attacks. Most defenses do not stop these attacks, while strong regularization or privacy protection reduces leakage at substantial accuracy cost.

  • Defense Categories: The evaluation covers generalization enhancement, privacy enhancement, and confidence-score perturbation defenses.These categories include regularization, dropout, data augmentation, differential privacy, MemGuard, and adversarial regularization.
  • Confidence-Score Perturbation: MemGuard cannot defend against decision-based attacks, although it is effective against previous score-based attacks.The comparison is reported in Table 6 using attack AUC.
  • Overall Results: Decision-based transfer and boundary attacks bypass most tested defense mechanisms.This includes defenses designed to reduce overfitting, add privacy, or perturb confidence scores.
  • Privacy and Regularization: Strong differential privacy and L1/L2 regularization reduce membership leakage but significantly degrade model accuracy.The associated decision boundary is described as heavily blurred.
  • Data Augmentation: Data augmentation reduces overfitting but does not reduce membership leakage in the reported evaluation.The paper attributes this to the model remembering both original samples and their augmentations.

7 Related Works

Prior work established membership inference through confidence scores and explored generalization, differential privacy, and score-alteration defenses. Related attack research also includes adversarial examples and property inference.

  • Membership Inference: Membership inference has been demonstrated across domains including biomedical data and mobility traces.Shokri et al. introduced an attack using shadow models and class-specific attack models based on confidence scores.
  • Defenses Against Membership Inference: Generalization-based defenses use techniques such as L2 regularization, loss penalties, and methods intended to reduce overfitting.These approaches may also decrease model accuracy.
  • Defenses Against Membership Inference: Differential-privacy defenses clip or noise gradients to limit memorization and membership leakage.Prior work also applies differential privacy to collaborative learning of deep neural networks.
  • Defenses Against Membership Inference: MemGuard alters confidence scores with carefully crafted noise to mislead membership classifiers while providing formal utility-loss guarantees.It is presented as a confidence-score alteration defense.
  • Attacks Against Machine Learning: Adversarial-example attacks add crafted noise to samples to mislead target classifiers, while property inference targets training-data properties.These attack types are distinct from membership inference.

8 Conclusion

The paper investigates membership leakage when ML models expose only predicted labels, introducing decision-based attacks and evaluating their effectiveness and defenses.

  • Decision-based attacks use only the target model’s final predicted label and include transfer and boundary attacks.
  • The proposed attacks outperform a baseline and can surpass prior score-based attacks in some cases.
  • Member samples are farther from the model’s decision boundary than non-member samples.
  • The attacks retain reasonable performance against multiple defenses unless heavy regularization is applied.
  • Confidence score perturbation is infeasible as a defense when only labels are exposed.

A.1 Datasets Description

The evaluation uses CIFAR-10, CIFAR-100, GTSRB, and Face datasets with varied image formats, class structures, and train-test partitions.

  • CIFAR-10 contains 50,000 training images and 10,000 test images across 10 classes of 32×32 color images.
  • CIFAR-100 uses the CIFAR format with 100 classes, each containing 500 training images and 100 testing images.
  • GTSRB contains 51,839 traffic-sign images across 43 classes, with 39,209 for training and 12,630 for testing.
  • GTSRB images vary in size and are resized to 64×64 before classification.
  • The Face dataset contains about 13,000 web-crawled face images from 1,680 participants, with the evaluation restricted to 19 classes.

A.2 Certified Radius

The certified-radius appendix applies randomized smoothing to a classifier, defining predictions from Gaussian perturbations and connecting them to a provable robustness radius.

  • Randomized smoothing replaces classifier M with a smoothed classifier G to obtain a certified radius for deep neural networks.
  • For arbitrary classifier M and σ > 0, G is defined through the probabilities of M’s labels under Gaussian noise.
  • The smoothed classifier G predicts the label most likely under Gaussian perturbations centered at input x.
  • When G classifies an input correctly, the appendix states that G is provably robust at that input with an analytic certified radius.
  • The certified-radius expression uses Φ, defined as the cumulative distribution function of the standard Gaussian distribution.
Loading 2007.15528v3…