PaperlayerView exact revision on arXiv

Source-linked AI summary

MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examples

Jinyuan Jia, Ahmed Salem, Michael Backes, Yang Zhang, Neil Zhenqiang Gong

arXiv:1909.10594v3cs.CRcs.LG

TL;DR

Membership inference attacks threaten sensitive training data, while existing training-time defenses lack formal confidence-score utility guarantees and have suboptimal privacy-utility tradeoffs. MemGuard instead adds carefully crafted adversarial noise to black-box confidence scores, with bounded distortion and label preservation, and outperforms existing defenses on three datasets. Its scope includes black-box attacks, with extensions to other attack settings left as future work.

  • Problem

    Existing membership-inference defenses lack formal utility-loss guarantees for confidence score vectors and achieve suboptimal privacy-utility tradeoffs.

  • Method

    MemGuard adds carefully crafted noise to target confidence score vectors in two phases, preserving predicted labels while bounding distortion and misleading attack classifiers.

  • Results

    MemGuard effectively defends against black-box membership inference attacks and outperforms existing defenses on three real-world datasets.

  • Takeaways & Limitations

    Adversarial examples can serve as defensive mechanisms against membership inference attacks while providing formal utility-loss guarantees on confidence scores.

  • Takeaways & Limitations

    MemGuard’s demonstrated scope is black-box membership inference, with extension to white-box and other machine-learning-based inference attacks left as future work.

Abstract

from arXiv · show

In a membership inference attack, an attacker aims to infer whether a data sample is in a target classifier's training dataset or not. Specifically, given a black-box access to the target classifier, the attacker trains a binary classifier, which takes a data sample's confidence score vector predicted by the target classifier as an input and predicts the data sample to be a member or non-member of the target classifier's training dataset. Membership inference attacks pose severe privacy and security threats to the training dataset. Most existing defenses leverage differential privacy when training the target classifier or regularize the training process of the target classifier. These defenses suffer from two key limitations: 1) they do not have formal utility-loss guarantees of the confidence score vectors, and 2) they achieve suboptimal privacy-utility tradeoffs. In this work, we propose MemGuard, the first defense with formal utility-loss guarantees against black-box membership inference attacks. Instead of tampering the training process of the target classifier, MemGuard adds noise to each confidence score vector predicted by the target classifier. Our key observation is that attacker uses a classifier to predict member or non-member and classifier is vulnerable to adversarial examples. Based on the observation, we propose to add a carefully crafted noise vector to a confidence score vector to turn it into an adversarial example that misleads the attacker's classifier. Our experimental results on three datasets show that MemGuard can effectively defend against membership inference attacks and achieve better privacy-utility tradeoffs than existing defenses. Our work is the first one to show that adversarial examples can be used as defensive mechanisms to defend against membership inference attacks.

1 INTRODUCTION

MemGuard defends black-box membership inference by perturbing confidence scores rather than retraining the target classifier. It uses adversarial noise with bounded utility loss and achieves stronger privacy-utility tradeoffs than existing defenses.

  • Motivation: Membership inference attacks use confidence score vectors from a black-box target classifier to predict whether samples belong to its training dataset.Such attacks threaten sensitive training data and can damage the model provider’s intellectual property.
  • Motivation: Existing defenses regularize or differentially private-train the target classifier, but provide no formal confidence-score utility guarantees and achieve suboptimal privacy-utility tradeoffs.Overfitting makes member and non-member confidence vectors distinguishable, motivating these defenses.
  • MemGuard: MemGuard adds noise to each confidence score vector without retraining, aiming to confuse the attacker while preserving the predicted label and bounding confidence-score distortion.The defense can be applied to an existing target classifier.
  • MemGuard: Phase I crafts a noise vector that turns the confidence score vector into an adversarial example, using a defender-trained membership classifier because the attacker’s classifier is unknown.Transferability makes noise that misleads the defender’s classifier likely to mislead the attacker’s classifier.
  • MemGuard: Phase II applies the crafted noise with an analytically selected probability so expected confidence-score distortion stays within budget while membership predictions approach random guessing.The probability is obtained by formulating and solving an optimization problem.
  • Evaluation: On three real-world datasets, larger L1-norm noise reduces inference accuracy, and MemGuard achieves better privacy-utility tradeoffs than state-of-the-art defenses.At equal average confidence-score distortion, it reduces attacker inference accuracy by the most.

2 RELATED WORK

Related work covers membership inference attacks, training-time defenses, and adversarial-example mechanisms. MemGuard differs by applying crafted perturbations to confidence scores while targeting formal utility constraints.

  • Membership Inference: Membership inference has also been studied against biomedical, location, white-box, and federated-learning settings.White-box attacks use gradients over the target classifier’s parameters as sample features.
  • Membership Inference: Membership inference attacks determine whether a sample belongs to a target model’s training dataset using confidence-score features and attack classifiers trained from shadow models.Later work relaxed assumptions by ranking confidence-vector entries and using one shadow classifier.
  • Defense Mechanisms: Existing defenses include L2 regularization, adversarial regularization, dropout, model stacking, and differential privacy applied during model training.These methods primarily reduce overfitting or perturb the training objective or optimization process.
  • Defense Mechanisms: The paper identifies two limitations of existing defenses: no formal confidence-score utility-loss guarantee and suboptimal privacy-utility tradeoffs.MemGuard is designed to address both limitations.
  • Adversarial Examples: MemGuard uses carefully crafted noise to turn confidence scores into adversarial examples that are likely to induce random membership predictions.This reframes adversarial examples as a defense against classifiers used by attackers.
  • Adversarial Examples: Adaptive attackers may use attack classifiers robust to adversarial examples, making robustness against such attackers an open challenge.The experiments consider adversarial training as an empirically robust attack-classifier strategy.

3 PROBLEM FORMULATION

The paper formulates black-box membership inference as a three-party setting in which a defender perturbs confidence scores to protect training-set privacy while preserving utility. MemGuard constrains this perturbation through label preservation, probability validity, and bounded confidence-score distortion.

  • 3.1 Model Provider: The model provider deploys a neural-network target classifier that returns confidence score vectors for user queries.Each vector represents predicted posterior probabilities, and the predicted label is the highest-scoring class.
  • 3.2 Attacker: The attacker uses black-box queries and confidence score vectors to train a binary attack classifier that predicts training-set membership.The attack classifier outputs non-member or member labels for queried samples.
  • 3.3 Defender: The defender adds noise to each target classifier confidence score vector before returning it to the user.The defender may be the model provider or a trusted third party, and users receive only the noisy vector.
  • 3.3 Defender: MemGuard seeks inaccurate membership predictions while bounding the utility loss of the returned confidence score vectors.These are the framework’s two stated defense goals.
  • 3.3 Defender: The defender trains its own binary classifier because it does not know the attacker’s classifier, then crafts perturbations intended to mislead that classifier.The defense classifier maps a confidence score vector to a membership probability and predicts membership when its output exceeds 0.5.
  • 3.3 Defender: A randomized noise mechanism samples noise conditional on the true confidence score vector and minimizes the expected distance of the membership probability from 0.5.Randomization makes the defense classifier’s membership probability vary across returned outputs.
  • 3.3 Defender: Label loss is zero when noise preserves the predicted label, and MemGuard targets zero label loss for every query.The condition is argmax_j {s_j} = argmax_j {s_j + n_j}.
  • 3.3 Defender: Confidence-score distortion requires a valid probability distribution and an expected distance no greater than the provider’s budget ϵ.The paper measures distortion with the L1 norm, d(s, s + n) = ||n||1.

4 OUR MemGuard

MemGuard formulates noise selection as an optimization problem that minimizes confidence-score distortion while preserving the query label and making the defense classifier predict randomly. It approximately solves this problem through representative noise vectors, a change of variables, loss-based constraint handling, and gradient descent.

  • Optimization objective: When д(s) = 0.5, adding the zero noise vector with probability 1 is optimal and gives the objective value 0.This is the closed-form solution for Scenario I.
  • Two-phase approximation: When д(s) is not 0.5, MemGuard uses a two-phase approximation because the mechanism is a distribution over continuous noise vectors.Phase I selects, for each noise group, the vector with minimum confidence-score distortion as its representative.
  • Optimization objective: MemGuard minimizes confidence-score distortion while preserving the query label, forcing the defense classifier’s output to 0.5, and maintaining a valid probability distribution.The optimization treats the original confidence vector as a normal example and the perturbed vector as an adversarial example for the defense classifier.
  • Two-phase approximation: MemGuard removes probability-distribution constraints by representing the true confidence vector as softmax(z) and the noisy vector as softmax(z + e).The resulting optimization minimizes d(softmax(z), softmax(z + e)) over e, after which the noise vector r is recovered.
  • Constraint handling: The remaining constraints are converted into loss terms: L1 encourages a zero defense-classifier logit, L2 preserves the predicted label, and L3 measures distortion.The unconstrained objective is L = L1 + c2 · L2 + c3 · L3, with c2 and c3 balancing the terms.
  • Constraint handling: Algorithm 1 uses normalized-gradient descent while searching c3 to find a low-distortion perturbation that changes the defense-classifier logit’s sign without changing the predicted label.The method returns the previous iteration’s vector when the label changes or the logit sign fails to change.

5 EVALUATION

MemGuard is evaluated across three real-world datasets using separate target, attack, and defense classifiers, with membership inference accuracy measured on members and non-members. Across experiments, MemGuard reduces attack accuracy while preserving labels and achieves favorable privacy-utility tradeoffs.

  • Datasets: The evaluation uses Location, Texas100, and CH-MNIST, representing 30-class, 100-class, and 8-class classification problems, respectively.Location contains 5,010 samples, Texas100 67,330 samples, and CH-MNIST 5,000 images.
  • Dataset splits and attacks: Each dataset is split into folds for training the target, attack, and defense classifiers, with D1 ∪ D4 used to evaluate attack accuracy.D1 contains members of the target classifier’s training data, while D4 contains non-members.
  • Membership inference attacks: Attack accuracy is the fraction of evaluation samples correctly classified as members or non-members; without defense, all attacks except random guessing exceed 0.5.Random guessing has inference accuracy 0.5.
  • Effectiveness: As the confidence score distortion budget increases, MemGuard lowers the inference accuracies of all evaluated membership inference attacks while guaranteeing 0 label loss.The expected L1-norm distortion lies in [0, 2], and the label guarantee follows from preserving the predicted label during noise-vector search.
  • Effectiveness: With distortion budget ϵ = 1, MemGuard reduces the maximum entropy gap from 0.27 to 0.11 on Location, from 0.41 to 0.05 on Texas100, and from 0.30 to 0.06 on CH-MNIST.The entropy gap between member and non-member confidence-score distributions represents information leakage.
  • Comparison with existing defenses: MemGuard achieves the best privacy-utility tradeoff, while other defenses incur large label losses to substantially reduce attack accuracy.Model Stacking approaches inference accuracy 0.5 but has intolerable utility loss; MemGuard and MemGuard-Random achieve 0 label loss.

6 DISCUSSION AND LIMITATIONS

Machine learning vulnerabilities can be exploited defensively against membership inference attacks, but extending MemGuard and selecting its privacy-utility tradeoff remain bounded by attack setting and dataset dependence.

  • Adversarial examples can mislead machine-learning classifiers used by attackers, motivating their use against automated inference attacks.
  • MemGuard adds carefully crafted noise to confidence scores so the attacker’s classifier is likely to predict membership incorrectly while preserving utility constraints.
  • MemGuard achieves the best privacy-utility tradeoff when comparing inference accuracy with average confidence score distortion.
  • MemGuard and MemGuard-Random achieve 0 label loss, whereas other defenses incur large label losses to substantially reduce inference accuracy.
  • Future work includes extending MemGuard to website fingerprinting, side-channel, and white-box membership inference attacks while retaining formal utility-loss guarantees.
  • The privacy-utility parameter ϵ may need dataset-dependent selection, using inference-accuracy-versus-ϵ curves to meet a desired threshold.

7 CONCLUSION AND FUTURE WORK

MemGuard defends against black-box membership inference by crafting utility-constrained adversarial noise and probabilistically applying it to confidence scores. The evaluation reports effective defense and improved performance over existing defenses, while identifying broader attack settings as future work.

  • MemGuard finds utility-constrained adversarial noise in Phase I and derives an analytical probability solution for adding it in Phase II.
  • Empirical evaluation shows that MemGuard effectively defends against black-box membership inference attacks and outperforms existing defenses.
  • Future work will extend MemGuard to white-box membership inference, website fingerprinting, and side-channel attacks.

A SYNTHESIZING NON-MEMBERS

MemGuard-S trains its defense classifier with synthesized non-members instead of splitting the training dataset. On Location, MemGuard-S and MemGuard achieve similar performance.

  • Figure 7 compares inference accuracy against increasing confidence-score distortion budgets for MemGuard-S on the Location dataset.
  • MemGuard-S synthesizes non-members by randomly perturbing feature values from the training data for defense-classifier training.Each feature is retained with probability 0.9 or replaced by a randomly sampled domain value with probability 0.1.
  • MemGuard-S and MemGuard achieve similar performance, indicating that MemGuard does not necessarily require splitting the training dataset to train its defense classifier.
Loading 1909.10594v3…