PaperlayerView exact revision on arXiv

Source-linked AI summary

Machine Learning with Membership Privacy using Adversarial Regularization

Milad Nasr, Reza Shokri, Amir Houmansadr

arXiv:1807.05852v1stat.MLcs.CRcs.LG

TL;DR

Machine learning models can leak whether records belonged to their training datasets through black-box membership inference attacks. The paper trains models with a min-max adversarial mechanism that targets the strongest attack while preserving accuracy, achieving near-random inference with small classification costs and improved generalization.

  • Problem

    Black-box model predictions can reveal whether sensitive records belonged to a training dataset, creating a fundamental privacy threat.

  • Method

    The paper formulates membership protection as a min-max game and adversarially trains classifiers to minimize prediction loss and the strongest attack’s gain.

  • Results

    The mechanism reduces membership inference toward random guessing with small accuracy costs, including 1.1% and 3% drops for CIFAR100 Alexnet and Densenet models.

  • Takeaways & Limitations

    Prediction indistinguishability provides membership privacy while also strongly regularizing models and reducing overfitting.

  • Takeaways & Limitations

    The evaluation assumes a strong adversary with substantial knowledge of the training set and focuses on supervised classification models.

Abstract

from arXiv · show

Machine learning models leak information about the datasets on which they are trained. An adversary can build an algorithm to trace the individual members of a model's training dataset. As a fundamental inference attack, he aims to distinguish between data points that were part of the model's training set and any other data points from the same distribution. This is known as the tracing (and also membership inference) attack. In this paper, we focus on such attacks against black-box models, where the adversary can only observe the output of the model, but not its parameters. This is the current setting of machine learning as a service in the Internet. We introduce a privacy mechanism to train machine learning models that provably achieve membership privacy: the model's predictions on its training data are indistinguishable from its predictions on other data points from the same distribution. We design a strategic mechanism where the privacy mechanism anticipates the membership inference attacks. The objective is to train a model such that not only does it have the minimum prediction error (high utility), but also it is the most robust model against its corresponding strongest inference attack (high privacy). We formalize this as a min-max game optimization problem, and design an adversarial training algorithm that minimizes the classification loss of the model as well as the maximum gain of the membership inference attack against it. This strategy, which guarantees membership privacy (as prediction indistinguishability), acts also as a strong regularizer and significantly generalizes the model. We evaluate our privacy mechanism on deep neural networks using different benchmark datasets. We show that our min-max strategy can mitigate the risk of membership inference attacks (close to the random guess) with a negligible cost in terms of the classification error.

1 INTRODUCTION

Machine learning services create privacy risks because black-box model predictions can reveal membership in sensitive training datasets. The paper proposes adversarially trained models that jointly preserve classification utility and provide membership privacy.

  • Privacy threat: Black-box membership inference attacks exploit statistical differences in model predictions to determine whether a record belonged to the training set.The attack is also called tracing and threatens sensitive datasets used to train machine learning models.
  • Privacy threat: Existing mitigations such as top-k predictions and L2 regularization may reduce utility loss but do not guarantee rigorous privacy.The paper contrasts these techniques with defenses designed around explicit privacy guarantees.
  • Proposed mechanism: The proposed mechanism guarantees membership privacy by making predictions on training records indistinguishable from predictions on samples drawn from the same distribution.Its privacy objective is targeted at prediction indistinguishability rather than differential privacy generally.
  • Proposed mechanism: The defender solves a min-max game that minimizes classification error while minimizing the strongest adaptive membership inference attack’s gain.The resulting model anticipates the adversary’s best response and is claimed to guarantee maximum privacy and utility at a given privacy level.
  • Results: 1.1% and 3% accuracy drops on CIFAR100 with Alexnet and Densenet accompany inference-accuracy reductions from 67.6% to 51.6% and from 63% to 51% on Purchase100 and Texas100.The corresponding classification-accuracy costs for Purchase100 and Texas100 are 3.6% and 4.4%; 50% inference accuracy is random guessing.
  • Results: 76.5% testing accuracy with 51.8% membership inference accuracy on Purchase100 contrasts with 32.1% classification accuracy for a similarly private L2-regularized model.The mechanism also reduces the training-testing accuracy gap and prevents overfitting.

2 MACHINE LEARNING

The paper formulates supervised classification as learning a probabilistic mapping from features to classes using empirical loss on sampled training data. Regularization modifies this objective to discourage overfitting, typically optimized with stochastic gradient descent.

  • Classification model: A supervised classifier maps each d-dimensional data point to a vector of k class probabilities that sum to 1.Each output component represents the relative association of the input with a class.
  • Classification model: The underlying distribution Pr(X, Y) describes features and classes, and the learning objective is to find a model that accurately maps inputs to their correct classes.The loss function measures prediction error for each data point.
  • Training objective: Samples from Pr(X, Y) form the training set D, over which machine learning algorithms minimize expected empirical loss instead of population loss.The training set is a subset of the feature-class space.
  • Training objective: The classification optimization problem combines prediction loss with a regularization function R(f) weighted by regularization factor λ.Table 1 identifies x, y, D, l(), R(), and λ as the core notation for the classifier objective.
  • Regularization: Regularization penalizes model behavior that fits the particular training dataset too closely, helping models generalize to other data points.Overfitting yields low training loss but poorer performance on data outside the training set.
  • Optimization: Stochastic gradient descent iteratively updates complex model parameters using small training-data mini-batches and typically converges to a local minimum.The method is commonly used for nonconvex losses such as those in deep neural networks.

3 MEMBERSHIP INFERENCE ATTACK

Membership inference attacks distinguish training-set members from non-members using a model’s released computations, including black-box predictions. For machine-learning models, the attack can be learned as an inference model trained on member and reference examples.

  • Membership inference attacks determine whether a target record belongs to a dataset by observing a function computed over that dataset.
  • In black-box machine-learning attacks, the adversary distinguishes members from non-members by observing the model’s predictions.These predictions are indirect nonlinear computations over the training data.
  • The inference model h maps a target record and classifier output to the probability that the record belongs to the classifier’s training set D.
  • Empirical inference gain is evaluated on disjoint attacker datasets representing training-set members and reference samples outside the training set.The classification loss uses D, while inference gain is computed on both D and D′.
  • The attacker optimizes membership inference by maximizing empirical gain, and the attack can alternatively be trained using shadow models matching the target model’s type and objective.

4 MIN-MAX MEMBERSHIP PRIVACY GAME

The paper formulates membership protection as a min-max game: the attacker maximizes inference gain while the classifier minimizes prediction loss and worst-case privacy loss. Adversarial training alternates these objectives, and sufficient model capacity theoretically yields indistinguishable member and non-member prediction distributions.

  • 4 MIN-MAX MEMBERSHIP PRIVACY GAME: The min-max game models an adaptive attacker maximizing inference gain against a classifier that minimizes both prediction loss and the attacker’s maximum gain.The defender must anticipate the strongest attack rather than optimize against a fixed attack.
  • 4 MIN-MAX MEMBERSHIP PRIVACY GAME: Adversarial training alternates k inference-model maximization steps with one classifier minimization step using training set D and disjoint reference set D′.The inference model distinguishes predictions on D from predictions on D′ before the classifier responds.
  • 4 MIN-MAX MEMBERSHIP PRIVACY GAME: The inference-attack term is added as a classifier regularizer, while λ controls the trade-off between classification accuracy and membership privacy.
  • 4 MIN-MAX MEMBERSHIP PRIVACY GAME: The optimal attack compares whether a classifier prediction is more likely under the training-member output distribution p_f or the non-member distribution p′_f.
  • 4 MIN-MAX MEMBERSHIP PRIVACY GAME: With fixed classification loss and sufficient learning capacity, the optimal classifier satisfies p_f*=p′_f*, making the optimal membership probability converge to 0.5.The stochastic-gradient procedure is stated to converge to the min-max equilibrium.

5 EXPERIMENTS

Experiments across CIFAR100, Purchase100, and Texas100 evaluate whether adversarial training reduces membership inference while preserving predictive accuracy. The mechanism approaches random-guess attack accuracy, improves generalization, and substantially narrows train–test output differences.

  • Experimental setup: Experiments cover CIFAR100, Purchase100, and Texas100 using multiple neural-network classification models.The evaluation includes AlexNet and DenseNet architectures for CIFAR100.
  • Training behavior: The min-max training process converges toward an equilibrium between classification loss and inference-attack gain.The classifier minimizes loss while the attack model maximizes membership inference gain.
  • Generalization: The mechanism reduces total generalization error by up to a factor of 4, including Texas100 from 29.7% to 7.5% and CIFAR100-AlexNet from 54.3% to 22.7%.For CIFAR100-DenseNet, the error decreases from 29.4% to 12.7%; Purchase100 remains almost unchanged.
  • Output indistinguishability: The privacy-preserving model reduces the gap between training and testing prediction distributions by one to two orders of magnitude.The gap measures information leakage exploitable by membership inference attacks.
  • Membership privacy: Attack accuracy becomes close to random guessing with only small testing-accuracy reductions: 3.5% for Purchase100, 4.4% for Texas100, 1.1% for CIFAR100-AlexNet, and 3% for CIFAR100-DenseNet.The reported tradeoff compares privacy-preserving models with regularly trained models.
  • Reference-set sensitivity: Increasing the reference-set size makes it better represent the underlying distribution and drives attack accuracy toward 50%.The experiment fixes the training-set size at 20,000 and λ = 3.

6 RELATED WORK

Prior work addresses privacy threats from model access, inference attacks, and privacy-aware optimization. These approaches range from protecting computation environments to minimizing privacy loss against adaptive attackers while preserving utility.

  • Trusted hardware and cryptographic methods protect data during training or prediction but do not limit leakage through model computations.Methods include homomorphic encryption, garbled circuits, secure multi-party computation, and trusted hardware.
  • Inference threats include input, membership, attribute, parameter, and side-channel attacks against computations over sensitive data.The paper focuses on membership inference among these attack types.
  • Membership inference and reconstruction attacks are treated as two major classes of privacy attacks.
  • Membership inference determines whether a target record belongs to the training dataset, whereas reconstruction attacks infer sensitive attributes across many individuals.Membership-attack accuracy measures how dependent a model is on individual training data.
  • Game-theoretic privacy frameworks minimize privacy loss against the strongest corresponding attack while incorporating utility into min-max optimization.This formulation aims for provable robustness under the chosen privacy-loss function while controlling privacy cost.

7 CONCLUSIONS

The paper introduces a min-max privacy mechanism that jointly optimizes prediction accuracy and membership privacy. Its experiments report negligible privacy cost and good generalization on benchmark machine-learning tasks.

  • The mechanism makes predictions on training records indistinguishable from predictions on other samples from the same underlying distribution.This provides membership privacy against the strongest inference attack under the stated setting.
  • The training objective minimizes classification loss while maximizing the gain of the membership inference attack in a min-max game.The resulting optimization jointly targets predictive accuracy and privacy.
  • Benchmark experiments show a negligible cost for achieving privacy and good generalization in privacy-preserving models.The experiments apply the method to benchmark machine-learning tasks.
Loading 1807.05852v1…