PaperlayerView exact revision on arXiv

Source-linked AI summary

ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models

Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, Michael Backes

arXiv:1806.01246v2cs.CRcs.AIcs.LG

TL;DR

Membership inference attacks previously relied on strong adversarial assumptions, limiting evidence about their broader applicability. This paper relaxes those assumptions and evaluates generalized attacks and defenses, finding strong attacks across diverse datasets and defenses that reduce attack performance while preserving utility.

  • Problem

    Prior membership inference attacks assumed multiple shadow models, knowledge of the target classifier, and training data from the same distribution.

  • Method

    The paper develops model- and data-independent attacks and evaluates dropout and model stacking as defenses against membership inference.

  • Results

    Across eight diverse datasets, one shadow model achieved 0.95 precision and 0.95 recall on CIFAR-100, while defenses largely reduced attack performance and maintained high utility.

  • Takeaways & Limitations

    Membership inference remains effective under minimal or generalized assumptions, while dropout and model stacking provide practical countermeasures with high target-model prediction accuracy.

Abstract

from arXiv · show

Machine learning (ML) has become a core component of many real-world applications and training data is a key factor that drives current progress. This huge success has led Internet companies to deploy machine learning as a service (MLaaS). Recently, the first membership inference attack has shown that extraction of information on the training set is possible in such MLaaS settings, which has severe security and privacy implications. However, the early demonstrations of the feasibility of such attacks have many assumptions on the adversary, such as using multiple so-called shadow models, knowledge of the target model structure, and having a dataset from the same distribution as the target model's training data. We relax all these key assumptions, thereby showing that such attacks are very broadly applicable at low cost and thereby pose a more severe risk than previously thought. We present the most comprehensive study so far on this emerging and developing threat using eight diverse datasets which show the viability of the proposed attacks across domains. In addition, we propose the first effective defense mechanisms against such broader class of membership inference attacks that maintain a high level of utility of the ML model.

I. INTRODUCTION

The paper broadens membership inference attacks by relaxing prior assumptions about shadow models, target-model structure, and data distribution, showing they remain effective across diverse settings. It also proposes dropout and model stacking as defenses that reduce attack performance while largely preserving model utility.

  • Motivation: Membership inference determines whether a data point was used to train a model, potentially revealing sensitive attributes such as a victim’s health status.The attack targets a model’s outputs in a black-box MLaaS setting.
  • Prior limitations: Prior attacks required multiple shadow models matching the target-model structure and shadow data drawn from the target training distribution.These assumptions restrict the scope of membership inference attacks.
  • Adversary 1: 0.95 precision and 0.95 recall: with one shadow model and one attack model, the simplified attack approaches prior performance on a CNN trained on CIFAR-100.The evaluation spans eight datasets and multiple machine-learning model types.
  • Adversary 2: 0.94 precision and 0.93 recall: data transferring enables attacks across totally different domains, using 20 Newsgroups text data against a CIFAR-100 image model.The attack does not require target-model structure knowledge or same-distribution data, and avoids synthetic-data queries.
  • Defense: Dropout reduces CIFAR-100 attack performance from 0.95 precision and 0.95 recall to 0.61 and 0.60, while prediction accuracy changes only from 0.22 to 0.21.Dropout mitigates overfitting by randomly deleting a fixed proportion of edges during each training iteration.
  • Defense: More than 30%: model stacking reduces both adversary-1 precision and recall on CIFAR-100, while the target model’s prediction performance stays almost unchanged.The method organizes three machine-learning models hierarchically to prevent overfitting and applies beyond deep neural networks.

II. PRELIMINARIES

This section defines membership inference attacks in the machine-learning setting and introduces the datasets used for evaluation.

  • II. PRELIMINARIES: The section establishes the membership inference attack definition for machine-learning models.It then presents the datasets used in the evaluation.

A. Membership Inference Against Machine Learning Models · B. Datasets Description · III. TOWARDS MODEL INDEPENDENT MEMBERSHIP INFERENCE ATTACKS (ADVERSARY 1)

The paper defines black-box membership inference for classification models and evaluates it across eight datasets. It then introduces an efficient adversary that uses one shadow model and does not require knowledge of the target model’s structure.

  • A. Membership Inference Against Machine Learning Models: ML classifiers map multidimensional feature vectors to output vectors whose length equals the number of classes and whose values typically represent posterior probabilities summing to 1.The attack model is a binary classifier, with 0 denoting nonmembership and 1 denoting membership.
  • A. Membership Inference Against Machine Learning Models: A membership inference attack determines whether a target data point belongs to a model’s training dataset.The target model is accessed as a black-box, allowing the adversary to submit a point and receive its probabilistic output.
  • B. Datasets Description: The evaluation uses 8 datasets, including MNIST, CIFAR-10, CIFAR-100, Location, Purchase, Adult, News, and Face.Six datasets follow Shokri et al.’s selection and preprocessing procedure, while News and Face provide two additional evaluation datasets.
  • B. Datasets Description: Purchase lacks prediction classes, so K-means defines 2, 10, 20, 50, and 100 classes, producing five variants such as Purchase-100.Purchase-100 denotes the Purchase dataset with 100 different classes.
  • B. Datasets Description: News contains 20,000 balanced newsgroup documents across 20 classes and is represented using a TF-IDF matrix after removing headers, footers, and quotes.The dataset is used for text classification and clustering.
  • B. Datasets Description: Face contains about 13,000 web-crawled facial images from 1,680 participants; restricting evaluation to people with more than 40 images leaves 19 classes.The images are collected under uncontrolled web conditions, making facial recognition challenging.
  • III. TOWARDS MODEL INDEPENDENT MEMBERSHIP INFERENCE ATTACKS (ADVERSARY 1): Adversary 1 relaxes shadow-model assumptions by using one shadow model instead of multiple and eliminating the need to know the target model’s structure.These simplifications make membership inference more efficient and less costly.

A. Threat Model · B. One Shadow Model

The threat model assumes black-box access and uses a shadow model to generate labeled membership examples for an attack model. The proposed one-shadow-model methodology infers membership from posterior probabilities, achieving performance comparable to prior work while revealing strong dependence on overfitting and dataset characteristics.

  • A. Threat Model: A. Threat Model: The adversary has black-box access to the target classifier and trains a shadow model to mimic its behavior and derive membership labels.The initial threat model assumes binary member/non-member classification and requires labeled training data for the attack model.
  • A. Threat Model: A. Threat Model: The shadow model is assumed to use the target model’s algorithm and hyperparameters, although this assumption can later be relaxed.The adversary may obtain this information through the same MLaaS provider or approximate the target through model extraction.
  • B. One Shadow Model: B. One Shadow Model: The attack has three stages: shadow model training, attack model training, and membership inference.The adversary trains one shadow model, uses it to construct labeled posterior-probability features, and then queries the target model for inference.
  • B. One Shadow Model: B. One Shadow Model: The adversary uses only one shadow model and one attack model, reducing the cost relative to Shokri et al.’s multiple-model strategy.Shokri et al.’s configuration trains 10 shadow models and multiple attack models, one for each class.
  • B. One Shadow Model: B. One Shadow Model: On CIFAR-100, the proposed attack achieves 0.95 precision and 0.95 recall, compared with Shokri et al.’s 0.95 precision and 0.94 recall.The attack also works on both balanced datasets such as CIFAR-10 and unbalanced datasets such as Face.
  • B. One Shadow Model: B. One Shadow Model: More-overfit target models are more vulnerable to membership inference, with Adult reaching around 0.5 precision and recall versus 0.95 precision and recall on CIFAR-100.Adult has only a 2% difference between target-model training and testing accuracy, whereas CIFAR-100 has a much larger training-versus-testing gap.
  • B. One Shadow Model: B. One Shadow Model: Increasing training epochs from 10 to 100 increases attack performance on Location and Purchase-100.The experiment treats a larger epoch count as producing higher overfitting.
  • B. One Shadow Model: B. One Shadow Model: CIFAR-100 yields 10% better attack performance than CIFAR-10, while adding more than three posterior features usually has no significant effect and increasing shadow models from 1 to 10 does not improve performance.Three posteriors generally perform best, especially on MNIST.

Evaluation on MLaaS.

The attack was evaluated against Google Cloud Prediction API, where users cannot choose the classifier, model structure, or parameters. It achieved strong membership inference performance on Purchase-100 and Location, matching or exceeding local results.

  • Real-world MLaaS evaluation: The evaluation used Google Cloud Prediction API as a real-world black-box MLaaS, following the same attack methodology as the local experiments.Users upload data and receive a Google-trained model without choosing its classifier, structure, or parameters.
  • Results: 0.90 precision and 0.89 recall were achieved on Purchase-100, exceeding the local baselines of 0.89 precision and 0.86 recall.The MLaaS attack performed stronger than the previous local evaluation on this dataset.
  • Results: 0.89 precision and 0.86 recall were achieved on Location, nearly matching the local evaluation of 0.88 precision and 0.86 recall.Location performance was almost similar to the local evaluation.

C. Target Model Structure

The attack remains effective when the adversary mismatches the target model’s hyperparameters and can be extended to unknown classification algorithms through a combining attack. This broadens the scope of membership inference attacks beyond targets whose model structure is known.

  • Hyperparameters: 0.86 precision and 0.83 recall are achieved when the shadow model uses half the target model’s batch size, hidden units, and regularization parameters.These results are almost the same as those reported in Figure 1 and are observed on Purchase-100.
  • Hyperparameters: 0.82 precision and 0.80 recall are obtained when the shadow model has double the target model’s number of parameters.Performance drops slightly but remains close to the original attack, with similar results on other datasets.
  • Target Model’s Algorithm: The combining attack constructs one shadow model from multiple sub-shadow models using different classification algorithms.Because classifier types are limited, the combined model can learn different classifier behaviors and attack an unknown target model.
  • Target Model’s Algorithm: Features generated by all sub-shadow models are stacked together, representing each shadow-data point multiple times for attack-model training.The sub-shadow models use the same training data while differing in classification algorithm.
  • Target Model’s Algorithm: The combining attack has similar performance for multilayer perceptron and logistic-regression targets but performs relatively worse against random-forest targets.The Purchase-100 experiment uses multilayer perceptrons, random forests with 1,000 trees, and logistic regression as sub-shadow models and evaluates all three target-model types.

IV. TOWARDS DATA INDEPENDENT MEMBERSHIP INFERENCE ATTACKS (ADVERSARY 2) … C. Evaluation

Adversary 2 removes the requirement for same-distribution shadow data by training on an existing dataset from a different distribution. The resulting data transferring attack remains broadly applicable and, in multiple cases, achieves performance similar to the first adversary.

  • A. Threat Model: The threat model removes the assumption that the adversary possesses shadow data from the target model’s training distribution.This substantially reduces the adversary’s attack capabilities.
  • B. Methodology: The data transferring attack trains the shadow model on an existing dataset from a different distribution than the target model’s training data.This strategy is otherwise very similar to the first adversary’s attack.
  • B. Methodology: The shadow model summarizes membership status rather than mimicking target-model behavior, using only the three—or two for binary datasets—largest posteriors.This allows the effects of differing numbers of classes to be neglected.
  • C. Evaluation: The evaluation applies the data transferring attack across all datasets using the prior attack-model and shadow-model setup, with precision and recall as metrics.The setup retains the earlier data-splitting strategy and ML-model types.
  • C. Evaluation: 0.95 for the Face-to-CIFAR-100 transfer demonstrates one instance where the second adversary achieves similar performance to the first adversary.Figure 7 compares attacked target datasets on the x-axis with shadow-model training datasets on the y-axis.
  • C. Evaluation: The shadow and target models may use different architectures because they are trained on different datasets, such as a multilayer perceptron for Purchase-20 and a CNN for CIFAR-100.This relaxation of the shadow-model design assumption also applies to the second adversary.
  • C. Evaluation: The attack is more applicable than synthetic-data generation, which is limited to binary-feature datasets and requires 156 queries for one synthetic data point.The query cost is especially problematic given the dataset quantities needed by ML models and MLaaS pay-per-query pricing.
  • C. Evaluation: t-SNE projects the highest three posteriors of member and non-member points into 2D to investigate why data transferring succeeds across datasets.The analysis includes CIFAR-100 and News, plus MNIST and Purchase-10.

D. Evaluation On MLaaS … C. Evaluation

The paper evaluates membership inference across MLaaS and a model- and data-independent adversary that uses only target-model outputs, finding strong performance across datasets and flexible thresholding needs. It also shows that maximal posterior, standard deviation, and entropy provide effective attack features, while fixed thresholds do not generalize across datasets.

  • D. Evaluation On MLaaS: Using a Location shadow model against a Purchase-100 target on Google’s MLaaS achieves 0.8 precision and 0.78 recall.Reversing the datasets still achieves 0.87 precision, with the reported recall value truncated in the supplied passage.
  • V. MODEL AND DATA INDEPENDENT MEMBERSHIP INFERENCE ATTACK WITHOUT TRAINING (ADVERSARY 3): The third adversary needs neither shadow-model training nor knowledge of the target model or data distribution.The evaluation uses only the target model’s output posteriors after querying a target data point.
  • A. Threat Model: The threat model relies only on M(xTarget), avoiding the target data point’s class label requirement and thereby covering a broader setting.The supplied passage contrasts this threat model with a similar attack requiring the class label.
  • B. Methodology: The attack is an unsupervised binary classifier that labels a point as a member when its maximum posterior exceeds a threshold.Higher thresholds favor precision, whereas lower thresholds favor recall; the general threshold method samples random points in the target-data feature space.
  • C. Evaluation: Across multiple datasets, maximal posterior reaches AUC values above 0.8, while standard deviation and entropy achieve almost the same AUC.AUC evaluates discrimination over multiple thresholds, and the entropy feature is defined from posterior probabilities.
  • C. Evaluation: Selecting the top 10 percentile as the threshold yields good precision and recall for most datasets, including CIFAR-100.The experiment generates 1,000 random data points for each dataset and tests multiple thresholds.
  • C. Evaluation: No single fixed threshold, such as maximal posterior above 50%, performs well across all datasets, supporting the proposed threshold-choosing method.The evaluation compares precision and recall as threshold varies and also compares the three adversaries’ performance.

D. Comparison of the Three Attacks · VI. DEFENSE

The three membership inference attacks perform similarly across most datasets, with the minimal-assumption attack only slightly worse, especially in precision. The proposed defenses target overfitting through dropout and model stacking to improve model generalizability.

  • D. Comparison of the Three Attacks: Our data transferring attack achieves the best performance among the three attacks.Performance is measured by precision and recall on the same dataset setting.
  • D. Comparison of the Three Attacks: The first two adversaries achieve very similar performance for most datasets.
  • D. Comparison of the Three Attacks: The third adversary with minimal assumptions performs only slightly worse, especially in precision.
  • D. Comparison of the Three Attacks: These results show that membership inference attacks are very broadly applicable and pose more severe risks than previously shown.
  • VI. DEFENSE: The defenses are designed to increase ML models’ generalizability by preventing overfitting, which mainly enables the attacks.
  • VI. DEFENSE: Dropout is proposed for neural network-based classifiers, while model stacking applies to all ML models regardless of classifier.
  • VI. DEFENSE: Defense effectiveness is evaluated against the first and third adversaries; for the first, the attacker is assumed to know and replicate the defense.

A. Dropout · B. Model Stacking

Dropout reduces membership-inference attack performance by reducing overfitting while often preserving or improving target-model accuracy. Model stacking extends this defense to non-neural classifiers, significantly reducing attacks but sometimes affecting accuracy more than dropout.

  • A. Dropout: Dropout randomly deletes a fixed proportion of input- and hidden-layer edges during training, with a default dropout ratio of 0.5.This regularization method targets overfitting in fully connected neural networks.
  • A. Dropout: 0.89 to 0.64: Purchase-100 attack precision fell under dropout, while recall decreased from 0.86 to 0.63.The evaluation tested dropout against the first and third adversaries across datasets, excluding News and Adult.
  • A. Dropout: More than 25% and 40%: CIFAR-100 attack precision and recall, respectively, decreased under dropout, although MNIST recall improved.The third adversary was more resistant to dropout than the first adversary.
  • A. Dropout: 0.72 to 0.83: Purchase-50 target-model accuracy increased after dropout, and accuracy improved on more than half of the datasets.More effective dropout, measured by larger overfitting reduction, led to better defense against membership-inference attacks.
  • A. Dropout: 0.75 dropout on both layers further reduced attack performance, confirming that higher dropout ratios strengthen the defense.The ratio variation was evaluated on Purchase-100 while monitoring the first adversary and target-model accuracy.
  • B. Model Stacking: Model stacking combines multiple models hierarchically and works independently of the target model’s ML classifier, unlike dropout.Its rationale is to train different model parts on different data subsets, reducing overfitting through ensemble learning.
  • B. Model Stacking: More than 30%: CIFAR-10 attack precision and recall decreased under model stacking, while News precision and recall fell by 28% where dropout had no effect.Model stacking sometimes reduced attack performance more effectively than dropout but affected target-model accuracy more in multiple cases.
  • B. Model Stacking: More than 20% and 30%: Location attack precision and recall, respectively, decreased under model stacking, although exceptions such as MNIST remained.For non-neural target models, model stacking is presented as effective; for neural networks, dropout is sufficient because of its high utility maintenance.

VII. RELATED WORK

Prior work established membership inference across biomedical and mobility data, and introduced shadow-model attacks against ML models. Related research also covers model inversion, model extraction, adversarial examples, and privacy-preserving ML, while this paper reduces attack requirements and proposes defenses.

  • Membership Inference: Membership inference has been demonstrated across domains including biomedical data and aggregate mobility traces.Homer et al. introduced genomic-data attacks, Backes et al. generalized them to other biomedical data, and Pyrgelis et al. studied aggregate mobility traces.
  • Membership Inference Against Machine Learning: Shokri et al. presented the first membership inference attack against ML models using shadow-model training to generate attack-model data.The shadow models aim to mimic the target model’s behavior.
  • Membership Inference Against Machine Learning: This paper shows that one shadow model and one attack model can suffice, data transfer can avoid synthetic-data generation, and two defense mechanisms are effective.These contributions address the multiple-shadow-model and expensive synthetic-data assumptions associated with earlier work.
  • Attacks Against Machine Learning: Other ML attacks include model inversion, which infers missing victim attributes, and model extraction, which steals learned parameters through MLaaS outputs.Model extraction work includes equation-solving attacks based on repeated API queries and output posteriors.
  • Adversarial Examples and Privacy-Preserving Machine Learning: Adversarial-example research studies controlled noise that induces misclassification, while privacy-preserving ML develops protocols for private training, secure aggregation, and encrypted classification.The privacy-preserving approaches include two-server computation, multi-party computation, and homomorphic encryption.

VIII. CONCLUSION

The paper progressively relaxes membership-inference threat-model assumptions, showing effective attacks under broadly applicable conditions and proposing defenses that reduce attack performance while preserving high utility. Its comprehensive evaluation across eight diverse datasets demonstrates the severity of membership-privacy risks in machine-learning models.

  • Generalized Attacks: The first adversary uses one shadow model and achieves performance very similar to a multiple-shadow-model attack, reducing the cost of attacks conducted through MLaaS.A combining attack further removes the need to know the target classifier type.
  • Generalized Attacks: The second adversary uses a dataset from another distribution to build a shadow model and generate corresponding attack data, avoiding the same-distribution assumption.This data-transferring approach addresses limitations of synthetic-data generation in specific cases.
  • Generalized Attacks: The third adversary performs unsupervised membership inference without constructing any shadow model, and the attack remains effective under this minimal-assumption setting.This adversary therefore requires neither shadow-model construction nor supervised attack training.
  • Evaluation: Evaluation across 8 diverse datasets demonstrates the severe threat to membership privacy under the generalized attack conditions.The paper describes this evaluation as comprehensive.
  • Defenses: The proposed defenses, dropout and model stacking, largely reduce membership-inference attack performance while maintaining high-level utility.The defenses target overfitting, which the paper connects to sensitivity to membership-inference attacks.
Loading 1806.01246v2…