ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models

Yugeng Liu; Rui Wen; Xinlei He; Ahmed Salem; Zhikun Zhang; Michael Backes; Emiliano De Cristofaro; Mario Fritz; Yang Zhang

Source-linked AI summary

ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models

Yugeng Liu, Rui Wen, Xinlei He, Ahmed Salem, Zhikun Zhang, Michael Backes, Emiliano De Cristofaro, Mario Fritz, Yang Zhang

arXiv:2102.02551v2Feb 4, 2021cs.CRcs.AIcs.LGstat.ML

TL;DR

Inference attacks expose sensitive training-data or model-parameter information, but prior work studied them mostly in isolation. This paper builds a holistic taxonomy and evaluation of four attacks across models and datasets, finding that dataset complexity shapes attack performance, attack effectiveness can be negatively related, and tested defenses mitigate only some attacks.

Problem
Inference attacks can reveal sensitive training data or model parameters, while existing studies lack a comprehensive view across scenarios, performance factors, attack relationships, and defenses.
Method
The paper establishes a threat-model taxonomy and evaluates membership inference, model inversion, attribute inference, and model stealing across five architectures and four image datasets.
Results
Dataset complexity plays an important role in attack performance, membership inference and model stealing are negatively correlated, and DP-SGD and Knowledge Distillation mitigate only some attacks.
Takeaways & Limitations
ML-DOCTOR provides reusable software for model owners to assess inference risks and for researchers to benchmark attacks and defenses.
Takeaways & Limitations
ML-DOCTOR currently concentrates on image-classification models and is planned for extension to other application scenarios and training paradigms.

Abstract

from arXiv · show

Inference attacks against Machine Learning (ML) models allow adversaries to learn sensitive information about training data, model parameters, etc. While researchers have studied, in depth, several kinds of attacks, they have done so in isolation. As a result, we lack a comprehensive picture of the risks caused by the attacks, e.g., the different scenarios they can be applied to, the common factors that influence their performance, the relationship among them, or the effectiveness of possible defenses. In this paper, we fill this gap by presenting a first-of-its-kind holistic risk assessment of different inference attacks against machine learning models. We concentrate on four attacks -- namely, membership inference, model inversion, attribute inference, and model stealing -- and establish a threat model taxonomy. Our extensive experimental evaluation, run on five model architectures and four image datasets, shows that the complexity of the training dataset plays an important role with respect to the attack's performance, while the effectiveness of model stealing and membership inference attacks are negatively correlated. We also show that defenses like DP-SGD and Knowledge Distillation can only mitigate some of the inference attacks. Our analysis relies on a modular re-usable software, ML-Doctor, which enables ML model owners to assess the risks of deploying their models, and equally serves as a benchmark tool for researchers and practitioners.

1 Introduction

The paper addresses the fragmented study of inference attacks by assessing four attacks, their threat models, performance factors, relationships, and defenses holistically. Across five architectures and four image datasets, dataset complexity strongly affects attack performance, attacks can trade off, and evaluated defenses mitigate only some risks.

Threat Model Taxonomy: The paper categorizes four attacks—membership inference, model inversion, attribute inference, and model stealing—across model access and auxiliary-dataset conditions.
Existing inference attacks were studied largely in isolation, motivating a holistic assessment of their scenarios, performance factors, relationships, and defenses.
Experimental Evaluation: The evaluation jointly examines five model architectures and four image datasets to study dataset complexity, overfitting, and relationships among attacks.
Main Findings: 68.4% increase in membership-inference performance occurs from FMNIST to STL10 for ResNet18 with black-box access and a shadow dataset.
Main Findings: Membership inference and model stealing are negatively correlated, while DP-SGD and Knowledge Distillation mitigate only some inference attacks.
ML-DOCTOR: ML-DOCTOR is a modular, reusable framework for model owners’ risk assessment and researchers’ benchmarking of attacks and defenses.

2 Threat Modeling

The threat model taxonomy organizes attacks by target-model access and auxiliary-data availability. These dimensions produce six theoretical combinations, five of which are evaluated, excluding black-box access without auxiliary data.

Threat models vary along two dimensions: white-box or black-box target-model access and partial, shadow, or absent auxiliary data.
Access to the Target Model: White-box access exposes model parameters and architecture, whereas black-box access provides API-like querying and model outputs.
Auxiliary Dataset: Auxiliary-data availability ranges from parts of the actual training set to a same-distribution shadow dataset or no dataset.
Considered Settings: The two access settings and three auxiliary-data settings yield six threat models, but experiments cover five because black-box access without auxiliary data is considered unlikely to succeed.

3 Inference Attacks

The paper measures four inference attacks that target training data, unintended attributes, or model parameters under distinct access and auxiliary-data assumptions. Their implementations use representative state-of-the-art methods tailored to each threat model.

The four attacks are membership inference, model inversion, attribute inference, and model stealing; the first three target training-data information, while the last targets model parameters.
Membership Inference: Membership inference determines whether a target sample belonged to training data, using model outputs and auxiliary data to train a binary attack classifier.
Membership Inference: In the black-box shadow-dataset setting, the adversary trains a shadow model, labels shadow samples as members or non-members, and applies the resulting classifier to the target model.
Model Inversion: Model inversion reconstructs representative or synthesized training samples and requires white-box access in the considered attacks because optimization uses target-model parameters.
Attribute Inference: Attribute inference predicts target attributes from model embeddings, using labeled auxiliary data to train an attribute classifier.
Model Stealing: Model stealing extracts a target model’s parameters by querying it with auxiliary samples and training a stolen model on its output posteriors.

4 ML-DOCTOR

ML-DOCTOR is a modular framework for jointly evaluating four inference-attack types and two defenses. It combines threat-model taxonomy, attack and defense modules, and reusable evaluation components for model owners and researchers.

ML-DOCTOR jointly evaluates four inference-attack types and two representative defenses.
The paper presents ML-DOCTOR as the first framework known to jointly consider different inference attacks.
Modules: The framework contains data-processing, attack, defense, and evaluation modules.The attack module currently supports ten attacks spanning four attack types.
Modules: Its modular design supports integrating additional attacks, defenses, datasets, and models.
Using ML-DOCTOR: ML-DOCTOR supports deployment-risk assessment for model owners and reusable benchmarking for researchers.Users provide a target model and its training dataset for a full-fledged privacy risk assessment.

5 Experimental Settings

The evaluation trains 20 target models across four benchmark image datasets and five architectures, partitions each dataset for the threat models, and runs attacks through ML-DOCTOR.

Evaluation Procedure: ML-DOCTOR receives each target model and corresponding dataset partitions, runs the attacks and defenses, and summarizes results for the research questions.
Datasets and Models: The datasets cover faces, clothing, and other image categories with differing classes, modalities, and complexity.CelebA is converted to an 8-class task from three binary attributes, while FMNIST contains 10 grayscale-image classes.
Dataset Partitioning: Each dataset is split into four disjoint parts: target training, target testing, shadow training, and shadow testing datasets.These partitions support target-model evaluation and attack models requiring partial or shadow auxiliary data.
Datasets and Models: 20 target models are trained from four benchmark image datasets and five model architectures.The architectures are AlexNet, ResNet18, VGG19, Xception, and SimpleCNN.
Training: Target models are trained for 300 epochs using SGD, cross-entropy loss, batch size 64, weight decay 5e-4, and momentum 0.9.
Attack Models: Membership inference uses black-box or white-box attack models with target-model outputs or embeddings as inputs.The black-box attack combines ranked posteriors with an indicator of correct prediction; accuracy, F1, and AUC are reported.

6 Experimental Evaluation

Across the evaluation, dataset complexity and overfitting shape attack performance, with membership inference improving while model stealing worsens; the two attacks are strongly negatively correlated.

Threat Models: White-box access generally produces better membership-inference performance than black-box access.The paper links this to the greater information available to the adversary, especially classification loss signals.
Model Stealing: Model stealing achieves agreement of 0.927 on ResNet18 trained on FMNIST, while partial training data perform worse than shadow data.The paper suggests that partial-data queries yield lower-entropy posteriors containing less exploitable information.
Dataset Complexity: More complex datasets improve membership inference but worsen model stealing; model inversion is less effective on STL10, while attribute inference shows no strong complexity effect.The paper attributes the membership-inference and model-stealing trends to overfitting and the difficulty of obtaining similarly complex query data.
Overfitting: Higher overfitting improves membership inference but makes model stealing more difficult.Overfitted models provide more confident member predictions, while memorized training data are harder for adversaries to reproduce.
Training Duration: After 100 epochs, attack performance becomes steady; membership inference rises through 100 epochs, while inversion and attribute inference fluctuate slightly.
Attack Relationships: The correlation between membership inference and model stealing is r = −0.821 for accuracy.Worse membership inference corresponds to better model stealing; membership inference and model inversion also show a strong negative correlation, with exceptions noted for FMNIST.

7 Defenses

The paper evaluates Differential Privacy and Knowledge Distillation as defenses against four inference attacks, finding that each mitigates only some attacks and that protection involves utility trade-offs.

Differential Privacy: Differential Privacy limits the influence of individual training samples, while the Gaussian mechanism adds noise scaled to a function’s global sensitivity.DP-SGD applies this principle by clipping gradients and adding Gaussian noise during training.
Knowledge Distillation: Knowledge Distillation transfers knowledge through soft labels from an original model to a lower-capacity distilled model.The training objective combines soft-target and hard-target losses.
Defense results: DP-SGD effectively defends membership inference, reducing CelebA accuracy from 0.721 to 0.500, but has limited or inconsistent effects on other attacks.It slightly reduces model inversion and attribute inference, reduces model stealing on some datasets, and can enhance model stealing on CelebA.
Defense results: Knowledge Distillation usually changes model inversion, attribute inference, and model stealing performance by less than 5%, while sometimes reducing membership inference risk less than DP-SGD.The comparison is between original and distilled models.
Utility trade-off: Both defenses reduce some inference risks but can lower target-model utility, with DP-SGD providing stronger defense and KD preserving utility better.On STL10, target testing accuracy falls from 0.818 to 0.698 or 0.680 with DP-SGD, versus 0.834 to 0.823 with KD.

8 Related Work

Prior work studies inference attacks and defenses across specialized settings, whereas ML-DOCTOR evaluates four attack types together under a comprehensive framework.

Inference attacks: Membership inference research progressed from shadow-model attacks toward settings with fewer assumptions and applications beyond standard image models.The cited work includes natural language processing, generative models, recommender systems, and federated learning.
Inference attacks: Attribute inference research covers both macro-level properties and sample-level attacks, with prior work linking its risks to machine-learning overlearning.These studies include centralized and federated-machine-learning settings.
Inference attacks: Model inversion research evolved from drug-dose classification to general back-propagation attacks and GAN-based synthesis of training data.Later work also examines model inversion against natural-language-processing models.
Defenses: Prior defenses include Differential Privacy methods such as DP-SGD and PATE, which limit individual-sample influence or use teacher models to label public data.DP-SGD adds Gaussian noise to target-model gradients during training.
Defenses: Knowledge Distillation has been studied as a defense against adversarial examples and membership privacy, including by reducing posterior differences between members and nonmembers.The paper reports that distillation can protect certain target models.
Evaluation tools: Unlike ML Privacy Meter, which focuses on membership inference, ML-DOCTOR considers four inference attacks simultaneously for comprehensive analysis.This positions ML-DOCTOR as a broader evaluation framework than membership-only tooling.

9 Conclusion

The paper presents a holistic assessment of four inference attacks, evaluates attacks and defenses across multiple architectures and datasets, and packages the analysis in reusable software. It finds dataset complexity and attack interactions shape risk, while current defenses mitigate only some attacks.

The study establishes a threat-model taxonomy for membership inference, model inversion, attribute inference, and model stealing.
The extensive measurement study spans five model architectures and four datasets, covering both attacks and defenses.
Training-dataset complexity importantly affects attack performance, and model stealing and membership inference effectiveness are negatively correlated.
DP-SGD and knowledge distillation can mitigate only some inference attacks.
ML-DOCTOR integrates the attacks and defenses into reusable modular software for pre-deployment risk assessment and benchmarking.
ML-DOCTOR currently concentrates on image classification models, with planned extensions to other model types, application scenarios, and defenses.

A Additional Experimental Results

The appendix provides additional experimental plots covering model inversion, membership inference, attribute inference, model stealing, and relationships among attacks across the evaluated settings.

The appendix reports plots for additional experiments mentioned throughout the paper.
Model inversion results are visualized for AlexNet trained on UTKFace, comparing two reconstruction methods with training-data samples.
Membership inference is evaluated with F1, AUC, and ROC plots across threat models, datasets, and target architectures.
Attribute inference is reported with F1-based plots across threat models, datasets, and target architectures, including ROC curves for UTKFace.
Model stealing is evaluated with accuracy plots across threat models, datasets, and target model architectures.
The relationship among attacks is plotted under the same threat model using AUC for membership inference, macro-F1 for model inversion, and accuracy for model stealing.