Source-linked AI summary
Privacy Analysis of Deep Learning in the Wild: Membership Inference Attacks against Transfer Learning
Yang Zou, Zhikun Zhang, Michael Backes, Yang Zhang
TL;DR
Membership inference had been studied mainly for models trained from scratch, despite the practical prevalence of transfer learning and the privacy risks of revealing training-data membership. This paper systematically evaluates three transfer-learning attacks using shadow-model training and finds effective inference across real-world datasets, including when target architectures are unknown.
Problem
Prior membership-inference research mainly targeted models trained from scratch, leaving the practical transfer-learning setting insufficiently evaluated despite its widespread use.
Method
The paper defines three transfer-learning membership attacks and trains attack classifiers from shadow-model outputs under black-box access.
Results
Attack AUC exceeds 0.9 in the reported experiments, while teacher-data membership is not leaked through the transferred student model and attacks remain effective across unknown architectures.
Takeaways & Limitations
Membership inference poses a privacy risk for transfer-learning models, including when the target architecture is unknown to the adversary.
Takeaways & Limitations
The attack taxonomy excludes teacher-model access combined with student-data inference because the student model is never used to train the teacher model.
Abstract
from arXiv · showhide
While being deployed in many critical applications as core components, machine learning (ML) models are vulnerable to various security and privacy attacks. One major privacy attack in this domain is membership inference, where an adversary aims to determine whether a target data sample is part of the training set of a target ML model. So far, most of the current membership inference attacks are evaluated against ML models trained from scratch. However, real-world ML models are typically trained following the transfer learning paradigm, where a model owner takes a pretrained model learned from a different dataset, namely teacher model, and trains her own student model by fine-tuning the teacher model with her own data. In this paper, we perform the first systematic evaluation of membership inference attacks against transfer learning models. We adopt the strategy of shadow model training to derive the data for training our membership inference classifier. Extensive experiments on four real-world image datasets show that membership inference can achieve effective performance. For instance, on the CIFAR100 classifier transferred from ResNet20 (pretrained with Caltech101), our membership inference achieves $95\%$ attack AUC. Moreover, we show that membership inference is still effective when the architecture of target model is unknown. Our results shed light on the severity of membership risks stemming from machine learning models in practice.
1 Introduction
This paper examines membership inference in transfer learning, where prior attacks mainly targeted models trained from scratch. It defines three attack settings and reports effective attacks across transfer-learning models, including under unknown architectures.
- Membership inference matters because revealing whether an individual’s data trained a model can leak sensitive information, including a patient’s disease status.
- Transfer learning addresses the data and computational costs of training from scratch, but its privacy risks had not been systematically evaluated.
- Three attack settings arise from two attack surfaces and two adversary capabilities: inferring teacher-data membership through either teacher or student models, and student-data membership through the student model.
- Attack AUC exceeds 0.9 across the reported transfer-learning experiments, while teacher-model membership information is not leaked through the transferred student model.
- Freezing more student-model layers significantly reduces membership-inference performance, yet attacks remain effective when target and shadow models use different architectures.
2 Related Work
Earlier membership-inference research covered many domains and model types but focused on models trained from scratch. This paper positions transfer-learning membership inference as a distinct practical privacy problem.
- Prior membership-inference studies span biomedical data, mobility traces, federated learning, generative models, language processing, and vision segmentation.
- Proposed defenses against membership inference include reducing overfitting, perturbing posteriors, and adversarial training.
- Existing work focused on target models trained from scratch, leaving membership inference in practical transfer-learning settings unstudied.
- Earlier transfer-learning attacks sought to mislead model behavior through adversarial examples or poisoning, whereas this paper infers private training-dataset information.
3 Background
Transfer learning reuses a pretrained teacher model to help a student model achieve strong performance without training a complex network from scratch. Membership inference tests whether a target sample belonged to a model’s training data.
- Transfer learning reduces the data and computational resources needed by individual researchers and small companies to train complex models.
- A teacher model is pretrained, while a downstream student model transfers its learned knowledge for a new dataset.
- Feature extraction freezes the first K layers and updates the remaining layers, whereas fine-tuning updates the whole model using the teacher parameters as initialization.
- Membership inference determines whether a target sample was used to train a model; its attack model is a binary classifier whose output indicates member or non-member status.
4 Attack Methodologies
The attack methodology distinguishes teacher- and student-data membership across teacher- and student-model access, using black-box queries and shadow models to train attack classifiers.
- Attack Taxonomy and Threat Model: The transfer-learning threat model contains three attacks: teacher-data inference through the teacher model, teacher-data inference through the student model, and student-data inference through the student model.
- Attack Taxonomy and Threat Model: The combination of teacher-model access with student-data inference is infeasible because the student model is never used to train the teacher model.
- Attack Taxonomy and Threat Model: Adversaries are assumed to have black-box access, allowing queries that return model posteriors rather than direct access to architecture or parameters.
- Attack Pipeline: For each target sample and membership objective, the attacker queries the relevant model, feeds its posterior to an attack model, and predicts membership.
- Attack Models Training: Shadow-model training supplies data for the binary attack classifiers that infer membership from target-model posteriors.
5 Evaluation
The evaluation tests membership inference across transfer-learning attacks, frozen-layer settings, and unknown target architectures using four image datasets and AUC among its metrics. Results show strong leakage for student-dataset membership, no effective teacher-dataset leakage through student models, and reduced attack performance with more frozen layers.
- 5.2 Q1: Effectiveness of Different Membership Inference Attacks: Attack-2 performs close to random guessing for teacher-dataset membership through student models, indicating that fine-tuning removes the teacher dataset’s membership information.By contrast, direct access to the teacher model yields 0.906 AUC in Attack-1 on Caltech101.
- 5.2 Q1: Effectiveness of Different Membership Inference Attacks: Membership inference remains effective against student models for determining membership in their student datasets.Attack-3 evaluates student-dataset membership after fine-tuning on CIFAR100, Flowers102, and PubFig83.
- 5.3 Q2: Impact of Frozen Layers: Attack performance consistently decreases as more layers are frozen, because freezing higher layers makes the student model less able to remember student training data.The drop becomes significant when frozen layers increase from 4 to 5, where no parameters are updated and the student model matches the teacher model.
- 5.4 Q3: Effectiveness of Transfer Attack: 0.9 AUC or higher is achieved across all datasets when the target architecture is unknown, with at most a 3% drop from Attack-3.The transfer attack uses VGG16 shadow models and ResNet20 target models, both pretrained on Caltech101 and fine-tuned on three student datasets.
6 Conclusion
The paper systematically evaluates membership inference against transfer learning models, finding effective attacks across teacher and student models while identifying important privacy boundaries and conditions.
- The study identifies two attack surfaces—training-data membership in teacher and student models—and develops three attacks for the transfer learning setting.
- Membership inference attacks against both teacher and student models are effective, with attack AUC larger than 0.9 across the evaluated real-world datasets.
- Transferring a teacher model to a student model does not leak the teacher model’s membership information.
- Freezing more layers during student training reduces membership-inference attack performance.
- Membership inference remains effective when the target model’s architecture is unknown to the adversary.