PaperlayerView exact revision on arXiv

Source-linked AI summary

Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data

Nicolas Papernot, Martín Abadi, Úlfar Erlingsson, Ian Goodfellow, Kunal Talwar

arXiv:1610.05755v4stat.MLcs.CRcs.LG

TL;DR

Sensitive-data models can memorize and expose training examples, so the paper introduces PATE to provide privacy through black-box knowledge transfer from disjoint-data teachers to a student. Noisy teacher voting, improved privacy analysis, and semi-supervised learning yield strong privacy guarantees and privacy/utility trade-offs that equal or improve upon prior methods on MNIST and SVHN.

  • Problem

    Machine-learning models may implicitly memorize sensitive training examples, which can be recovered through model inspection or repeated queries.

  • Method

    PATE trains teachers on disjoint sensitive-data subsets, privately aggregates their noisy top votes, and transfers the resulting labels to a student using public unlabeled data.

  • Results

    On MNIST and SVHN, the techniques provide a privacy/utility tradeoff that equals or improves upon bespoke learning methods.

  • Takeaways & Limitations

    The black-box learning strategy and privacy analysis apply to deep, non-convex, and other learning methods without depending on their training details.

  • Takeaways & Limitations

    The analysis is conservative because a single changed training example may arbitrarily change its teacher, although stronger guarantees may be possible under algorithm-specific assumptions.

Abstract

from arXiv · show

Some machine learning applications involve training data that is sensitive, such as the medical histories of patients in a clinical trial. A model may inadvertently and implicitly store some of its training data; careful analysis of the model may therefore reveal sensitive information. To address this problem, we demonstrate a generally applicable approach to providing strong privacy guarantees for training data: Private Aggregation of Teacher Ensembles (PATE). The approach combines, in a black-box fashion, multiple models trained with disjoint datasets, such as records from different subsets of users. Because they rely directly on sensitive data, these models are not published, but instead used as "teachers" for a "student" model. The student learns to predict an output chosen by noisy voting among all of the teachers, and cannot directly access an individual teacher or the underlying data or parameters. The student's privacy properties can be understood both intuitively (since no single teacher and thus no single dataset dictates the student's training) and formally, in terms of differential privacy. These properties hold even if an adversary can not only query the student but also inspect its internal workings. Compared with previous work, the approach imposes only weak assumptions on how teachers are trained: it applies to any model, including non-convex models like DNNs. We achieve state-of-the-art privacy/utility trade-offs on MNIST and SVHN thanks to an improved privacy analysis and semi-supervised learning.

1 INTRODUCTION

Sensitive-data models can memorize and expose private training examples, motivating PATE, which transfers knowledge from disjoint-data teachers to a student through noisy aggregation and semi-supervised learning. The approach provides black-box differential privacy and achieves privacy/utility trade-offs that equal or improve upon prior methods on MNIST and SVHN.

  • Motivation: Sensitive training data can be implicitly memorized by models and recovered through internal analysis or repeated queries.Attacks have recovered private examples from model behavior, including individual faces from a computer-vision classifier.
  • Approach: PATE trains teachers on disjoint sensitive-data subsets, then trains a student on public unlabeled data labeled by the teacher ensemble.The student learns from aggregate teacher outputs rather than directly accessing an individual teacher or its training data.
  • Approach: PATE strengthens privacy by limiting teacher votes and revealing only a carefully noise-perturbed top vote to the student.Its improved privacy analysis supports meaningful guarantees when combined with semi-supervised learning.
  • Contributions: PATE provides differential privacy independently of the learning algorithm, including for deep and non-convex models without requiring restricted convex losses.The approach does not depend on details such as batch selection, loss function, or optimization algorithm.
  • Results: On MNIST and SVHN, the techniques provide a privacy/utility tradeoff that equals or improves upon bespoke learning methods.The paper combines semi-supervised knowledge transfer with a precise, data-dependent privacy analysis.

2 PRIVATE LEARNING WITH ENSEMBLES OF TEACHERS

PATE partitions sensitive data among teacher models, aggregates their predictions with noise, and trains a deployable student on nonsensitive unlabeled data. GAN-based semi-supervised learning reduces the labels needed from the ensemble, while the student separates deployment queries from teacher consultations.

  • Data partitioning and teachers: PATE partitions the dataset into n disjoint subsets and trains one teacher classifier on each subset.The teachers form an ensemble that predicts on unseen inputs and aggregates their outputs.
  • Aggregation: The ensemble counts teacher votes for each class and adds random noise before selecting the aggregated label.Noise prevents a single teacher vote from determining the output when class counts are close.
  • Student transfer: A student trained on nonsensitive unlabeled data is deployed instead of the teacher ensemble, fixing privacy loss independently of end-user query volume.Privacy loss depends on teacher queries during student training, while the student’s public parameters may be inspected or reverse-engineered.
  • GAN-based student training: PATE-G combines GANs with semi-supervised learning to reduce the number of labeled examples the student needs from teachers.The generator and discriminator are trained competitively, and the method uses unlabeled inputs to estimate a distribution prior.
  • GAN-based student training: GAN-based semi-supervised learning has been empirically shown to improve classifier training on several datasets, especially with feature matching loss.The passage notes that formal results explaining why this improvement occurs are not currently available.

3 PRIVACY ANALYSIS OF THE APPROACH

The privacy analysis tracks PATE’s differential privacy cost across student-training queries, using moments accounting and data-dependent bounds that exploit strong teacher quorums. It also identifies trade-offs in teacher count and reports the conservative assumptions and evaluation caveats underlying the guarantees.

  • Privacy accounting: PATE tracks privacy loss across successive student-training queries with a moments accountant, then converts the accumulated moments into an (ε, δ)-differential privacy guarantee.The analysis uses composability and tail bounds to combine adaptive mechanisms.
  • Aggregation bounds: The aggregation mechanism’s privacy analysis bounds the probability of reporting a label different from the highest-scoring label, using the label-score vector and neighboring databases.These bounds are combined with the data-dependent moment bounds to estimate total privacy cost.
  • Reported privacy costs: With γ = 0.05, the reported composition bound is ε ≈26 for SVHN with T = 1000 and ε ≈5.80 for MNIST with T = 100.Both examples use the aggregation mechanism with Laplace noise and δ values specified in the passage.
  • Data-dependent analysis: Strong teacher quorums reduce privacy cost because the majority outcome is overwhelmingly likely, making the data-dependent analysis tighter.The moments accountant provides a unified way to compose these quorum-dependent bounds.
  • Teacher-count trade-off: More teachers can lower privacy cost by increasing vote gaps, but excessive partitioning leaves each teacher with too little training data to remain accurate.The benefit therefore holds only up to a point as the number of teachers grows.
  • Assumptions and caveats: The analysis is conservative because it allows one changed training example to alter its teacher’s classifier arbitrarily, while evaluation reports un-noised ε despite requiring noise for private release.The paper reports that the required noise scale is smaller than 0.5 on MNIST and 1.0 on SVHN.

4 EVALUATION

The evaluation examines how teacher-ensemble size, vote confidence, noisy aggregation, and semi-supervised student training affect privacy and accuracy on MNIST and SVHN. PATE-G students achieve strong accuracy with formal differential-privacy bounds while using limited teacher queries.

  • Training an Ensemble of Teachers Producing Private Labels: Larger vote gaps indicate greater ensemble confidence and robustness to additional noise injection.The gap is the normalized difference between the most and second most frequent labels, averaged over test data.
  • Training an Ensemble of Teachers Producing Private Labels: 83.86% MNIST and 83.18% SVHN are the average test accuracies of individual teachers even with n = 250.The larger SVHN dataset compensates for its greater task complexity.
  • Training an Ensemble of Teachers Producing Private Labels: Large teacher ensembles are essential because they permit stronger noise injection with limited impact on aggregation accuracy.The evaluation considers ensembles with 10, 100, and 250 teachers, and reports that large ensembles compensate for noisy votes.
  • Semi-Supervised Training of the Student with Privacy: 98.00% MNIST and 90.66% SVHN student accuracies are obtained with (ε, δ) guarantees of (2.04, 10^-5) and (8.19, 8×10^-6), respectively.These results use semi-supervised learning and improve the reported differential-privacy state of the art for the datasets.
  • Semi-Supervised Training of the Student with Privacy: 98% MNIST accuracy is achieved with only 100 teacher label queries, within 1% of a model trained on the entire dataset.The SVHN student reaches 90.66%, compared with 92.80% for a teacher trained on the entire dataset.
  • Semi-Supervised Training of the Student with Privacy: The private student outperforms noisy or non-noisy aggregation outputs in accuracy, although performance may vary on data whose categories are less salient in the input space.The paper identifies medical data as an example where the student may not learn as well.

5 DISCUSSION AND RELATED WORK

The discussion places PATE within differential-privacy and knowledge-transfer research. It emphasizes applicability beyond restricted model classes and contrasts PATE with approaches tailored to particular algorithms or parameter-level guarantees.

  • Privacy Definitions: Differential privacy is presented as a randomized-algorithm property, unlike k-anonymity, which lacks randomization and can permit dataset-property inference.The discussion identifies differential privacy as a rigorous standard for privacy guarantees.
  • Related Work: Prior private machine-learning methods include shallow-model approaches, randomized response, distributed SGD, and noisy SGD with a moments accountant.The related work spans multiple mechanisms and model settings.
  • Knowledge Transfer: PATE differs from Hamm et al. by using only teachers’ top vote and avoiding restrictions to convex student losses.The paper describes PATE as applicable to non-convex models.
  • Related Work: Unlike decision-tree-specific privacy transfer, PATE applies to decision trees and other more complex machine-learning algorithms.The comparison specifically contrasts PATE with Jagannathan et al.’s tailored random-forest approach.

6 CONCLUSIONS

The paper concludes that PATE combines knowledge transfer from disjoint-data teachers with privacy analysis to provide strong privacy and utility on MNIST and SVHN. It also identifies requirements for many teachers and uncertainty beyond these benchmark settings.

  • Conclusions: PATE transfers knowledge from teachers trained on disjoint data to a student whose attributes may be made public.The approach is paired with a corresponding privacy analysis.
  • Conclusions: MNIST and SVHN experiments demonstrate excellent utility alongside a formal, state-of-the-art bound on users’ privacy loss.The conclusion characterizes the result as encouraging while acknowledging limits.
  • Conclusions: The approach requires disjoint training data for many teachers, and the required number may increase for tasks with many output classes.The conclusion identifies this as a limitation of the reported results.
  • Conclusions: The learning and analysis methods are black-box and therefore apply generally to non-convex, deep-learning, and other learning methods.The paper also notes applicability when data are naturally or non-randomly partitioned, or teachers use different methods.

A MISSING DETAILS ON THE ANALYSIS

The appendix supplies missing analysis details, including a theorem for privacy transfer under differential privacy and lemmas analyzing noisy label aggregation. The proofs use differential-privacy bounds, monotonicity, integration, convolution, and a union bound.

  • Missing Proofs: The appendix states that it provides missing proofs from Section 3.These proofs support the paper’s privacy analysis.
  • Theorem 3: Theorem 3 bounds privacy for a (2γ, 0)-differentially private mechanism using an outcome error probability q and neighboring databases.The theorem imposes conditions on γ and q before giving the resulting guarantee.
  • Theorem 3: The proof transfers the error probability across neighboring databases by multiplying it by exp(2γ).This follows directly from the differential-privacy inequality used in the proof.
  • Theorem 3: The proof concludes by showing the relevant function is increasing in both arguments, yielding f(q′) ≤ f(q).The monotonicity argument completes the theorem’s bound.
  • Noisy Aggregation Lemma: Lemma 4 analyzes the probability that noisy voting changes the leading label when the top label’s count exceeds every competing count.The analysis models the difference using sums of independent Laplace variables and Gamma distributions.
  • Noisy Aggregation Lemma: The tail probability is obtained by integration and extended to all candidate labels with a union bound.The resulting expression depends on γ and the vote-count gap.

B APPENDIX: TRAINING THE STUDENT WITH MINIMAL TEACHER QUERIES

This appendix examines methods for reducing student queries to the teacher ensemble, because querying directly affects total privacy cost.

  • Distillation is the first approach considered for reducing queries to the teacher ensemble.
  • Three additional query-reduction techniques were also considered after distillation.
  • The appendix frames query reduction as a privacy-cost objective during student training.

B.1 TRAINING STUDENTS USING DISTILLATION

Distillation transfers ensemble knowledge to a smaller student through probability-vector labels, but its accuracy gains were too limited to justify the additional privacy cost.

  • Distillation compresses teacher-ensemble knowledge into a student using probability vectors produced by the larger model.The student has n times fewer trainable parameters than the n teachers.
  • In the SVHN experiment, 50 teachers labeled student data either by plurality or by distillation at T ∈{1, 5}.The first 10,000 test samples supplied unlabeled training data, and accuracy was evaluated on the remaining 16,032 samples.
  • Distillation produced more accurate classifiers, but the increase was too limited to justify revealing the ensemble’s full probability vectors.The comparison therefore motivated investigation of active learning instead.

B.2 ACTIVE LEARNING OF THE STUDENT

Active learning reduces teacher queries by prioritizing uncertain student-training points, following unsuccessful Siamese and binary-expert approaches.

  • B.2 ACTIVE LEARNING OF THE STUDENT: Active learning prioritizes student-training points with high potential to contribute to learning, avoiding queries for confidently predicted labels.
  • B.2 ACTIVE LEARNING OF THE STUDENT: The Siamese-network attempt proved too complicated to reduce the number of teacher queries effectively.
  • B.2 ACTIVE LEARNING OF THE STUDENT: Figure 5 compares student accuracy against initial training-sample counts for plurality labels and aggregated probability vectors from 50 teachers.
  • B.2 ACTIVE LEARNING OF THE STUDENT: The binary-expert approach selected points using class-score thresholds after an initial batch of teacher queries.
  • B.2 ACTIVE LEARNING OF THE STUDENT: The final approach used the student’s maximum predicted probability to sort unlabeled points and query teachers for the least confident.

C APPENDIX: ADDITIONAL EXPERIMENTS ON THE UCI ADULT AND DIABETES DATASETS

Additional experiments applied the approach with random forests to the UCI Adult and Diabetes datasets, achieving meaningful privacy guarantees with high accuracy.

  • The additional experiments used random forests for both teachers and students on the Adult and Diabetes datasets.This tested applicability beyond the convolutional-network and GAN architectures used on MNIST and SVHN.
  • 250 random-forest teachers privately labeled 500 test inputs using vote counts perturbed with Laplacian noise of scale 0.05.
  • 83% Adult accuracy was achieved at (ε, δ) = (2.66, 10^-5), versus 85% for the non-private model.
  • 93.94% Diabetes accuracy was achieved at (ε, δ) = (1.44, 10^-5), versus 93.81% for the non-private model.
Loading 1610.05755v4…