Source-linked AI summary
Scalable Private Learning with PATE
Nicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, Úlfar Erlingsson
TL;DR
PATE had not been realistically evaluated beyond simple classification tasks, despite the need to protect sensitive training data. This paper introduces selective, lower-noise teacher aggregation mechanisms and evaluates them on larger, imperfect datasets. The mechanisms improve privacy and utility together, scaling to high-utility learning with ε < 1.0 privacy.
Problem
PATE had previously been evaluated only on simple tasks such as MNIST, leaving its utility on larger-scale and realistic datasets unclear.
Method
The paper develops selective, lower-noise aggregation mechanisms for teacher ensembles and applies PATE to larger tasks using public data labeled by aggregated teacher answers.
Results
The mechanisms improve privacy and model accuracy over original PATE and support high utility with ε ≈1.0 privacy on a real-world character-recognition task.
Takeaways & Limitations
PATE can be used on large, imbalanced, error-containing datasets while achieving a joint gain in privacy, utility, and practicality.
Takeaways & Limitations
On Glyph, the student remained seven percentage points below the non-private model, potentially because of class imbalance and mislabeled inputs.
Abstract
from arXiv · showhide
The rapid adoption of machine learning has increased concerns about the privacy implications of machine learning models trained on sensitive data, such as medical records or other personal information. To address those concerns, one promising approach is Private Aggregation of Teacher Ensembles, or PATE, which transfers to a "student" model the knowledge of an ensemble of "teacher" models, with intuitive privacy provided by training teachers on disjoint data and strong privacy guaranteed by noisy aggregation of teachers' answers. However, PATE has so far been evaluated only on simple classification tasks like MNIST, leaving unclear its utility when applied to larger-scale learning tasks and real-world datasets. In this work, we show how PATE can scale to learning tasks with large numbers of output classes and uncurated, imbalanced training data with errors. For this, we introduce new noisy aggregation mechanisms for teacher ensembles that are more selective and add less noise, and prove their tighter differential-privacy guarantees. Our new mechanisms build on two insights: the chance of teacher consensus is increased by using more concentrated noise and, lacking consensus, no answer need be given to a student. The consensus answers used are more likely to be correct, offer better intuitive privacy, and incur lower-differential privacy cost. Our evaluation shows our mechanisms improve on the original PATE on all measures, and scale to larger tasks with both high utility and very strong privacy ($\varepsilon$ < 1.0).
1 INTRODUCTION
The paper extends PATE beyond simple benchmarks by introducing selective, lower-noise aggregation mechanisms and evaluating them on larger, imperfect datasets. Experiments report joint gains in privacy, utility, and practicality, including very strong privacy with high utility.
- PATE had previously been evaluated only on simple tasks such as MNIST, without realistic larger-scale evaluation.
- The paper introduces aggregation mechanisms that are more selective and add less noise than the original PATE mechanism.
- The evaluation includes a 150-class character-recognition task with class imbalance and erroneous labels, showing improved privacy and model accuracy.
- When teacher consensus is weak, the mechanisms can omit labels because disagreement raises privacy cost and may indicate an incorrect answer.
- Gaussian noise reduces aggregation errors relative to Laplacian noise, which is especially important for tasks with many output classes.
- ε ≈1.0 privacy is achieved alongside high utility on a real-world character-recognition task, while VAT outperforms improved GANs for Glyph semi-supervised learning.
2 RELATED WORK
Related work situates PATE within differential privacy, private selection mechanisms, and learning settings that use selective labeled examples. The paper connects its aggregation strategy to broader approaches for learning under limited supervision.
- Differential privacy provides a rigorous privacy framework designed to handle adversaries with strong capabilities.
- Earlier differentially private learning methods often used linear or convex models, while later work developed private stochastic-gradient methods for deep and federated learning.
- GNMax and LNMax are differentially private selection mechanisms used in hypothesis testing, frequent-itemset mining, and more complex private mechanisms.
- The Confident and Interactive Aggregators select samples under constraints, paralleling active learning and model-stealing settings that seek useful information from limited labeled examples.
3 BACKGROUND AND OVERVIEW
PATE protects sensitive training data by transferring knowledge from independently trained teachers to a student through private aggregation. This section introduces the framework and three improvements: Gaussian noise, selective querying, and interactive use of student predictions.
- PATE framework: PATE trains teachers on nonoverlapping sensitive-data partitions and transfers their aggregated knowledge to a student using public unlabeled data.Teacher predictions are aggregated before labeling selected public examples for semi-supervised student training.
- PATE framework: Strong teacher consensus is intuitively private because any individual training point can affect only one teacher’s prediction.The original aggregation mechanism adds calibrated noise to class vote counts and outputs the class with the highest noisy count.
- Improved aggregation: Gaussian noise reduces the noise needed for the same privacy cost per student query and supports a cleaner RDP-based analysis.RDP also composes conveniently and converts to (ε, δ)-differential-privacy bounds.
- Improved aggregation: Selective aggregation evaluates which student queries are worth answering by jointly considering privacy cost and utility.Teacher agreement is associated with both lower privacy cost and labels more likely to be correct and useful.
- Improved aggregation: The interactive mechanism additionally uses student predictions, answering when teachers agree but the student lacks confidence in that consensus.Queries where the student already agrees confidently with the teachers are not worth spending the privacy budget on.
- Privacy analysis: Data-dependent Gaussian-noise analysis is technically more complex than the Laplace case but improves the privacy-utility tradeoff without complicating the algorithm.The approach shares privacy budget between private query selection and answer computation.
4 IMPROVED AGGREGATION MECHANISMS FOR PATE
The improved PATE mechanisms replace Laplace noise with Gaussian noise and privately filter queries lacking strong consensus or useful new information. Their data-dependent analysis assigns lower privacy cost to likely consensus outcomes and selective answering.
- GNMax aggregator: The paper replaces Laplace noise with Gaussian noise and adapts the data-dependent privacy analysis to the new aggregation mechanism.The Gaussian distribution is more concentrated, improving utility especially when the number of classes is large.
- GNMax aggregator: GNMax adds Gaussian noise to each class vote count and outputs the class with the highest noisy vote count.For a class i, n_i(x) denotes its teacher vote count, and plurality is the class receiving the most votes.
- GNMax privacy guarantee: A data-dependent RDP analysis bounds each query’s privacy cost using q̃, the probability that GNMax fails to output the most common answer.The bound approaches 0 as q̃ approaches 0, while the analysis takes the minimum with the data-independent λ/σ^2 bound in practice.
- GNMax privacy guarantee: Strong teacher disagreement produces expensive queries that are also likely to receive incorrect labels, whereas strong consensus enables tighter privacy bounds.Proposition 7 estimates q̃ from teacher vote counts, after which RDP composition yields an (ε, δ)-DP bound.
- Confident-GNMax Aggregator: The Confident Aggregator privately checks whether plurality exceeds a threshold, runs GNMax only for passing queries, and returns ⊥ otherwise.The student discards examples for queries that fail the noisy threshold check.
- Confident-GNMax Aggregator: Compared with original PATE, the Confident Aggregator pays for GNMax only on threshold-passing queries and therefore expends lower privacy cost overall.The threshold is chosen to reject many low-consensus queries while retaining a high yield among strongly consensual queries.
- Interactive-GNMax Aggregator: The Interactive Aggregator discards queries when the student already confidently predicts the same label as the teachers.It can instead reinforce the student prediction, provide a teacher label when disagreement is detected, or return ⊥ when neither condition applies.
5 EXPERIMENTAL EVALUATION
The evaluation tests PATE on standard benchmarks and the larger, imbalanced, partly mislabeled Glyph task. The new GNMax-based mechanisms improve privacy–utility tradeoffs, achieving strong Glyph accuracy with privacy costs below 1 in a two-round setup.
- Experimental setup: Teachers use convolutional networks for MNIST and SVHN, random forests for UCI Adult, and ResNet teachers trained on partitioned Glyph data.The student uses dataset-specific training: GANs for MNIST and SVHN, Virtual Adversarial Training for Glyph, and random forests for Adult.
- Aggregator comparison: GNMax has a better privacy–utility tradeoff than LNMax because Gaussian noise makes incorrect plurality answers far less likely at the same normalized vote margin.At a margin of 4σ, the cited error likelihood is approximately 0.018 for LNMax versus approximately 10^-7 for GNMax.
- Aggregator comparison: Larger teacher ensembles further reduce privacy costs by tolerating larger noise scales while preserving fixed label accuracy.This comparison is reported for GNMax when teachers have sufficient data to learn good-enough models.
- Student results: On Glyph, Confident-GNMax achieves 73.5% accuracy at ε = 1.02, while answering roughly 1,300 fewer than the 12,000 submitted queries.Across the four original PATE datasets, Confident-GNMax reduces privacy cost, increases accuracy, or does both.
- Student results: A two-round Confident- and Interactive-GNMax setup reaches 73.2% accuracy with ε = 0.84 after just over 10,422 answered queries.The total privacy cost is below 0.59 × 2 = 1.18 because of improved composition.
6 CONCLUSIONS
The proposed aggregation mechanisms improve privacy and accuracy by selectively withholding low-consensus labels and using Gaussian noise, while semi-supervised learning supports student utility with limited labels.
- Noisy thresholding withholds labels when teacher consensus is too low, reducing privacy cost while preserving higher-quality supervision.Labels with small privacy cost are more likely to be correct, and private labels almost always outperformed non-private labels for a fixed number of labels.
- Gaussian noise was essential to preserve aggregated-label accuracy on Glyph data with many output classes.The Gaussian distribution has more rapidly diminishing tails than Laplace noise, increasing the chance of the correct consensus answer.
- Semi-supervised learning was instrumental for achieving strong student utility from the aggregation mechanism’s limited labels.Virtual adversarial training outperformed the approach of Salimans et al. (2016) in experiments with Glyph data.
A APPENDIX: PRIVACY ANALYSIS
The appendix develops privacy analysis for GNMax and related aggregation mechanisms, combining data-dependent bounds, Gaussian-noise reasoning, and asymptotic guarantees based on vote-count gaps.
- GNMax applies the argmax operator to a histogram whose vote counts are independently perturbed with Gaussian noise.Because each teacher can increment one count and decrement another, the Gaussian mechanism yields the stated Rényi differential-privacy guarantee.
- Theorem 6 converts RDP guarantees and an upper bound on the probability of an incorrect aggregate into a data-dependent privacy bound.The theorem assumes a likely outcome and a bound on its error probability, then analyzes neighboring datasets.
- The practical GNMax analysis computes an error-probability bound and uses the smaller of data-dependent and data-independent privacy bounds.Parameters are selected using Proposition 10, with applicability conditions checked through a critical threshold in the implementation.
- When the top three vote counts are well separated relative to σ, GNMax satisfies (λ, exp(−2λ/σ^2)/λ)-RDP for λ = (n1 − n2)/4.The condition requires n1 > n2 > n3 and both adjacent vote-count gaps to be much larger than σ.
B SMOOTH SENSITIVITY AND PUBLISHING THE PRIVACY PARAMETER
Because GNMax’s privacy parameters depend on teacher votes, the paper sanitizes the privacy loss before publishing it and uses smooth sensitivity to support this release.
- The privacy parameters depend on teacher votes through an upper bound on the aggregate’s error probability.Publishing this data-dependent parameter directly could create a privacy breach.
- The paper adds noise to a computed privacy loss and analyzes the procedure using smooth sensitivity.The section introduces an algorithm for computing GNMax’s smooth sensitivity and proves conditions sufficient for correctness.
B.1 COMPUTING SMOOTH SENSITIVITY
The smooth-sensitivity procedure operates on teacher-vote histograms, bounding how the error-probability function can change across neighboring histograms before releasing a private privacy parameter.
- A teacher-vote dataset is represented as a histogram, and q maps that histogram to an error-probability bound used in Theorem 6.The histogram distance defines neighboring vote configurations, while n(i) denotes the i-th highest bar.
- The procedure computes a β-smooth sensitivity of the privacy-loss function β(q(n̄)), which upper-bounds local sensitivity.Smooth sensitivity is defined relative to neighboring histograms and the smoothness parameter β.
- Algorithms 3–5 assume computable functions q, BL, BU, and a threshold q0 to calculate smooth sensitivity.BL and BU bound q on neighboring histograms, while [0, q0) limits the data-dependent analysis range.
- Algorithm 3 estimates local sensitivity by evaluating β at upper and lower bounds on neighboring q values.It returns the larger of the two differences between β(q) and β evaluated at BU(q) or BL(q).
B.2 NOTATION AND CONDITIONS
This section relaxes histograms to nonnegative real vectors for analysis while retaining integer-valued computations, then defines distance, neighbor relations, q, and β-based conditions used by the sensitivity algorithms.
- Notation and relaxation: Histograms are generalized to nonnegative real vectors for analysis, while actual computations remain exclusively over integer-valued inputs.
- Distance and neighbors: A move changes one bar upward and another downward by values in [0, 1], and neighboring histograms have distance d = 1.
- Sensitivity procedures: Algorithm 4 computes sensitivity at distance d by case analysis on q, histogram gaps, and transformations that increase or decrease q.
- Sensitivity procedures: Algorithm 5 computes β-smooth sensitivity by maximizing distance-specific sensitivities weighted by e^−βd.
- Conditions: The analysis assumes continuity and neighbor bounds for q, monotonicity and inverse relationships for BL and BU, a plateau-shaped β, and structural properties of q.
- Conditions: q is invariant to adding a common nonnegative constant and to permuting histogram coordinates; q is also differentiable under the stated range condition.
B.3 CORRECTNESS OF ALGORITHMS 3–5
The section establishes that Algorithms 3–5 provide valid sensitivity bounds: Algorithm 3 upper-bounds local sensitivity, Algorithm 4 finds maximal sensitivity at each distance, and Algorithm 5 computes a smooth upper bound.
- Algorithm 3: Algorithm 3 computes an upper bound on the local sensitivity of β(q(n̄)) under conditions C2–C6.
- Sensitivity analysis: The correctness argument bounds sensitivity by analyzing how q and Δβ vary across the regions determined by BL(q0) and q0.
- Algorithm 3: ˜LS(q) is non-decreasing below BL(q0), constant between BL(q0) and q0, and non-increasing above q0.
- Algorithm 4: Algorithm 4 computes the maximum ˜LS over histograms within distance d of the input under conditions C1–C8.
- Algorithm 4: The proof uses dominance between histograms and a monotone chain connecting histograms through unit-distance or permutation steps.
- Algorithm 5: Algorithm 5 computes SSβ(n̄), a β-smooth upper bound on the smooth sensitivity of β(q(·)).
B.4.1 CONDITIONS C1–C4, C7 AND C8
This section defines q as a GNMax error-probability bound and verifies the structural conditions needed for the sensitivity analysis, using calculus and symbolic-numeric checks.
- Definition of q: The q construction uses the histogram’s largest coordinate and error-function expressions involving the number of classes and noise scale.
- Definition of q: q(n̄) upper-bounds the probability that GNMax outputs anything other than the histogram’s highest coordinate.
- Conditions: Conditions C1, C7, and C8 follow from calculus, with q0 at most 0.5.
- Conditions: Conditions C3 and C4 follow from basic properties of erfc, while restricting q to [0, q0] makes BU(q) strictly less than one.
- Verification: For fixed σ and λ, the authors verify the assumptions by symbolically differentiating β and Δβ and numerically minimizing the resulting expressions over their specified ranges.
B.5 RÉNYI DIFFERENTIAL PRIVACY AND SMOOTH SENSITIVITY
The paper defines a Gaussian mechanism that scales noise by smooth sensitivity and proves its Rényi differential-privacy guarantee, potentially improving over global-sensitivity scaling.
- Mechanism: The (β, σ)-GNSS mechanism adds Gaussian noise scaled by a β-smooth sensitivity bound.
- Privacy guarantee: Theorem 23 proves that the (β, σ)-GNSS mechanism satisfies Rényi differential privacy for finite orders in a specified range.
- Privacy guarantee: When λ ≫ 1, σ ≪ λ, and β ≪ 1/(2λ), GNSS satisfies (λ, (λ+1)/σ2)-RDP.
- Comparison: The standard additive Gaussian mechanism satisfies (λ, λ/σ2)-RDP, while GNSS scales noise by smooth sensitivity that can be smaller than global sensitivity.
B.6 PUTTING IT ALL TOGETHER: APPLYING SMOOTH SENSITIVITY
This section combines data-dependent privacy accounting with smooth sensitivity to release privacy guarantees privately and efficiently. The resulting pipeline reports expected privacy costs while acknowledging tuning assumptions and additional sanitization costs.
- Privacy-preserving release: Publishing data-dependent privacy costs requires sanitization because the costs themselves may leak information about the underlying data.The pipeline applies Gaussian noise scaled by smooth sensitivity through the GNSS mechanism.
- Privacy accounting: The total privacy cost is formed from the RDP costs of the aggregator’s sequence of answered queries.A neighboring dataset may change one teacher’s output across all student queries, making the sum’s global sensitivity large.
- Smooth sensitivity: Smooth sensitivity can be extremely small even when the global sensitivity of the total privacy cost is high.This is particularly relevant for the second step of the Confident GNMax aggregator.
- Smooth sensitivity: A theorem bounds the smooth sensitivity of a sum using smooth bounds on the local sensitivities of its component functions.This enables more efficient computation than summing the components’ smooth sensitivities separately.
- Empirical results: For MNIST, 283 of 640 randomly selected examples were expected to receive answers, with expected RDP ε = 1.18 at order λ = 14.The reported expectation translates to (2.00, 10^-5)-differential privacy.
- Practical considerations: Some parameters were selected with full visibility of the data, giving a somewhat optimistic view of practical performance.The cited example tunes σSS to minimize total privacy cost plus two standard deviations of noise.