Evaluating Differentially Private Machine Learning in Practice

• Implementations often choose arbitrarily large ϵ values to obtain acceptable utility, which severely undermines the value of their formal privacy guarantees.One cited deep-learning implementation may require ϵ proportional to model size, potentially reaching a few million.Source: §1 · p. 1
• Tighter composition analyses can lower the apparent privacy budget for the same noise, but their practical privacy effects under adversarial attacks remain poorly understood.The paper evaluates advanced composition, zCDP, and RDP to examine this trade-off empirically.Source: §1 · p. 1
• The study evaluates differential privacy mechanisms across privacy budgets and variants, measuring both model utility and privacy leakage in machine learning.It focuses on gradient perturbation mechanisms applicable to empirical risk minimization and deep learning, including logistic regression and neural networks.Source: §1 · pp. 1–2
• For acceptable utility, formal privacy guarantees are essentially meaningless across the evaluated differential privacy variants, although observed attack leakage remains relatively low.The experiments cover logistic regression and neural networks on two multi-class classification datasets.Source: §1 · p. 2
• Increasing the privacy budget improves utility but also increases inference-attack success, creating a direct utility–privacy balancing problem.The paper studies this relationship across different privacy budgets and model types.Source: §1 · p. 2

Definitions

• Differential privacy bounds information leakage for neighboring datasets through a privacy budget ϵ and failure probability δ.The mechanism is evaluated over datasets differing by one record, with δ representing the failure probability.Source: §2.1 · p. 2

Mechanisms

• Machine-learning mechanisms commonly add Laplace or Gaussian noise proportional to mechanism sensitivity, with this paper assuming ℓ2-sensitivity.Sensitivity is the maximum output change between neighboring datasets.Source: §2.1 · p. 2

Composition

• Basic composition adds privacy budgets linearly, motivating advanced, concentrated, zero-concentrated, and Rényi variants with tighter cumulative analyses.These variants bound cumulative privacy loss using properties of the privacy-loss distribution and generally yield sublinear growth with the number of compositions.Source: §2.1 · pp. 2–4

Privacy–utility trade-offs

• Tighter analyses can reduce the noise needed for a fixed guaranteed budget, improving utility while potentially reducing actual privacy because the mechanism adds less noise.The variants themselves analyze composition; the added noise determines concrete privacy, so relaxed definitions can permit less noise at the same ϵ.Source: §2.1 · p. 3

Machine-learning applications

• Complex learning tasks often require high privacy budgets, while deep learning lacks direct applicability of convex objective- and output-perturbation methods.High budgets can permit strong inference of record presence, and adapting convex approaches to non-convex learning may limit model capacity or require major infrastructure changes.Source: §2.3 · p. 5, §2.2 · p. 5

Evaluation focus

• The paper evaluates whether privacy variants and implementation choices translate into concrete leakage under membership and attribute inference attacks.Its experiments target the gap between formal privacy accounting and attacker-exploitable leakage in iterative learning.Source: §2.3 · p. 6

Membership Inference

• Membership inference attacks determine whether a record was included in a model’s training set, using black-box or white-box access assumptions.Attacks may exploit confidence scores, shadow models, or the model’s loss on an input record.Source: §3.1 · pp. 6, 8
• Membership inference is directly connected to differential privacy because both concern obscuring or detecting a record’s presence in the training data.The paper presents membership advantage as the difference between an adversary’s true-positive and false-positive rates.Source: §3.1 · p. 8

Attribute Inference

• Attribute inference attacks recover a hidden sensitive attribute from non-sensitive features and model outputs, often by maximizing a posterior probability.The paper also describes a loss-based variant that evaluates candidate sensitive-attribute values using the model’s empirical loss.Source: §3.2 · p. 8
• Differential privacy should also limit attribute leakage because sufficient noise limits the influence of a sensitive feature on the model output.This extends the record-level privacy intuition to attributes or features within a record.Source: §3.2 · p. 8

Other Attacks on Machine Learning

• The evaluation focuses on membership and attribute inference because other attacks leak application-specific information or do not closely match differential privacy’s threat model.Attacks requiring active interference with training are excluded under the assumed clean training process.Source: §3.3 · p. 9

Clipping

• Per-instance clipping improves logistic-regression accuracy for every ϵ, and RDP reaches accuracy close to the non-private model at ϵ = 100 on CIFAR-100.With batch clipping, none of the models learn anything useful, so subsequent results use per-instance clipping.Source: §4.1 · p. 10

Logistic Regression

• RDP and zCDP reach near-zero accuracy loss at ϵ = 50 and ϵ = 500 respectively, whereas naïve composition remains near random guessing for ϵ ≤ 10 on CIFAR-100.Advanced composition adds more noise than naïve composition when ϵ ≥ 100, making it unsuitable in those settings.Source: §4.2 · pp. 10–11

Inference Attacks

• At ϵ = 1000, membership leakage reaches 0.065 ± 0.004 for naïve composition and about 0.080 ± 0.004 for RDP and zCDP under the Shokri attack.Under the Yeom attack, naïve composition reaches 0.077 ± 0.003 at ϵ = 1000, while RDP and zCDP leak the most; RDP at ϵ = 10 has similar utility and leakage to naïve composition at ϵ = 500.Source: §4.2 · p. 11
• Attribute-inference leakage rises from 0.005 ± 0.007 at ϵ = 10 to 0.093 ± 0.002 at ϵ = 1000 under naïve composition.Across variants, increasing the privacy budget raises both attacker advantage and model accuracy.Source: §4.2 · p. 11

Repeated-Run Vulnerability

• Across repeated runs, membership predictions overlap substantially but provide only limited additional advantage on CIFAR-100, whereas Purchase-100 shows PPV 0.63 for members identified in all five runs.On CIFAR-100, the non-private model still identifies 5,179 members with PPV 0.56, indicating that the observed privacy benefit is not attributable solely to privacy noise; on Purchase-100, PPV 0.62 poses some threat with additional information.Source: §4.2 · pp. 12–13

Purchase-100

• At ϵ = 1000, RDP achieves accuracy loss close to 0.02 on Purchase-100, while naïve composition reaches 0.116 ± 0.003 and advanced composition 0.513 ± 0.003.RDP achieves the best utility across all tested privacy budgets, whereas naïve and advanced composition provide essentially no utility until ϵ exceeds 100.Source: §4.2 · p. 13

• The authors call for careful assessment of the practical privacy value of formal differential privacy claims when protecting sensitive data.Source: §5 · p. 16
• Practical combinations of ε values and differential privacy variants may provide unacceptable utility–privacy trade-offs.Source: §5 · p. 16
• A large gap remains between what state-of-the-art inference attacks can infer and the guarantees provided by differential privacy.Source: §5 · p. 16
• Further research is needed to understand inference-attack limitations and develop desirable, well-understood utility–privacy trade-offs.Source: §5 · p. 16