PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Learning with Gaussian Differential Privacy

Zhiqi Bu, Jinshuo Dong, Qi Long, Weijie J. Su

arXiv:1911.11607v3cs.LGcs.CRstat.ML

TL;DR

Sensitive data and the weaknesses of existing privacy analyses for composition and subsampling motivate a refined framework for private deep learning. The paper applies f-DP to analyze SGD and Adam, deriving closed-form bounds and showing stronger privacy guarantees and improved prediction accuracy. Experiments support these improvements across image classification, text classification, and recommender systems.

  • Problem

    Existing privacy definitions can provide loose or complicated analyses of deep-learning training because composition and subsampling are difficult to handle tightly.

  • Method

    The paper applies f-DP to private deep learning with SGD and Adam, using its composition and subsampling properties to derive analytically tractable privacy bounds.

  • Results

    The f-DP analysis yields a sharper privacy bound than the moments accountant and supports stronger privacy guarantees and improved prediction accuracy.

  • Takeaways & Limitations

    The results suggest that less training noise can provide the same privacy guarantees as the moments accountant while improving model utility.

  • Takeaways & Limitations

    The paper leaves complex neural-network architectures and broader optimizer guidance for future research.

Abstract

from arXiv · show

Deep learning models are often trained on datasets that contain sensitive information such as individuals' shopping transactions, personal contacts, and medical records. An increasingly important line of work therefore has sought to train neural networks subject to privacy constraints that are specified by differential privacy or its divergence-based relaxations. These privacy definitions, however, have weaknesses in handling certain important primitives (composition and subsampling), thereby giving loose or complicated privacy analyses of training neural networks. In this paper, we consider a recently proposed privacy definition termed \textit{$f$-differential privacy} [18] for a refined privacy analysis of training neural networks. Leveraging the appealing properties of $f$-differential privacy in handling composition and subsampling, this paper derives analytically tractable expressions for the privacy guarantees of both stochastic gradient descent and Adam used in training deep neural networks, without the need of developing sophisticated techniques as [3] did. Our results demonstrate that the $f$-differential privacy framework allows for a new privacy analysis that improves on the prior analysis~[3], which in turn suggests tuning certain parameters of neural networks for a better prediction accuracy without violating the privacy budget. These theoretically derived improvements are confirmed by our experiments in a range of tasks in image classification, text classification, and recommender systems. Python code to calculate the privacy cost for these experiments is publicly available in the \texttt{TensorFlow Privacy} library.

1 Introduction

The paper addresses privacy risks in sensitive machine-learning data and the difficulty of analyzing composition and subsampling under existing privacy definitions. It develops an f-DP-based analysis of private deep learning that yields closed-form bounds, stronger guarantees, and improved prediction accuracy.

  • Motivation: Sensitive datasets and inference attacks motivate privacy-preserving machine-learning methods.The motivating examples include location, contacts, media consumption, and medical records.
  • Limitations of prior analyses: (ε, δ)-DP is rigorous but difficult to use tightly for deep-learning composition, while divergence relaxations may lack usable subsampling analyses.The moments accountant addressed these challenges through numerical tracking of privacy loss.
  • f-DP framework: f-DP supports principled analysis of composition and privacy amplification by subsampling while retaining a hypothesis-testing interpretation.GDP is a canonical f-DP family supported by a central-limit-theorem characterization of composition.
  • Contributions: The paper derives an amenable closed-form privacy bound for private deep-learning training, unlike the moments accountant’s numerical computation.The closed-form expression makes the effect of tuning parameters on privacy bounds more transparent.
  • Contributions: The f-DP approach gives stronger privacy guarantees than the earlier approach, including when expressed in (ε, δ)-DP.The paper attributes this improvement to the central limit theorem’s treatment of per-iteration privacy loss.
  • Utility improvement: f-DP permits trading some privacy for improved prediction accuracy by reducing training noise to match a target (ε, δ)-DP level.Experiments cover image classification, text classification, and recommender systems.

2 Preliminaries

The preliminaries formulate privacy through neighboring datasets and hypothesis testing, then introduce f-DP, GDP, composition, and subsampling properties used for private deep-learning analysis.

  • Privacy setup: Differential privacy considers whether an algorithm’s output reveals an individual’s presence in one of two neighboring datasets.The adversary tests the distributions produced on datasets differing by one individual.
  • f-Differential privacy: f-DP measures privacy through the trade-off between type I and type II errors in testing two output distributions.A larger trade-off function means the testing problem is harder and therefore indicates more privacy.
  • f-Differential privacy: f-DP is interpretable through hypothesis testing and contains (ε, δ)-DP as a special instance, with a dual conversion between the definitions.The conversion yields δ(ε) from the convex conjugate of the trade-off function.
  • Gaussian differential privacy: µ-GDP compares membership testing difficulty with distinguishing N(0, 1) from N(µ, 1), and Gaussian noise achieves µ-GDP when σ = sens(θ)/µ.The Gaussian trade-off function has a closed form involving the standard normal CDF Φ.
  • Composition: Composition releases the information from a sequence of analyses, where later algorithms may depend on earlier outputs.For f-DP, composition is represented by an associative and commutative operation on trade-off functions.
  • Composition: For many very private composed algorithms, a central-limit-theorem approximation makes the accumulated privacy loss close to GDP.This is especially suited to deep learning’s large number of training iterations, while (ε, δ)-DP does not losslessly track composition.
  • Subsampling: f-DP provides a privacy-amplification-by-subsampling bound that strictly improves the corresponding (ε, δ)-DP subsampling theorem.This property is relevant because subsampling is intrinsic to neural-network training.

3 Algorithms and Their Privacy Analyses

The paper analyzes private SGD and Adam using f-DP's composition and subsampling properties, deriving tractable privacy bounds and comparing them with the moments accountant. The f-DP CLT approach yields stronger privacy guarantees and supports utility-oriented parameter tuning.

  • NoisySGD: NoisySGD clips gradients, adds Gaussian noise, and uses Poisson subsampling before composing privacy guarantees across iterations.Clipping bounds sensitivity, while Gaussian noise and f-DP's composition and subsampling properties support the overall analysis.
  • CLT privacy analysis: The central limit theorem gives an analytically tractable approximation in which NoisySGD is approximately p√(T(e^(1/σ^2)−1))-GDP.The bound indicates better privacy when the mini-batch fraction is small and the noise scale is not too small.
  • NoisyAdam: NoisyAdam has the same f-DP privacy bound as NoisySGD because its momentum updates are deterministic post-processing of noisy gradients.The same reasoning also applies to AdaGrad in the stated asymptotic regime.
  • Comparison with the Moments Accountant: The f-DP CLT approach is asymptotically sharper than the moments accountant for NoisySGD and NoisyAdam in both f-DP and (ε, δ)-DP terms.At the same ε, the CLT approach can achieve a smaller δ than the moments accountant, while the moments accountant requires numerical computation.
  • Comparison with the Moments Accountant: The refined privacy analysis can trade privacy guarantees for improved utility, including prediction accuracy, without exceeding the privacy budget.The paper also reports numerical comparisons consistently demonstrating the superiority of the CLT approach.

4 Results

Experiments evaluate private neural networks across image, text, recommendation, and binary-classification tasks. The f-DP/CLT analysis provides stronger or more interpretable privacy guarantees and can support utility improvements through reduced noise, while complex architectures remain outside the study’s scope.

  • Experimental setup: Experiments cover MNIST, IMDb, MovieLens 1M, and Adult income using NoisySGD or NoisyAdam on sensitive-data tasks.The study includes image classification, text classification, recommender systems, and regular binary classification.
  • The f-DP Perspective: Across six MNIST tuning settings, the CLT approach gives significantly smaller ε than the moments accountant for δ = 10^-5.The comparison also emphasizes that f-DP gives a more comprehensive interpretation than (ε, δ)-DP.
  • The f-DP Perspective: For a model with 96.6% test accuracy, the minimum sum of type I and type II errors is at least 77.6% under CLT versus 9.4% under the moments accountant.Figure 3 presents these privacy bounds as hypothesis-testing trade-off functions.
  • Scope: The experiments focus on shallow neural networks and leave complex architectures for future research.This limits the demonstrated scope of the empirical evaluation.
  • The f-DP Perspective: On Adult income, the private model achieves test accuracy comparable to baselines in the MLC++ library.The reported NoisySGD model is a single-layer multilayer perceptron with 16 neurons and ReLU activation.
  • The f-DP Perspective: On MovieLens 1M, the private model has a relatively large RMSE difference from the non-private network but outperforms several popular non-private models.Those models include user-based collaborative filtering and nonnegative matrix factorization.
  • The (ε, δ)-DP Perspective: At ε = 4, the CLT approach reaches 96% test accuracy while the moments accountant reaches 92%.Using less noise also accelerates MNIST convergence: 95%, 96%, and 97% accuracy require 18, 26, and 45 epochs versus 23, 33, and 64.

5 Discussion

The paper concludes that f-DP provides sharper privacy analysis for private deep learning with SGD and Adam, and that this can improve model utility. It also identifies broader extensions for future work.

  • f-DP enables closed-form privacy bounds for private deep learning with SGD and Adam.Its composition and subsampling properties support the paper’s privacy analysis.
  • 1.13-GDP can correspond to (7.10, 10^-5)-DP under conservative (ε, δ)-DP accounting.The paper uses this contrast to characterize models as private from the f-DP viewpoint but not under the conservative bound.
  • Using less training noise at the same privacy guarantee can improve model utility.The paper frames this as a consequence of conservative moments-accountant bounds.
  • Future work includes time-dependent noise and learning rates, more complex architectures, optimizer-selection guidance, hyperparameter tuning, and adversarial robustness.These directions target better privacy–utility trade-offs and broader applicability.

A Omitted Details in Section 2

The appendix supplies omitted details for f-DP subsampling, composition, and the central-limit approximation used for SGD privacy analysis. Numerical evidence shows rapid agreement between the computed trade-off function and its GDP limit.

  • Subsampling: Proposition A.1 establishes the subsampling privacy relation for an f-DP mechanism applied after Poisson sampling.The proof represents sampled datasets as mixtures of neighboring distributions.
  • Subsampling: The mixture lemma transfers the f-DP guarantee from neighboring subsample components to the full sampled mechanism.Each component pair consists of the mechanism applied with and without the additional individual.
  • CLT approximation: The appendix compares the composed trade-off function with the GDP limit predicted by the central limit theorem.The numerical setup uses σ = 1.1, final GDP parameter µ = 0.57, and test accuracy 96.6%.
  • CLT approximation: The computed trade-off function and GDP prediction are almost identical after one epoch.The comparison uses 234 numerical iterations to illustrate early convergence.
  • Numerical caveat: The appendix leaves a rigorous and complete study of a numerical-computation challenge to future work.The challenge arises when e^(1/σ^2) − 1 becomes large.

B.1 Privacy Property of Algorithms 1 and 2

This section proves privacy guarantees for Algorithms 1 and 2 by combining per-iteration subsampling bounds with a refined composition theorem. NoisySGD and NoisyAdam receive the same resulting f-DP guarantee.

  • Privacy theorem: Algorithms 1 and 2 are both min{f, f^-1}**-DP under the theorem’s composition analysis.The bound is established for neighboring datasets and then composed across iterations.
  • Composition: The standard composition lower bound is insufficient because the relevant trade-off function is not necessarily lower bounded by fp.The appendix therefore requires a more specific composition theorem than the one stated in the foundational work.
  • Composition: The proof applies a refined composition theorem to the T-fold composition of the gradient-update mechanism.The composition theorem is invoked after establishing the per-step relation for both algorithms.
  • NoisyAdam: NoisyAdam has the same privacy property as NoisySGD in each iteration.The appendix states that the same argument therefore applies to NoisyAdam.

B.2 Justifying CLT for Algorithms 1 and 2

The appendix justifies the asymptotic GDP approximation for the algorithms using an asymmetric privacy central limit theorem and moment calculations. Under the stated scaling and integrability conditions, the composed mechanism converges uniformly to a Gaussian trade-off function.

  • Asymptotic theorem: Theorem 6 gives uniform convergence to a GDP limit when p√T → ν under stated regularity conditions.The theorem assumes conditions on f and an integrability bound involving the fourth power of f′(x) + 1.
  • Asymmetric CLT: The asymmetric central limit theorem is needed because the subsampled trade-off function pG1/σ + (1−p)Id is not symmetric.The appendix introduces additional functionals to handle this setting.
  • Limit calculations: The proof identifies the limiting variance scale through χ2(f), with K and s^2 equal to ν^2·χ2(f).The calculation uses the χ2-divergence representation and limits of the relevant functionals.
  • Limit calculations: The third-order terms vanish under the sampling-rate scaling, completing the GDP-limit calculation.The appendix states that T·κ3(fp) → 0 when p√T → ν.

B.3 Proof of Theorems 1 and 2

This section relates the moments accountant and central-limit-theorem privacy analyses through f-DP and (ε, δ)-DP perspectives. It derives the theorem comparison by expressing the accountant quantities with Rényi divergence and taking asymptotic limits.

  • Theorem comparison: Theorems 1 and 2 compare the CLT approach with the moments accountant from f-DP and (ε, δ)-DP perspectives, respectively.The section first derives Theorem 1 from Theorem 2 and then proves a refined version of Theorem 2.
  • Privacy quantities: δMA(ε; σ, p, T) is the moments-accountant privacy cost for NoisySGD with subsampling probability p, iteration count T, and noise scale σ.The corresponding CLT quantity δCLT(ε; σ, ν) is defined for the same algorithm under a central-limit-theorem assumption.
  • Asymptotic proof: Theorem 2’s strict supporting-function gap from GµCLT transfers through the limit, showing that lim supT→∞ fT(α) retains at least that gap.The proof uses convergence of fT to GµCLT, including uniform convergence and derivative convergence, together with a supporting-function construction.
  • Moments accountant: The numerical moments accountant uses a discretized infimum over Rényi orders, producing δnMA(ε; σ, p, T), which upper-bounds δMA.The analysis uses the scaled Rényi divergence of a Gaussian mixture, with GMp,σ = pQ + (1 − p)P for Gaussian distributions P and Q.
  • CLT analysis: The CLT analysis first establishes asymptotic µCLT-GDP for Algorithm 1, with µCLT determined by ν and the algorithm’s parameters.The supplied passage introduces the asymptotic GDP relation but does not provide the completed expression for µCLT.
  • Technical ingredients: The proof connects Gaussian-mixture Rényi divergences to trade-off functions and uses Rényi-divergence tensorization before converting GDP guarantees to (ε, δ)-DP.The appendix establishes the needed bridge through identities relating αGM and fT, followed by limiting arguments.
Loading 1911.11607v3…