PaperlayerView exact revision on arXiv

Source-linked AI summary

Diffusion Sensitivity Geometry and Strong Duality for Optimal Privacy-Utility Tradeoffs

Tue Do, Daniel Alabi

arXiv:2610.28991v1cs.ITcs.CRmath.IT

TL;DR

Diffusion-model privacy methods can use loose sensitivity bounds and isotropic noise that overlook data geometry. This paper characterizes ideal-denoiser sensitivity exactly, derives privacy guarantees, and optimizes covariance through an SDP. The resulting dual certificates establish global optimality for the privacy-utility tradeoff.

  • Problem

    Existing private diffusion approaches use loose sensitivity bounds and isotropic Gaussian noise, although these calibrations ignore the geometry of the protected data.

    Source: §I · p. 1

  • Method

    The paper analyzes the ideal denoiser under a general-covariance Gaussian mechanism, characterizes sensitivity using record pairs, and formulates least-error covariance design as an SDP.

    Source: §I · p. 1

  • Results

    The record-pair characterization yields exact Rényi, approximate, Gaussian, and binary-record mutual-information privacy guarantees, and the covariance SDP has strong duality with attained optima.

    Source: §I · p. 1, §V · p. 4, §VII · p. 5

  • Takeaways & Limitations

    Dual solutions certify global optimality of the least-error covariance under the specified privacy budgets.

    Source: §I · p. 1

Abstract

from arXiv · show

Existing approaches to differential privacy in diffusion models rely on loose sensitivity bounds and isotropic Gaussian noise, leading to unnecessary estimation error. We develop a framework for the optimal privacy-utility tradeoff of ideal diffusion denoisers, replacing conservative bounds with exact sensitivity characterizations and provably optimal noise calibration. First, we show that worst-case sensitivity over datasets of arbitrary size can be characterized exactly using only two records. For a data domain of size $m$, this reduces the analysis to at most $m(m-1)$ constraints, regardless of dataset size. Our characterization can be arbitrarily tighter than conventional diameter-based sensitivity bounds, revealing potentially unbounded improvements in noise calibration. Second, we obtain exact privacy guarantees for R\'enyi, approximate, and Gaussian differential privacy, as well as mutual-information privacy for binary records, avoiding the looseness of previous generic privacy conversions. Finally, we show that the minimum achievable weighted mean-squared error can be computed through a semidefinite program and prove strong duality, providing certificates of global optimality.

I. INTRODUCTION

The introduction frames diffusion-denoiser privacy as a geometry-aware privacy-utility problem and proposes exact sensitivity, privacy accounting, and optimal covariance design. Related work motivates the gap, while the normalized empirical posterior mean connects the ideal denoiser to a conditional expectation.

I. INTRODUCTION

  • Diffusion models can expose training records, while DP-SGD’s fixed-norm clipping and isotropic noise ignore data geometry; the paper studies privacy at the ideal denoiser.The denoiser is released with a Gaussian mechanism using general covariance, and utility is measured by weighted mean-squared error.Source: §I · p. 1
  • Worst-case one-record sensitivity over datasets of any size reduces to two records, requiring at most m(m −1) constraints for a finite domain.For one covariance serving every query, the worst case is the largest Mahalanobis distance between domain points, which can be far smaller than a diameter-based bound.Source: §I · p. 1
  • A single ρ∗(Σ) gives exact RDP, zCDP, Gaussian-DP, and approximate-DP guarantees, while least-error covariance design under these budgets is an SDP with strong duality.The primal and dual optima are attained, and dual solutions certify global optimality.Source: §I · p. 1
  • For binary records, mutual-information privacy reduces to binary-input Gaussian-channel capacity, which is strictly increasing in SNR and yields another exact instance of the program.The monotonicity follows from the I-MMSE identity.Source: §I · p. 1

A. Related Work

  • Prior privacy-utility and Gaussian-mechanism work does not directly characterize diffusion denoisers, whose nonlinear data-dependent sensitivity set must first be determined.The cited related work includes privacy-funnel and MMSE approaches, staircase noise for scalar queries, and optimized correlated Gaussian noise for linear queries.Source: A. Related Work · pp. 1–2

A. Normalized empirical posterior mean

  • The ideal denoiser is a softmax-weighted average of records and, under the stated Gaussian observation model, equals the conditional expectation of the sampled record given the query.The formulation uses a rescaled diffusion query and noise scale for the ideal predictor.Source: A. Normalized empirical posterior mean · p. 2

B. Mechanism and a linear objective

The mechanism adds dataset-independent Gaussian noise with a fixed covariance, and its quadratic error is expressed through a linear weight. The exact deletion analysis ties a record’s effect to its posterior responsibility and its location relative to the remaining mean.

B. Mechanism and a linear objective

  • The expected additional quadratic loss is evaluated using W = LᵀQL, with W assumed positive definite for compactness and easier matrix square roots.A semidefinite task weight is also valid, but may leave directions with unbounded variance and an unattained infimum.Source: B. Mechanism and a linear objective · p. 2
  • The same covariance is used for every private dataset, independent of the data and the Gaussian draw.Mean prediction MSE uses W = I; adding noise to the ideal noise prediction instead changes the reconstruction cost by a time-dependent factor.Source: B. Mechanism and a linear objective · p. 2

A. Exact deletion

  • A deleted record’s effect depends on its posterior responsibility and its position relative to the mean of the remaining records.The posterior spread also determines the query Jacobian.Source: A. Exact deletion · p. 2
  • Replacing the ideal denoiser with a learned or nonideal denoiser results in a significantly different analysis.Source: A. Exact deletion · p. 2

B. Adjacency Notions

The paper treats add/remove privacy over nonempty finite multisets and replacement privacy at a fixed dataset size. For replacement adjacency, an add/remove Mahalanobis bound yields a replacement bound by the triangle inequality.

1) Add/Remove:

  • Add/remove privacy is defined over all nonempty finite multisets in the public domain K.Source: 1) Add/Remove · p. 3

2) Replacement:

  • Under replacement adjacency at fixed size N ≥ 1, neighboring datasets share all but one record, which is replaced by another record.Source: 2) Replacement · p. 3
  • A squared add/remove Mahalanobis bound ρ_ar implies a replacement bound of 4ρ_ar by the triangle inequality.Source: 2) Replacement · p. 3

C. A finite exact reduction for one fixed query

For a fixed query, the sensitivity analysis reduces exactly to two-record datasets, making a finite public domain tractable even when dataset size is unbounded. When one covariance must cover every query, the worst-case bound is determined by domain-point differences.

C. A finite exact reduction for one fixed query

  • The convex-hull argument uses convexity to show that maximizing a convex function over a set’s convex hull gives the same supremum as over the set.Source: C. A finite exact reduction for one… · p. 3
  • For fixed query g, the sensitivity set is characterized through terms generated by pairs of records, yielding an exact two-record reduction.The theorem equates the closed symmetric convex hull of the sensitivity set with that generated by the pairwise terms.Source: C. A finite exact reduction for one… · p. 3
  • A finite public domain of size m requires at most m(m−1) nonzero constraints for an exact finite SDP, despite unbounded dataset size.The constraints correspond to the unique pairwise terms generated from the domain.Source: C. A finite exact reduction for one… · p. 3
  • For a single covariance that works uniformly over query inputs, the worst-case sensitivity is bounded by the largest Mahalanobis distance between domain points.The proof establishes this by relating the pairwise terms to domain-point differences.Source: C. A finite exact reduction for one… · p. 3

IV. RÉNYI DP AND RELATED GAUSSIAN PRIVACY DEFINITIONS

A single sensitivity quantity determines the Gaussian mechanism’s Rényi and related Gaussian privacy guarantees. The paper also derives the exact approximate-DP curve, while noting that finite covariance cannot give pure DP for a nonzero shift.

IV. RÉNYI DP AND RELATED GAUSSIAN PRIVACY DEFINITIONS

  • The quantity ρ*(Σ), the largest squared Mahalanobis sensitivity, determines the smallest valid Rényi privacy parameter at every order.Source: §IV · p. 4
  • The mechanism is exactly (ρ*/2)-zCDP and √ρ*-GDP, with Rényi budgets equivalent to bounds on ρ*.Source: §IV · p. 4
  • The exact worst-case approximate-DP guarantee is δ = h_ε(ρ*); for any nonzero shift, finite Gaussian covariance cannot provide pure DP at finite ε.Source: §IV · p. 4

V. THE COVARIANCE PRIMAL AND ITS CONIC DUAL

The covariance optimization minimizes loss over a general Gaussian covariance subject to privacy constraints. Its conic dual has attained, equal-valued optima under the stated conditions, and dual feasibility yields a certificate of global optimality.

V. THE COVARIANCE PRIMAL AND ITS CONIC DUAL

  • Under the exact sensitivity hull, the covariance program minimizes loss among common-covariance Gaussian mechanisms satisfying the RDP budget.The passage states ρ = 2εα/α and distinguishes a fixed query from a uniform-query covariance.Source: §V · p. 4
  • For finite J, W ≻ 0, ρ > 0, and finite shifts, the primal and dual optima are both attained and have equal values.Source: §V · p. 4
  • The dual function is finite only when the covariance coefficient satisfies the dual constraint, identifying the stated program as the Lagrangian dual.The constraint is P_j A_j + H = W.Source: §V · p. 5
  • Slater’s condition establishes strong duality using strict primal feasibility and a lower bound on the primal objective.The passage supplies Σ = cI as a strictly feasible choice under its stated conditions.Source: §V · p. 5

A. Definition and conditional channel capacity

The section formulates mutual-information privacy through a channel for the record being protected while other records are fixed. Taking the supremum over priors reduces the privacy quantity to the capacity of such conditional channels.

A. Definition and conditional channel capacity

B. Exact binary record calibration and its dual

For binary records, the Gaussian mechanism induces a binary-input channel whose capacity provides an exact mutual-information privacy calibration. The resulting privacy constraint can be expressed using the full replacement sensitivity set, with a separate stated condition for add/remove adjacency.

B. Exact binary record calibration and its dual

VII. CONCLUSION

The conclusion presents exact sensitivity, privacy accounting, and covariance optimization as a framework for the ideal diffusion denoiser. The acknowledgment states that generative AI tools were used to generate and optimize code for the simulations and experiments.

VII. CONCLUSION

  • The framework characterizes sensitivity through record pairs, derives exact privacy guarantees, and uses strong duality to certify globally optimal noise design for the ideal diffusion denoiser.Source: §VII · p. 5

VIII. AI USE ACKNOWLEDGMENT

  • Generative AI tools were used to generate and optimize code for the simulations and experiments.Source: §VIII · p. 7

Related papers

Loading 2610.28991v1…