Performative Privacy: When Differential Privacy Maximizes Utility

Uddalak Mukherjee; Edwige Cyffers; Yann Chevaleyre

Source-linked AI summary

Performative Privacy: When Differential Privacy Maximizes Utility

Uddalak Mukherjee, Edwige Cyffers, Yann Chevaleyre

arXiv:2608.28198v1Aug 28, 2026cs.LGcs.AIstat.ML

TL;DR

The paper formalizes how privacy leakage can reduce future participation in repeated learning, addressing an unformalized connection between privacy protection and long-term utility. It introduces performative privacy through a model of repeated mean estimation with leakage-triggered departures, and shows that finite privacy can outperform non-private estimation when the participation feedback loop is sufficiently strong.

Problem
The paper studies the previously unformalized question of whether protecting users’ data can preserve participation and improve long-term utility.
Method
The paper introduces performative privacy and analyzes repeated mean estimation where data leakage triggers user departures under privacy-preserving mechanisms.
Results
Finite privacy can be optimal and private mean estimation can outperform non-private estimation in the long term when participation feedback is sufficiently strong.
Takeaways & Limitations
Utility maximization can align with strong privacy protection when repeated deployment creates a sufficiently strong feedback loop between leakage and participation.
Takeaways & Limitations
The analysis focuses on mean estimation and uses a simple memoryless participation model with fixed departure probability after leakage.

Abstract

from arXiv · show

Privacy-preserving learning is often motivated by the idea that protecting users' data can preserve trust and thus participation, improving utility in the long term. However, this claim has not been formalized so far. In parallel, performative learning provides a framework for studying learning systems whose deployment affects the data they later observe. In this work, we bring these two perspectives together and introduce \emph{performative privacy}, where data leakage reduces future participation. We study a simple model where agents repeatedly contribute data for mean estimation but may leave the system when their data is leaked. Privacy is implemented through differentially private mechanisms, creating a trade-off between estimation noise and future participation. We show, through a theoretical study of the dynamics and numerical experiments, that a finite privacy budget can outperform non-private estimation in the long term when the feedback loop between leakage and participation is sufficiently strong. This provides first evidence that differential privacy can be optimal not only as a protection mechanism, but also from the perspective of long-term utility.

1 INTRODUCTION

The paper introduces performative privacy to study how privacy leakage can reduce future participation in repeated learning. It develops models and analyses showing that a finite privacy budget can maximize long-term utility when this feedback loop is sufficiently strong.

The framework treats agents as participants who may withdraw after their data is leaked, linking privacy outcomes to future participation.
The paper models repeated mean estimation with differential privacy, creating a trade-off between estimation noise and participation dynamics.
Performative privacy studies the long-term utility of repeated learning tasks involving privacy-sensitive data.
The setting is extended from binary values to d-dimensional averages with membership inference attacks defining data leakage.
A finite privacy budget can be optimal in both studied scenarios, showing that privacy can maximize utility when participation feedback is sufficiently strong.

2 RELATED WORK

Prior work studies privacy, participation, performative learning, retention dynamics, and membership inference separately. This paper connects privacy leakage to user departure as a stateful performative process.

Earlier work analyzed privacy protection, participation incentives, and privacy-budget selection, but generally treated the optimal budget as known rather than dynamically determined.
Performativity in Trustworthy Machine Learning: Performative learning studies deployments that change future data distributions and optimizes the overall system rather than a model for a fixed distribution.
Performativity in Trustworthy Machine Learning: The paper’s closest performative connection is retention dynamics, but it studies users leaving after privacy leakage rather than after repeatedly receiving poor utility.
Membership Inference Attacks: Membership inference attacks show that aggregate statistics and model outputs can reveal whether an individual contributed to a dataset.

3 WARM-UP: ESTIMATION OF BERNOULLI PARAMETERS

The warm-up studies repeated Bernoulli mean estimation with randomized response while population size changes through recruitment and leakage-triggered departure. Its growth-factor analysis identifies when privacy can prevent population collapse.

Learning task: The learner repeatedly estimates a changing Bernoulli parameter from independent samples, targeting squared error at the final horizon or regret over the trajectory.
Population evolution: A leakage event occurs when randomized response preserves the true answer, after which a participant leaves with probability q.
Randomized response: Responses receive local differential privacy through randomized response, and the estimator is chosen to remain unbiased for a given privacy budget.
Population evolution: Participants whose data is not leaked can recruit an additional participant with probability r, while the model assumes no memory of prior leakage events.
Population evolution: The expected growth factor C(β) determines population trajectory: C(β) ≥1 indicates stability or growth, whereas C(β) <1 indicates decline.
Population evolution: If q > r, the population always collapses; if r > q, the privacy-dependent value of C(β) imposes a threshold on β.
Empirical Simulation: Simulations track population size and moving-averaged variance across privacy budgets, averaging results over 30 independent trials.

4 MEAN ESTIMATION WITH MEMBERSHIP INFERENCE DETECTION

The paper extends performative privacy to continuous d-dimensional mean estimation, where membership leakage drives user departures and population dynamics. It derives leakage and utility characterizations, survival thresholds, and a numerically tractable finite-horizon optimization of the noise scale.

Mean estimation and membership inference: The model estimates a d-dimensional population mean with clipped records and Gaussian-mechanism noise, while user vulnerability is assessed through membership inference.Records are clipped to radius R, and the private output is an empirical mean.
Mean estimation and membership inference: The leakage threshold τ is based on a record’s log-likelihood-ratio skew, with compromise occurring when Λ(ot|z) > τ.The threshold is kept constant for simplicity, although fixed false-positive rates would generally require scaling with d and Nt.
Gaussian leakage analysis: The Gaussian analysis gives an exact leakage probability and shows that outliers face exponentially increasing leakage risk as their deviation from the empirical mean grows.Increasing population size decreases divergence through aggregation-based privacy amplification.
Population evolution: The expected population evolution is governed by a growth factor determined by marginal leakage probability and parameters including d, τ, R, and γ.This characterization replaces dependence solely on the privacy parameter with an integral capturing continuous-data geometry and mechanism noise.
Population survival and utility: A critical noise scale γc is sufficient to ensure Ct(γ) ≥ 1 and prevent demographic collapse, while the expected Mahalanobis error is bounded conditional on population size.The model therefore links minimum privacy noise to population survival and estimation utility.
Population survival and utility: For finite horizons, the paper optimizes a smooth upper bound on cumulative Mahalanobis error, with theoretical optimal γ closely aligning with empirical grid-search parameters.The upper bound shares the exact cumulative error’s convergence rate in the growth regime and can be solved numerically.

5 DISCUSSION AND FUTURE WORK

The paper finds that private mean estimation can outperform non-private estimation in the long term when leakage causes participants to drop out, while identifying extensions to richer tasks and feedback dynamics.

Private mean estimation can outperform non-private estimation in the long term when participants leave after their data is leaked.
The result shows that utility maximization can align with strong privacy protection when the leakage-participation feedback loop is sufficiently strong.
The study introduces performative privacy as the long-term interplay between privacy and utility under repeated model deployment.
Future work could extend the framework beyond mean estimation and use more realistic feedback models, privacy preferences, and privacy-budget schedules.

A MORE EXPERIMENTS ON MEMBERSHIP INFERENCE LEAVING PROCESS

The experiments compare theoretical and empirical optimal leakage thresholds across population regimes and show that finite-population effects create larger theory–experiment discrepancies for larger τ.

Figure 5 compares theoretical and empirical optimal γ using heatmaps and join-probability curves for three τ settings.
As τ increases, empirical noise increasingly underestimates the theoretical prediction because leakage becomes more sensitive to rare, extreme outliers.
The experimental population cap of 20,000 limits extreme samples, making the theory–experiment discrepancy more pronounced at larger τ.
Figure 6 contrasts a growing regime, where the system reaches capacity, with a shrinking regime, where performative collapse eliminates the initial advantage of weak privacy.

B THE AGNOSTIC MODEL

In the agnostic model, population growth depends on prior participation regardless of leakage, so rapid growth favors minimal privacy while strict privacy helps only near the growth–attrition boundary.

Population growth is proportional to the previous iteration’s participant count, regardless of whether those participants’ data was leaked.
When r > q, rapid growth drives the population to maximum capacity regardless of the privacy budget, and the highest ε yields the lowest variance.
When q > r, the variance follows the original model’s pattern under the agnostic population dynamics.
For growing populations, the agnostic model overwhelmingly favors minimal privacy, unlike the original model’s broad transition toward intermediate privacy.
Maximum privacy is beneficial only within a narrow band just below r = q; outside this regime, strict privacy provides no utility gain.

C PROOFS FROM SECTION 3

This section proves unbiasedness of the privatized Bernoulli mean estimator, derives participant-population dynamics, and establishes a critical perturbation threshold for population survival.

The estimator ˆµt for the population Bernoulli mean µt is unbiased.
The unbiasedness proof computes the expected privatized response, then applies linearity of expectation to the sample mean and final estimator.
The perturbation parameter β is selected to satisfy ε-local differential privacy through the maximum ratio of true- to false-reporting probabilities.
The participant population evolves through a balance equation combining arrivals and departures, with deterministic expectations substituted into the dynamics.
For sufficiently large populations, arrivals and departures concentrate around their expected Binomial values by the Law of Large Numbers.
Maintaining C(β) ≥ 1 requires β ≤ βc, with βc = (r − q) / 2(q + r) when r > q and clipping to zero when r ≤ q.

D.1 GENERIC RESULTS

The generic analysis derives the conditional Mahalanobis error and bounds leakage using Rényi divergence and exponential-tail arguments.

Conditional estimation error: The conditional expected squared Mahalanobis error is expressed for a dataset with true mean µ and covariance Σ, under negligible clipping bias.
Conditional estimation error: The expected squared Mahalanobis error separates into privacy-noise variance and intrinsic data-sampling variance.The derivation assumes independent injected noise and uses the trace trick for quadratic forms.
Leakage analysis: The factual and counterfactual mechanism-output distributions define structural leakage for a target record.The leakage tail is evaluated at threshold τ and Rényi order α > 1.
Leakage analysis: For any α > 1, the leakage tail is bounded by applying the Cramér–Chernoff technique and Markov’s inequality to an exponential transform of the log-likelihood ratio.The proof expands the likelihood-ratio definition, isolates the expectation, substitutes it into the numerator, and combines exponents.

D.2 RESULTS FOR GAUSSIAN MECHANISM

For the Gaussian mean mechanism, the analysis characterizes Gaussian leakage, obtains an exact purge probability, and derives a noise threshold ensuring non-collapsing population dynamics.

Gaussian leakage: The Rényi divergence between adjacent Gaussian output distributions reduces to a scaled Mahalanobis distance between their means.The distributions share covariance Cγ, and the mean shift depends on the target record and dataset sizes.
Gaussian leakage: The Gaussian mechanism adds isotropic noise with covariance scaled by γ2/Nt, enabling an explicit conditional log-likelihood ratio.The likelihood ratio compares outputs with and without a target record.
Gaussian leakage: The conditional log-likelihood ratio is Gaussian, with randomness arising from an inner product involving the aggregated noise vector.Its conditional variance follows from Gaussian noise covariance and scaling by the dataset size.
Gaussian leakage: The exact probability that a record is purged is the Gaussian tail probability for its log-likelihood ratio exceeding τ.The closed form uses the standard normal cumulative distribution function Φ.
Population evolution: The expected population evolves through a growth factor determined by the marginal leakage probability pτ and parameters d, τ, R, and γ.
Population evolution: γ ≥ γc guarantees an expected growth rate C(γ) ≥ 1, preventing demographic collapse under the stated clipping-bound condition.The threshold is obtained by restricting expected marginal leakage and solving the resulting quadratic bound.

D.4 BOUNDS AND ASYMPTOTICS OF THE ERROR

The error bounds show a finite-horizon and asymptotic trade-off: vanishing noise can cause population collapse, whereas sufficient noise enables population growth and eventual error decay.

Collapse and finite-sample bounds: Theorem 3 bounds the expected squared Mahalanobis error from both below and above when the population is floored at one participant.
Collapse and finite-sample bounds: For q > 0, zero algorithmic noise makes expected estimation error diverge over time under the mean-field population dynamics.As γ → 0+, leakage drives the population toward N0(1−q)^t, shrinking the denominator in the error bound.
Population growth: If C_t(γ) > 1 for every round, the expected growth factor is strictly increasing under the mean-field approximation.Population growth makes inverse population size shrink, increasing the growth factor.
Asymptotics: O(C1(γ)^−t) is the asymptotic error-decay rate when C_t(γ) > 1, with the fastest rate in the infinite-noise limit γ → ∞.In that limit, leakage vanishes and C(γ) approaches 1 + r.
Asymptotics: Even as q → 1, the expected error still converges to zero for r > 0 when C(γ) > 1.The multiplicative constant may grow, but the exponentially growing population eventually dominates fixed noise and numerator terms.
Finite horizon: The cumulative expected Mahalanobis error over a finite horizon T is analyzed by summing single-round errors under a positive per-round growth factor.

E PAC GUARANTEE FOR MAHALANOBIS ERROR AND COMPARISON WITH ℓ2

The paper measures noisy mean-estimation accuracy with scale-invariant Mahalanobis error and derives a finite-sample PAC guarantee, contrasting its dependence with standard differential privacy.

Error measure: Mahalanobis error accounts for covariance structure and, unlike ℓ2 error, is dimensionless and scale-invariant.For non-isotropic covariance, whitening incorporates directional variances and correlations.
PAC guarantee: The PAC analysis bounds the combined effects of sampling variability and independent Gaussian noise for a noisy empirical mean.The theorem provides a finite-sample, high-probability guarantee parameterized by ϵ and δ.
Comparison with ℓ2: Under standard DP, the sample size grows linearly with injected noise variance γ2 and dimension d.The paper states that its setting has a substantially different dependence.