PaperlayerView exact revision on arXiv

Source-linked AI summary

Privacy for Free: How does Dataset Condensation Help Privacy?

Tian Dong, Bo Zhao, Lingjuan Lyu

arXiv:2206.00240v1cs.CRcs.LG

TL;DR

Existing differentially private generators trade privacy for data quality and training efficiency, motivating whether both goals can be achieved together. The paper uses dataset condensation, connects it theoretically to differential privacy, and evaluates visual and membership privacy. DC achieves privacy-preserving, data-efficient training, with a one-sample parameter-distribution effect bounded by O(m/n) and favorable empirical comparisons.

  • Problem

    Differentially private generators can suffer poor data quality and expensive training, raising whether privacy and training efficiency can be achieved simultaneously.

  • Method

    The paper connects dataset condensation to differential privacy theoretically and evaluates DC-synthesized data with visual comparison, loss-based MIA, and LiRA.

  • Results

    DC limits the one-sample effect on trained-model parameter distributions to O(m/n) and empirically preserves membership privacy, visual privacy, and training efficiency.

  • Takeaways & Limitations

    DC-synthesized data can provide privacy-preserving model training while retaining data efficiency, including at least 2 times faster training than GAN-based methods in the reported comparison.

  • Takeaways & Limitations

    The authors leave generalization of the theoretical findings to other DC methods and formal DP-satisfying DC methods as future work.

Abstract

from arXiv · show

To prevent unintentional data leakage, research community has resorted to data generators that can produce differentially private data for model training. However, for the sake of the data privacy, existing solutions suffer from either expensive training cost or poor generalization performance. Therefore, we raise the question whether training efficiency and privacy can be achieved simultaneously. In this work, we for the first time identify that dataset condensation (DC) which is originally designed for improving training efficiency is also a better solution to replace the traditional data generators for private data generation, thus providing privacy for free. To demonstrate the privacy benefit of DC, we build a connection between DC and differential privacy, and theoretically prove on linear feature extractors (and then extended to non-linear feature extractors) that the existence of one sample has limited impact ($O(m/n)$) on the parameter distribution of networks trained on $m$ samples synthesized from $n (n \gg m)$ raw samples by DC. We also empirically validate the visual privacy and membership privacy of DC-synthesized data by launching both the loss-based and the state-of-the-art likelihood-based membership inference attacks. We envision this work as a milestone for data-efficient and privacy-preserving machine learning.

1. Introduction

The paper asks whether dataset condensation can provide privacy while retaining training efficiency, addressing the quality and efficiency costs of differentially private generators. It connects DC to differential privacy and reports theoretical and empirical evidence for privacy, utility, and efficiency.

  • Differentially private generators can have low-quality outputs, requiring more generated data and reducing training efficiency.The added noise impedes model accuracy and increases the data needed for generalization.
  • Dataset condensation compresses large training sets into small synthetic sets that support data-efficient deep-network training.
  • DC connects to differential privacy through a parameter-distribution change bounded by O(m/n) when one raw sample is added or removed.The result is established for linear DC extractors and extended analytically and empirically to nonlinear feature extractors.
  • 33.4% higher test accuracy is achieved by DC than by DP-generators on FashionMNIST under the same empirical privacy budget.
  • At most 50% of GAN-based data is needed by DC to reach the same test accuracy, speeding training by at least 2 times.
  • Experiments report zero adversary advantage for membership privacy and perceptual irreversibility under L2 and LPIPS similarity metrics.

2. Background and Related Work

This section introduces dataset condensation as synthetic-set learning and reviews membership-inference attacks used to assess privacy. It covers condensation objectives and methods alongside loss-based and likelihood-based attack procedures.

  • Dataset Condensation: Dataset condensation learns a small synthetic set whose trained networks achieve testing performance comparable to networks trained on the target dataset.
  • Dataset Condensation: Gradient matching condenses data by matching gradients from real and synthetic batches, with differentiable Siamese augmentation improving training with augmentation.
  • Dataset Condensation: Distribution matching learns synthetic data by aligning real and synthetic data distributions in randomly sampled embedding functions.The embedding functions are randomly sampled feature extractors, and differentiable Siamese augmentation is used in the objective.
  • Membership Privacy: Loss-based membership inference classifies an input as a member when its predicted loss is below a threshold.The threshold may be selected using shadow models or an optimal Bayesian strategy.
  • Membership Privacy: Likelihood Ratio Attack estimates membership from differences between victim-model confidence and confidence distributions of IN and OUT shadow models.The attack thresholds the resulting likelihood ratio, and low false-positive rates are emphasized for evaluating real membership threats.

3. Problem Statement

The threat model considers organizations that send synthetic data derived from personal data to servers for downstream model training. The adversary seeks dataset membership with strong access and computational capabilities.

  • Sending raw personal data to servers can expose it to membership attacks and potential leakage by honest-but-curious operators.
  • The proposed protocol learns from a raw target dataset T, generates a synthetic dataset S, and sends S for downstream model training.
  • The adversary infers whether a target sample x belongs to the target dataset T.
  • The adversary has white-box access to S and the model trained on S, knows the distribution of T, and lacks direct access to T.
  • The threat model grants unlimited computation for generating shadow synthetic datasets and training shadow models from the same distribution as T.

4. Theoretical Analysis

The analysis connects dataset condensation with privacy by characterizing how synthetic data depend on the target dataset and how removing one non-initialization sample affects trained-model parameter distributions. Under stated assumptions, the resulting privacy leakage is limited when the synthetic set is much smaller than the target set, while initialization choice remains an important boundary.

  • Analysis setup: Theoretical analysis studies DM-based dataset condensation because of its condensation efficiency and utility for model training.The analysis focuses on the relationship between target and synthetic datasets and the privacy guarantees this relationship provides.
  • Synthetic-data structure: For linear extractors, the analysis derives properties of DM minimizers, including relationships between synthetic-data barycenters and the target dataset.The derivation is conducted under assumptions on the target-data span and DM convergence, with extensions discussed for nonlinear extractors.
  • Initialization effects: Initialization affects privacy: samples used for real-data initialization can remain inferable, whereas data not used for initialization have only O(1/|T|) weights in synthetic data.The paper proposes hard-to-infer initialization samples as a countermeasure and reports limited effects for data outside initialization.
  • Synthetic-data structure: Nonlinear-extractor analysis uses distance-preserving embeddings to generalize the relationship between real and synthetic data beyond the linear case.The paper specifically discusses two-layer random networks with ReLU activation and corresponding distance bounds.
  • Privacy bound: For a removed sample x not used in initialization, model parameter distributions change by O(|S|/|T|), limiting membership information when |S| ≪ |T|.The result models privacy leakage through parameter distributions and interprets the bound under the paper’s MIA threat model.
  • Privacy bound: The empirical privacy budget is not equivalent to the formal DP budget, but DC-trained models achieve ˆϵ ≈2 against LiRA while attaining higher accuracy than DP generators.The comparison is made under the paper’s black-box MIA interpretation of model privacy.

5. Evaluation

The evaluation tests membership privacy, utility, training efficiency, and visual privacy across DC methods and generator baselines. Randomly initialized DC-synthesized data resist loss-based and likelihood-based membership inference while preserving utility and reducing the data needed for training.

  • Membership Privacy: Real-data initialization can still leak membership privacy because initialization samples receive lower victim-model losses than other samples.On CIFAR-10 and CelebA, leakage is lower than direct real-data training, but FashionMNIST reverses this comparison.
  • Membership Privacy: Randomly initialized DC models show approximately zero membership-inference advantage across cGAN, DSA, DM, and KIP for all evaluated ripc values.The result indicates that adversaries cannot infer membership of the condensation training set from the victim models, although GAN training data may remain vulnerable when generated images are accessible.
  • Comparison with Different Generators: Under ripc = 0.01, DM and DSA outperform other methods under the same training-efficiency constraint, while KIP can underperform because its loss is hard to converge for large ripc.For small ripc, KIP significantly outperforms baselines on CIFAR-10 and CelebA.
  • Comparison with Different Generators: DC improves training efficiency by at least 2 times over cGAN at ripc = 0.01 because cGAN requires more synthetic data to match DC accuracy.The comparison uses the ripc value at which cGAN reaches the accuracy of models trained on DSA- or DM-synthesized data.
  • Visual Privacy: DC-synthesized data lose private properties of real data, preventing adversaries from inferring privacy from models trained on the synthetic data.Visual evaluation compares synthetic images with nearest real images using L2 distance and LPIPS similarity metrics.

6. Discussion and Conclusion

The paper audits dataset condensation as a privacy-preserving approach, combining theoretical analysis with empirical evaluations of visual and membership privacy. It identifies remaining scope limitations in the theoretical generalization and task coverage.

  • Contributions: The study provides a systematic audit of dataset condensation through theoretical privacy analysis and empirical visual and membership-privacy evaluations.The evaluations cover loss-based MIA and LiRA on FashionMNIST, CIFAR-10, and CelebA.
  • Limitations and future work: Theoretical findings have not yet been generalized to other DC methods, motivating analysis through information loss and formal Rényi differential privacy.The paper specifically identifies (α, ϵ)-Rényi DP as a direction for future work.
  • Limitations and future work: Current DC privacy evidence mainly concerns image classification, leaving object detection, text, and graph applications as extension directions.The paper notes that DC can generalize to other machine-learning tasks, but privacy benefits remain to be extended there.

A. Proof of Proposition 4.3

The analysis models DM with a linear feature extractor and random Gaussian parameters, then expresses its objective as an expectation over the squared representation discrepancy. Its gradient moves each synthetic sample toward aligning the synthetic and target-set barycenters.

  • Setup: The proof assumes a linear extractor ψ_θ(x)=θx with k<d and i.i.d. standard-normal extractor parameters.Differentiable Siamese augmentation is omitted to simplify the analysis.
  • Objective: DM’s objective becomes the expectation of the squared discrepancy d_DM over θ∼N(0,1).The synthetic set S is optimized with SGD using gradients of this expected objective.
  • Optimization: The gradient for a synthetic sample determines the SGD update direction used to optimize S.The analysis introduces this gradient after sampling a model parameter θ.
  • Optimization: The expected optimization direction moves the barycenter of S toward the barycenter of T until they coincide at a minimizer.This interpretation follows from the vanishing left-hand side at the assumed minimizer.

B. Proof of Proposition 4.4

For real-data initialization, DM preserves an initialization-related component and can therefore retain membership information, while random initialization separates synthetic samples into target-span and orthogonal components. The orthogonal component remains noise-like, and the target-span component is optimized toward the target data structure.

  • Real data initialization: Real-data initialization optimizes samples until the synthetic and target-set barycenters coincide.The proof treats each initialized synthetic sample as a sampled target point.
  • Real data initialization: A minimizer can retain initialization information, creating a membership-privacy risk when the synthetic set and target set are large.For |S| and |T| greater than 50, the analysis states s_i*≈x_i.
  • Random initialization: Random initialization draws synthetic samples from a standard multivariate normal distribution and decomposes them across span(T) and its orthogonal complement.The decomposition uses the covariance-preserving behavior of orthogonal transformations.
  • Random initialization: Random initialization makes the empirical average of orthogonal components close to zero, so the corresponding minimizer projection stays near its initialized value.This conclusion invokes the law of large numbers for the initialization noise.
  • Random initialization: The target-span components are optimized, while orthogonal components remain unchanged and can preserve initialization noise.The proof describes the orthogonal projection as remaining zero during optimization in the corresponding decomposition.

B.1. Empirical verification

Experiments on CIFAR-10 compare DM synthesis under random and real-data initialization with linear and ReLU-activated extractors. Random initialization produces noise-like, class-dependent images, whereas real-data initialization preserves initialization content and ReLU improves image quality.

  • Initialization: CIFAR-10 DM synthesis with a linear extractor of embedding dimension 2048 produces random-initialization images combining noise with class-dependent background.Each plotted line contains images from the same class.
  • Initialization: Real-data initialization changes the initialization images little, consistent with synthetic samples retaining the initialization images plus a barycenter-distance vector.This empirically supports the corresponding real-data initialization analysis.
  • Activation function: ReLU-activated extractors produce better image quality than linear extractors for both random and real-data initialization.The paper attributes this potentially to altered optimization and different local minima.
  • Experimental comparison: Figure 6 compares synthetic images across random-noise and real-data initialization and across linear and ReLU-activated extractors.The figure is presented as an empirical verification of the theoretical conclusions.

C. Proof of Proposition 4.10

The proof quantifies membership privacy by comparing parameter distributions produced from condensation sets that differ by one raw sample. Under shared initialization, the resulting KL-divergence bound is controlled by the limited influence of that sample.

  • Membership leakage is quantified by the KL divergence between parameter distributions trained on synthetic datasets generated with and without one raw sample.The construction compares T with T′ = T \ {x_|T|}, and their corresponding synthetic datasets S and S′.
  • The analysis studies how removing one raw sample changes the distributions p(θ) = P(θ|S) and q(θ) = P(θ|S′).
  • Shared initialization lets the proof apply the same per-synthetic-sample comparison to S and S′.
  • The resulting privacy bound follows by combining the preceding estimates with the local approximation exp(x) − 1 = O(x).The proof also notes that K_S decreases as |S| increases because each additional synthetic sample contributes a factor exp(−l(θ, s)) ≤ 1.
  • For a 2-layer ReLU extractor, the analysis connects the theoretical comparison to Proposition 4.3 and empirical barycenter experiments.

D.1. Analysis for 2-layer Network as Extractor

The analysis extends barycenter matching to a 2-layer network with ReLU activation by rewriting the gradient through Gaussian and Bernoulli representations. Experiments on CIFAR-10 show decreasing barycenter distance, with near-zero distance for the 2-layer network.

  • ReLU extractor analysis: ReLU outputs are represented using a Bernoulli variable multiplied by the absolute value of a Gaussian variable.For Y = max(0, X) with X Gaussian, the analysis uses Y distributed as B|X| and computes its expectation through the Bernoulli factor.
  • ReLU extractor analysis: The derivation simplifies to the scalar case k = 1 and rewrites absolute values using the sign function before expressing the DM gradient.
  • Pseudo-barycenter interpretation: The matrix M(x, y) depends on the angle between x and y, while M(x, x) = I_d and M(x, −x) = −I_d.
  • Pseudo-barycenter interpretation: Each synthetic sample is updated toward reducing the distance between the pseudo-barycenters of the raw and synthetic datasets.
  • Empirical verification: The experiments verify Proposition 4.3 on CIFAR-10 while varying r_ipc and initialization conditions.The experimental setup uses PyTorch 1.10 on Ubuntu 20.04, and the DC settings specify learning rates for the compared methods.
  • Empirical verification: The barycenter distance decreases over DM iterations and reaches a minimum; for the 2-layer ReLU network, it remains around 0.For ConvNet, the distance decreases more slowly, which the passage attributes as a suspicion to convolutional optimization reaching a local minimum.

E.2. Loss distribution of data used for DC initialization and test data on fS

The experiments examine privacy and utility of synthesized data through loss distributions, visualizations, membership attacks, and accuracy comparisons. DC methods show distinct initialization-related leakage patterns, while DM and DSA retain similar distributions and strong utility under the reported setting.

  • Loss distribution: Initialization members have smaller f_S losses than non-initialization data, making their membership easier to infer.The loss-distribution difference is reported to explain the high advantage scores in Table 1.
  • Visual privacy and utility: DM- and DSA-synthesized distributions are more similar to each other than KIP- and GAN-synthesized distributions on CIFAR-10 and CelebA.The visualization clips DC-synthesized data to [0, 1] for comparison with GAN-synthesized data.
  • Visual privacy and utility: DM- and DSA-synthesized data enable higher accuracy than KIP and GAN under the same r_ipc, according to the reported distribution comparison.
  • Membership attacks: The GAN-leak evaluation uses a white-box threat model in which the adversary has access to the generators.The attack varies z to optimize L_cal and estimate whether a sample belongs to the training dataset.
  • Accuracy comparison: KIP significantly outperforms the baselines and achieves performance similar to DSA and DM on CIFAR-10 at r_ipc = 0.002.The passage also reports that ZCA preprocessing improves the utility of KIP-synthesized data.
Loading 2206.00240v1…