Source-linked AI summary
The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks
Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxiao Wang, Bo Li, Dawn Song
TL;DR
Model-inversion attacks had not convincingly inverted DNNs, raising privacy concerns for sensitive training data. The paper introduces GMI, which learns a prior from generic public data with GANs and uses it to guide inversion. Experiments report state-of-the-art DNN attack success, including about 75% higher face identification accuracy than existing work, while canonical differential privacy offers little protection.
Problem
Prior effective model-inversion attacks were demonstrated mainly on simple models, leaving convincing DNN reconstruction unresolved despite privacy risks from sensitive training data.
Method
GMI trains GAN-based generators and discriminators on public data, then optimizes a latent vector under the target network while constraining outputs to the learned data manifold.
Results
About 75% higher identification accuracy was achieved for reconstructing face images from a state-of-the-art face recognition classifier than existing work.
Takeaways & Limitations
Theoretical analysis connects model predictive power with inversion vulnerability, while experiments show GMI remains effective with generic, unlabeled, small, or distribution-shifted public data.
Takeaways & Limitations
The threat model focuses on white-box access to the target network, and experiments use public data without identity overlap with private data.
Abstract
from arXiv · showhide
This paper studies model-inversion attacks, in which the access to a model is abused to infer information about the training data. Since its first introduction, such attacks have raised serious concerns given that training data usually contain privacy-sensitive information. Thus far, successful model-inversion attacks have only been demonstrated on simple models, such as linear regression and logistic regression. Previous attempts to invert neural networks, even the ones with simple architectures, have failed to produce convincing results. We present a novel attack method, termed the generative model-inversion attack, which can invert deep neural networks with high success rates. Rather than reconstructing private training data from scratch, we leverage partial public information, which can be very generic, to learn a distributional prior via generative adversarial networks (GANs) and use it to guide the inversion process. Moreover, we theoretically prove that a model's predictive power and its vulnerability to inversion attacks are indeed two sides of the same coin---highly predictive models are able to establish a strong correlation between features and labels, which coincides exactly with what an adversary exploits to mount the attacks. Our extensive experiments demonstrate that the proposed attack improves identification accuracy over the existing work by about 75\% for reconstructing face images from a state-of-the-art face recognition classifier. We also show that differential privacy, in its canonical form, is of little avail to defend against our attacks.
1. Introduction
Model-inversion attacks seek private training features, but prior success was limited to simple models because DNN inversion is nonconvex, ill-posed, and high-dimensional. The paper proposes GMI, which uses generic public data to guide realistic DNN reconstruction and links inversion vulnerability to predictive power.
- Motivation: Model-inversion attacks reconstruct sensitive training features by exploiting their correlation with model outputs.They formulate inference as finding sensitive values with maximum likelihood under the target model.
- Challenge: Prior effective attacks targeted simple models, leaving DNN inversion an open question.DNN attacks face intractable, ill-posed optimization problems.
- Challenge: Nonconvex optimization can trap gradient descent in local minima, while unconstrained high-dimensional search can produce unrealistic, semantically meaningless features.These problems are especially relevant when recovering face images from recognition models.
- Approach: GMI uses generic public image information to learn a distributional prior that regularizes inversion and synthesizes private training data with high fidelity.The approach can use facial images collected from the Internet even when they exclude target individuals.
- Contributions: The paper theoretically connects high predictive power with inversion vulnerability because feature-label correlations are what attackers exploit.It also reports a GAN-based attack, extensive experiments, and limited protection from canonical differential privacy.
2. Related Work
Related work distinguishes membership inference from model inversion, reviews early attacks and theoretical analyses, and contrasts GMI with feature visualization. Differential privacy formally limits membership inference but does not explicitly target attribute privacy.
- Privacy attacks: Membership attacks determine whether an individual’s data entered training, whereas model-inversion attacks reconstruct features associated with target labels.Both belong to broader privacy attacks against machine-learning models.
- Defenses: Differential privacy limits membership-attack success by reducing dependence on individual records, but it does not explicitly protect attributes targeted by model inversion.This distinction motivates separate consideration of attribute privacy.
- Prior inversion attacks: Early model-inversion evaluations focused on linear models despite broader algorithmic frameworks.The first attack recovered genetic markers from a linear regression model, and later work addressed unknown nonsensitive features through data poisoning.
- Theory: Theoretical studies characterize invertibility, individual risk, and overfitting, but their assumptions do not fully cover the paper’s setting.The supplied passage indicates that at least one theory assumes a restricted adversary setting.
- Relation to feature visualization: GMI differs from feature visualization by using a novel optimization objective and auxiliary attacker knowledge to produce more realistic image recovery.Feature visualization also seeks images that maximally activate a target network.
3. Generative MI Attack
GMI attacks DNNs by training generative models on public data and optimizing latent vectors against the target network while preserving image realism. The method supports auxiliary information and combines realism, identity, and diversity objectives.
- Threat model: The threat model assumes white-box access to a target network and seeks the feature x associated with a chosen label y.Face recognition classifiers provide the running example, with the goal of recovering an identity-specific face image.
- Public knowledge distillation: The generator and discriminator are trained on public data to produce realistic images, using Wasserstein-GAN training as the first-stage foundation.The generator can also accept blurred or corrupted auxiliary images as input.
- Pipeline: GMI has two stages: public knowledge distillation trains generative components, followed by secret revelation that optimizes a latent representation.Public data may be unlabeled and need not share identities with private data.
- Auxiliary knowledge: With auxiliary corrupted images, separate global and local discriminators enforce whole-image coherence and consistency near mask boundaries.The authors report that separate discriminator outputs make the training loss converge faster empirically.
- Diversity objective: A diversity loss encourages generated images to differ in the target network’s feature space, helping the network identify the image most likely to occur in private training data.The full distillation objective subtracts the diversity term from the Wasserstein-GAN loss with weight λ_d.
- Secret revelation: Secret revelation minimizes prior and identity losses, penalizing unrealistic images while encouraging high target-network likelihood.The target-network probability C(G(z)) defines the identity-loss likelihood term.
4. Connection Between Model Predictive Power and MI Attacks
The paper formalizes predictive power through log-likelihood loss when sensitive features are removed and relates it to model-inversion vulnerability. Its theorem and intuition state that more predictive models yield better attacks under the paper's distribution-recovery formulation.
- Predictive-power measure: Predictive power is measured by the change in log likelihood when the sensitive feature is excluded from the model input.The analysis uses log likelihood because maximizing it is equivalent to minimizing cross-entropy loss.
- MI formulation: The adversary seeks to recover the sensitive-feature posterior given the label, model, and auxiliary non-sensitive features.The best achievable modeled posterior is proportional to the model likelihood multiplied by a prior over sensitive features.
- Theoretical connection: Theorem 1 states that, for fixed labels and non-sensitive features, higher conditional predictive power produces greater symmetric-KL divergence from the true sensitive-feature posterior.The theorem compares models f1 and f2 through their conditional predictive-power measure and attack posteriors.
- Theoretical connection: Highly predictive models establish stronger feature-label correlations, which the adversary exploits, so greater predictive power leads to higher attack performance.This connection is stated for the paper's general model-inversion formulation rather than the alternative setup discussed in prior work.
- Relation to prior work: The paper contrasts its formulation with prior work that assumes adversarial access to the joint distribution and targets a feature from a specific training dataset.Here, the adversary learns the private feature distribution from model parameters without assuming that joint-distribution knowledge.
5. Experiments
The experiments evaluate the attack across three image datasets and varied target networks, using disjoint public data to learn generic priors. They introduce quantitative metrics that assess pixel-level similarity, semantic identity exposure, and feature-space proximity.
- Datasets: Experiments use MNIST, ChestX-ray8, and CelebA, with CelebA containing 202,599 face images from 10,177 identities resized to 64×64.The datasets cover handwritten digits, chest X-rays, and face images.
- Protocol: Each dataset is split into private data for target-network training and public data for prior knowledge distillation.The public data have no class intersection with the private training data, so they provide generic rather than class-specific information.
- Models: The target models vary by task and complexity, including a three-convolutional-layer MNIST network, ResNet-18 for ChestX-ray8, and adapted face-recognition architectures.The excerpt specifies task-specific architectures and adaptation of final outputs.
- Training: Training uses SGD for target networks and a two-stage GAN-based attack pipeline with Adam in the first stage and latent-vector optimization in the second.The reported settings include learning rate 10^-2 for target training, λd = 0.5, λi = 100, and batch size 64 where specified.
- Evaluation metrics: Four metrics enable large-scale quantitative evaluation of model-inversion efficacy rather than relying mainly on visual inspection.The metrics include PSNR, Attack Accuracy, Feature Distance, and K-Nearest Neighbor Distance.
- Evaluation metrics: PSNR measures pixel-wise similarity through maximum squared pixel fluctuation relative to mean squared error, with higher values indicating better reconstruction quality.It evaluates closeness between the target and reconstructed images at the pixel level.
- Evaluation metrics: Semantic metrics are needed because identity information can remain exposed despite translation, scale, or rotation differences that reduce pixel-wise similarity.The evaluation therefore supplements PSNR with identity- and feature-based measures.
- Evaluation metrics: Attack Accuracy uses a separate evaluation classifier to determine whether reconstructed images expose the target identity.The classifier is distinct from the target network to avoid rewarding features that merely overfit the target model.
5.3. Experimental Results
Experiments evaluate GMI against EMI and PII across face-recognition settings, auxiliary knowledge conditions, public-data regimes, predictive powers, privacy budgets, and datasets. GMI consistently reconstructs more informative private data, with performance affected by public-data distribution, model predictive power, and differential privacy.
- Face recognition classifiers: GMI improves Top-5 attack accuracy against the face.evoLVe classifier over EMI by 75%.EMI achieves around zero attack accuracy on deep networks, often producing recognizable-free patterns that nevertheless match the target label.
- Auxiliary knowledge: PII produces realistic face recoveries but does not reproduce the target identity features, whereas GMI reveals private information from the target network.This comparison distinguishes inversion of private information from generating realistic-looking images using public data alone.
- Auxiliary knowledge: GMI outperforms EMI and PII when blurred or corrupted private images are available as auxiliary knowledge.The center mask is easier to attack than the face T mask because it may leave mouth information exposed, whereas the face T mask hides identity-revealing features.
- Public knowledge: Attack performance varies by less than 7% when same-distribution public data shrinks tenfold.The evaluated public-to-private identity ratios are 1:1, 1:4, 1:6, and 1:10.
- Public knowledge: Using PubFig83 public data for CelebA attacks drops accuracy by more than 20% without preprocessing, while landmark alignment and cropping improve accuracy significantly.The distribution shift includes more sunglasses in PubFig83 than in CelebA, making generic face information harder to distill.
- Predictive power and differential privacy: GMI performance increases with models’ feature predictive power across training-size, dropout, and batch-normalization variations, consistently across architectures.Figure 3 also evaluates attack accuracy under different differential-privacy budgets; on MNIST, GMI remains above the PII baseline even at ϵ = 0.1.
- Other datasets: GMI significantly outperforms EMI on MNIST and by a large margin on ChestX-ray8.The MNIST experiment uses disjoint private and public label sets, while ChestX-ray8 uses private and public images from different labels.
6. Conclusion
The paper presents a generative approach that uses GAN-learned knowledge from public datasets to regularize model inversion against DNNs with high-dimensional inputs. Experiments and theory support strong attack performance across varied public-data conditions and connect model predictive power with inversion vulnerability.
- The generative model-inversion approach uses GANs to extract generic knowledge from public datasets and regularize inversion of DNNs with high-dimensional inputs.
- The attack remains highly performant when public data excludes target identities, is unlabeled, is small, or comes from a different distribution.
- Theoretical analysis identifies a fundamental connection between a model’s predictive power and its vulnerability to inversion attacks.
A. Proof of Theorem 1
The theorem compares two models through their predictive quantities and shows that stronger predictive performance corresponds to a larger KL divergence in the stated inversion setting.
- Theorem 2 states that if Uf1(xns, y) ≥ Uf2(xns, y) for every fixed label y, then f1 has at least as large an inversion-related KL divergence.The comparison is between DKL(p(Xs|y, xns)||pf1(Xs|y, xns)) and the corresponding divergence for f2.
- The proof begins by expanding the KL divergence DKL(p(Xs|y, xns)||pf1(Xs|y, xns)).
- It then considers the difference between the divergences associated with f1 and f2.
- The displayed derivation includes the term log pf2(y|Xs, xns)p(Xs|xns).
B.1. Network Architecture
The experiments use GAN-based architectures tailored to three auxiliary-knowledge settings, with the same GAN designs also serving the PII baseline and GMI attacks. Additional architecture specifications cover the encoders, generator decoder, discriminators, and several target networks.
- The GMI attack uses separate GAN architectures without auxiliary knowledge, with a corrupted private image, and with a blurred private image.These architectures are shown in Figures 4, 5, and 6, respectively.
- The same GAN architectures are used for both the PII baseline and the GMI attacks.
- Detailed designs are provided for the two encoders, generator decoder, local discriminator, and global discriminator.These components are specified in Tables 6 through 10.
- LeNet has three convolutional layers, two max pooling layers, and one fully connected layer, while SimpleCNN has five convolutional layers with batch normalization and leaky ReLU.
- SoftmaxNet consists of a single fully connected layer.
B.2. The Detailed Setting of the Experiments on “Attacking Differentially Private Models”
The differential-privacy experiments use a private/public MNIST split, a multilayer perceptron target, and a convolutional evaluation classifier. The appendix specifies auxiliary-input encoders, GAN components, privacy training settings, optimization details, and attack-evaluation sampling.
- MNIST digits 0–4 form the private training set, while digits 5–9 form the public set used to distill prior knowledge.
- The target network is a multilayer perceptron with hidden layers of 512 and 256 neurons, and the evaluation classifier reaches 99.2% MNIST test accuracy.
- For corrupted-image auxiliary knowledge, the upper encoder receives the corrupted RGB image and binary mask; for blurred-image knowledge, it receives only an image.
- The generator also includes a latent-vector lower encoder, a decoder, a global discriminator, and a local discriminator used only for corrupted-image attacks.
- Differential privacy is implemented by adding Gaussian noise to every stochastic-gradient-descent step and tracking the privacy budget with moment accounting.Training uses batch size 256, 40 epochs, and per-sample gradient L2 clipping at 1.5.
- The MNIST GAN uses batch size 64 and Adam in stage one, while stage two uses batch size 64 and SGD.
- Attack accuracy is averaged over 640 randomly sampled private images and five random latent-vector initializations per image.