PaperlayerView exact revision on arXiv

Source-linked AI summary

A Probabilistic Fluctuation based Membership Inference Attack for Diffusion Models

Wenjie Fu, Huandong Wang, Liyuan Zhang, Chen Gao, Yong Li, Tao Jiang

arXiv:2308.12143v5cs.LGcs.AIcs.CRcs.CV

TL;DR

Existing generative-model MIAs often depend on overfitting, which can be reduced by regularization, whereas memorization remains a more persistent membership signal. PFAMI estimates local probabilistic fluctuations around records to infer membership, and experiments report higher attack performance across generative models and datasets, including about 27.9% ASR improvement over the best baseline.

  • Problem

    Existing generative-model MIAs mainly rely on overfitting, but regularization can avoid overfitting while leaving memorization as a more prevalent phenomenon.

    Source: §I · p. 2

  • Method

    PFAMI uses probability estimation, dynamic perturbations, and metric-based or neural-network-based inference to analyze local probabilistic fluctuations.

    Source: §I · p. 2

  • Results

    PFAMI achieves higher ASR and stability than existing MIAs across multiple generative models and datasets, with 19.8% improvement on diffusion models and 35.9% on VAEs.

    Source: §I · p. 2

  • Takeaways & Limitations

    Probabilistic fluctuations provide a membership signal for generative models whose memorization is not captured reliably by probability-magnitude attacks.

    Source: §I · p. 2, §VII · p. 11

Abstract

from arXiv · show

Membership Inference Attack (MIA) identifies whether a record exists in a machine learning model's training set by querying the model. MIAs on the classic classification models have been well-studied, and recent works have started to explore how to transplant MIA onto generative models. Our investigation indicates that existing MIAs designed for generative models mainly depend on the overfitting in target models. However, overfitting can be avoided by employing various regularization techniques, whereas existing MIAs demonstrate poor performance in practice. Unlike overfitting, memorization is essential for deep learning models to attain optimal performance, making it a more prevalent phenomenon. Memorization in generative models leads to an increasing trend in the probability distribution of generating records around the member record. Therefore, we propose a Probabilistic Fluctuation Assessing Membership Inference Attack (PFAMI), a black-box MIA that infers memberships by detecting these trends via analyzing the overall probabilistic fluctuations around given records. We conduct extensive experiments across multiple generative models and datasets, which demonstrate PFAMI can improve the attack success rate (ASR) by about 27.9% when compared with the best baseline.

I. INTRODUCTION

Existing generative-model MIAs often rely on overfitting, while PFAMI targets the more persistent memorization signal by analyzing local probabilistic fluctuations. The framework combines probability estimation, dynamic neighbor perturbation, and inference functions, and improves attack performance across models and datasets.

I. INTRODUCTION

  • PFAMI addresses the limitation that probability-magnitude attacks can fail when regularization removes overfitting but memorization remains.Memorization produces increased probability density around member records, motivating local fluctuation analysis.Source: Figure 1 · p. 1, §I · p. 2
  • PFAMI estimates probabilities through variational inference and samples representative neighbors using dynamically adjusted perturbations.These components characterize probabilistic fluctuations across local directions and distances.Source: §I · p. 2
  • PFAMI uses both statistical metric-based and neural-network-based inference functions to distinguish members from non-members.The functions aggregate fluctuation features extracted from neighboring records.Source: §I · p. 2
  • 35.9% improvement in ASR on VAEs and 19.8% on diffusion models are reported over existing MIAs.Experiments evaluate PFAMI across multiple generative models and datasets.Source: §I · p. 2

A. Generative Models

  • VAEs and diffusion models are evaluated because they are prominent probabilistic generative-model families with analyzable learned probabilities.The paper positions both model families as targets for vulnerability evaluation.Source: A. Generative Models · p. 2

B. Membership Inference Attack

Prior generative-model MIAs use access-dependent proxies such as distances, discriminator confidence, or estimation error, but are limited on models exhibiting memorization without overfitting. The paper frames PFAMI around probabilistic generative models, including VAEs and diffusion models, whose probability structure can be analyzed directly.

B. Membership Inference Attack

  • Existing generative-model MIAs use white-box optimization, large synthetic datasets, discriminator confidence, or estimation error as membership signals.The taxonomy distinguishes methods by access requirements and feasibility across generative models.Source: B. Membership Inference Attack · p. 3, Table I · p. 3
  • Existing methods can fail on models exhibiting memorization without overfitting, whereas PFAMI detects distinctive memorization characteristics with reduced access requirements.PFAMI is motivated by probabilistic fluctuations rather than probability magnitude alone.Source: B. Membership Inference Attack · p. 3

A. Generative Models

  • PFAMI focuses on diffusion models and VAEs because both are probabilistic deep generative models amenable to direct probability analysis.VAEs use probabilistic encoders and decoders, while diffusion models learn forward and reverse processes.Source: A. Generative Models · p. 3
  • VAEs encode inputs into a Gaussian approximate posterior and decode latent codes through a probabilistic distribution.The decoder distribution is Gaussian for image generation and depends on the latent code.Source: A. Generative Models · p. 3
  • Diffusion models combine a fixed forward diffusion process with a learned reverse denoising process over T steps.The forward process supports sampling at arbitrary time steps, while the reverse probability is tractable when conditioned on x_0.Source: A. Generative Models · p. 3

B. Threat Model

The threat model asks whether a queried record belongs to the target model’s training set under gray-box access. PFAMI then assesses probabilistic fluctuations through three modules designed around probability estimation, neighbor generation, and membership inference.

B. Threat Model

  • The adversary predicts whether a record x^(i) belongs to the target training subset D_mem rather than D_non.The target model is trained on D_mem, while the adversary does not know its membership assignments.Source: B. Threat Model · p. 4
  • The study adopts a gray-box setting where users can access intermediate generative results but not the target model’s parameters.Examples include latent codes in VAEs and unfinished denoised images in diffusion models.Source: B. Threat Model · p. 4

IV. METHODOLOGY

  • PFAMI assesses probabilistic fluctuations and introduces three modules to address probability estimation, neighbor construction, and inference-function design.The framework is presented as a response to the three practical challenges identified for fluctuation-based attacks.Source: §IV · p. 4

A. Framework

PFAMI shifts membership inference from absolute generation probabilities, which rely on overfitting, to probabilistic fluctuations around target records associated with memorization. Its practical framework combines efficient probability assessment, local neighbor sampling, and inference functions.

A. Framework

  • Existing probability-threshold MIAs rely on overfitting, but regularization and early stopping reduce overfitting and undermine this attack paradigm.The framework instead motivates exploiting memorization, which persists earlier in training and can support generalization.Source: A. Framework · p. 4
  • PFAMI detects local probability maxima by analyzing fluctuations between a target record and representative neighboring records.The fluctuation is summarized by an inference function over the neighbors.Source: A. Framework · p. 5
  • Memorization motivates the attack because member records tend to have higher generation probabilities than neighboring records, often forming local maxima.This distinction makes memorization a target signal even when overfitting is controlled.Source: A. Framework · pp. 4–5
  • The framework introduces three practical modules: variational probability assessment, dynamic perturbation-based neighbor sampling, and inference functions for identifying members.The framework overview presents these modules as the practical deployment of PFAMI.Source: A. Framework · p. 5, Figure 2 · p. 5

B. Variational Probability Assessment

The variational probability assessment estimates a record's relative generation probability from ELBO components using only a few black-box or gray-box queries. Diffusion models use sampled estimation errors across time steps, while VAEs use reconstruction and KL terms.

B. Variational Probability Assessment

  • Existing distance-based probability estimates require tens of thousands of generated records, limiting attack efficiency.PFAMI seeks an alternative that avoids massive synthetic datasets.Source: B. Variational Probability Assessment · p. 5
  • For VAEs, the ELBO decomposes into expected reconstruction likelihood and KL divergence, both approximable through model queries.The reconstruction mean is obtained from decoder outputs, while the Gaussian KL term has a closed-form calculation.Source: B. Variational Probability Assessment · pp. 5–6
  • For diffusion models, the ELBO supplies an estimation-error term that can be evaluated across sampled time steps and fused to estimate relative pθ(x0).The attacker uses the fixed forward process, predicted means, and Monte Carlo sampling.Source: B. Variational Probability Assessment · p. 5
  • Combining the variational terms allows the attacker to approximate relative pθ(x) with only a few target-model queries and without internal parameters.The method applies the derived estimates to the ELBO-based probability approximation.Source: B. Variational Probability Assessment · p. 6

C. Neighbor Records Sampling

PFAMI samples local neighbors through controlled data perturbations, varying perturbation strength to characterize probability changes around a target record. Crop is selected as the default direction for its performance and stability.

C. Neighbor Records Sampling

  • Neighbor sampling is critical because the attack must explore a local, representative scope without entering meaningless regions of high-dimensional data space.The method therefore favors small, observable distribution shifts.Source: C. Neighbor Records Sampling · p. 6
  • PFAMI uses perturbations such as crop, rotation, downsampling, and brightening to generate neighboring records.These perturbations are adapted from data-augmentation techniques but are used here to probe local probability fluctuations.Source: C. Neighbor Records Sampling · p. 6
  • Crop is the default perturbation direction because experiments found it combines excellent performance and stability.The method also varies perturbation strength to sample neighbors at different distances.Source: C. Neighbor Records Sampling · p. 6
  • A dynamic perturbation mechanism applies increasing strengths {λ_j} across neighbors to characterize fluctuations around the target record.This mechanism is defined for arbitrary perturbation methods.Source: C. Neighbor Records Sampling · p. 6

D. Probabilistic Fluctuation based Inference Function

PFAMI estimates local probabilistic fluctuations around a target record and uses either statistical averaging or a neural network to infer membership. Its NNs-based variant represents fluctuation measurements as an image and trains with shadow-model data rather than extensive target-model labels.

D. Probabilistic Fluctuation based Inference Function

1) Metric-based Inference Function:

  • Metric-based PFAMIMet averages fluctuation estimates across M neighbors sampled with increasing perturbation strengths.Approximate probabilities are repeatedly queried for VAEs and estimated across sampled diffusion time steps.Source: 1) Metric-based Inference Function · p. 6

2) NNs-based Inference Function:

  • NN-based PFAMINNs arranges M × N fluctuation measurements as an image and uses a CNN classifier to capture variation patterns.The model outputs the probability that the target record is a member.Source: 2) NNs-based Inference Function · p. 6
  • PFAMINNs trains its attack model with auxiliary shadow-model data instead of requiring extensive member and non-member labels from the target model.This design reduces the method’s dependence on ground-truth target-model labels.Source: 2) NNs-based Inference Function · p. 7
  • The total query cost is (M + 1) × N, so selecting M and N balances attack performance against query-related risk.The paper evaluates PFAMI under different query numbers.Source: 2) NNs-based Inference Function · p. 7

V. EXPERIMENTS

The experiments evaluate PFAMI on image datasets and multiple generative models under regularized training, comparing it with white-, gray-, and black-box MIA baselines. PFAMI achieves the strongest reported attack performance across the main model–dataset evaluations.

1) Datasets and Target models:

  • The study evaluates DDPM and vanilla VAE on Celeba-64 and Tiny-ImageNet, plus additional diffusion and VAE variants.Target models use regularization mechanisms, including AdamW and early stopping, to avoid overfitting.Source: 1) Datasets and Target models · p. 7

2) Implementation Details:

  • The implementation uses diffusers for diffusion models and pythae for VAEs, with calibrated approximate probabilities and perturbation-based PFAMI configurations.PFAMIMet uses N = 10 and M = 1, while PFAMINNs uses M = 10 and a ResNet attack model trained on 2,000 shadow-model samples.Source: 2) Implementation Details · p. 7

3) Baselines:

  • PFAMI is compared with six generative-model MIAs across white-box, black-box, and gray-box access settings.SecMI provides the gray-box comparison sharing PFAMI’s access setting.Source: 3) Baselines · p. 7

B. Attack Performance

  • The main comparison reports AUC and ASR for PFAMI and baselines across two generative models and two datasets, alongside ROC curves.The table also records each attack’s query count and marks diffusion-specific baselines unavailable on VAEs.Source: B. Attack Performance · pp. 7–8, Table IV · p. 8
  • PFAMINNs and PFAMIMet achieve the highest average ASRs of 90.0% and 86.1%, respectively, across the main generative-model and dataset evaluations.PFAMI also improves AUC by approximately 35.9% over the strongest VAE baseline.Source: B. Attack Performance · p. 8, Table IV · p. 8

C. Generalizability Study

PFAMI generalizes across generative-model variants and Stable Diffusion, remains stable across memorization degrees, and distinguishes members through fluctuation patterns rather than approximate probability alone.

C. Generalizability Study

  • PFAMI maintains notable inference performance on Stable Diffusion after fine-tuning a pretrained model on the Pokemon dataset.The generalizability study also evaluates diffusion and VAE variants trained on Celeba-64.Source: C. Generalizability Study · p. 8, Table VI · p. 9

D. Impact of Memorization Degree on Attack Performance

E. How the Probabilistic Fluctuation Works

  • Approximate probabilities for members and non-members can nearly overlap in non-overfitting diffusion models, limiting probability-magnitude inference.Existing probability-based attacks therefore lack reliable discrimination in this setting.Source: E. How the Probabilistic Fluctuation… · p. 9, Figure 6 · p. 9
  • Members tend to be probability-distribution peaks with lower-probability neighbors, producing positive fluctuation, while non-members near inflection points remain near zero.This separation makes threshold-based membership discrimination easier.Source: E. How the Probabilistic Fluctuation… · pp. 9–10
  • Regular fluctuation patterns across diffusion time steps and perturbation strengths help PFAMINNs extract member–non-member differences.Approximate probability alone leaves the records indistinguishable on sampled diffusion time steps.Source: E. How the Probabilistic Fluctuation… · p. 10, Figure 7 · p. 10

F. Ablation Study

The ablation shows that all PFAMI modules contribute to attack performance, while the probabilistic fluctuation is interpreted as an estimate of local second-order behavior around a record.

F. Ablation Study

  • All PFAMI modules improve performance, and their combination achieves the highest ASR for PFAMINNs.Removing probability assessment, neighbor sampling, or the NNs-based inference function reduces performance.Source: F. Ablation Study · p. 10
  • Approximate probability assessment is foundational because removing it makes probabilistic fluctuations difficult to estimate accurately.Without this module, attack effectiveness becomes poor.Source: F. Ablation Study · p. 10

G. Interpretation of Probabilistic Fluctuation as Directional Second Derivative

  • PFAMI treats member records as approximate local maxima of the probability distribution, where directional second derivatives tend to be negative.The method uses expected second-order directional derivatives rather than requiring every derivative to be strictly negative.Source: G. Interpretation of Probabilistic… · p. 10
  • Symmetric perturbations x ± hz approximate second-order behavior around x, with z acting as a perturbation shift and h approaching zero.The symmetric form motivates treating perturbed records as neighboring records around the target.Source: G. Interpretation of Probabilistic… · p. 10
  • A perturbation distribution q(·∣x) mildly shifts the original record while preserving key features, and Monte Carlo sampling estimates the overall probabilistic fluctuation.This sampled quantity corresponds to the fluctuation measure used by PFAMI.Source: G. Interpretation of Probabilistic… · p. 10

H. How to Choose the Perturbation Mechanism

Crop is selected as PFAMI’s default perturbation because it combines performance and stability, while query studies show stable NN-based attacks and efficient metric-based attacks near ten samples. The paper also acknowledges privacy risks and reports that its experiments use public datasets.

H. How to Choose the Perturbation Mechanism

I. Impact of the Query Times

  • NN-based attacks remain exceptionally stable as query numbers vary, whereas metric-based attacks balance performance and efficiency with around 10 sampled points.The query analysis varies both neighbor records M and sampled points N.Source: §I · p. 11, Figure 8 · p. 11

VI. ETHICAL STATEMENT AND BROADER IMPACTS

  • PFAMI can create considerable privacy risk because it can infer whether an individual’s data was used to train a probabilistic generative model.The authors frame this risk as a motivation for studying defenses and privacy implications.Source: §VI · p. 11

VII. CONCLUSION

  • Across diffusion models and VAEs on different datasets, PFAMI maintains higher attack success rates and robustness than the evaluated baselines.The authors identify extending the framework to GANs as future work.Source: §VII · p. 11
Loading 2308.12143v5…