PaperlayerView exact revision on arXiv

Source-linked AI summary

Membership is Ownership: A Robust Ownership Verification Framework for Diffusion Models

Feng Jiang, Zuobin Xiong, An Huang, Zhipeng Cai, Yingshu Li

arXiv:2608.28929v1cs.CRcs.CV

TL;DR

Diffusion models are valuable IP but vulnerable to theft, while watermarking can reduce utility and weaken after fine-tuning. MiO uses a population-level hypothesis test over a private member evidence set, combining membership-based attribution with separation from public references. Experiments on DDIM and Stable Diffusion report strong verification performance, minimal utility impact, and stability under post-theft modifications.

  • Problem

    Existing diffusion-model ownership protections can impose utility costs and weaken under post-hoc fine-tuning, motivating verification with minimal model disruption.

  • Method

    MiO uses a private member evidence set as a non-invasive watermark and combines population-level membership attribution with statistical separation from reference models.

  • Results

    MiO outperforms existing methods in verification accuracy, costs, utility loss, and post-theft robustness across two evaluation scenarios.

  • Takeaways & Limitations

    MiO verifies ownership without modifying the owner model or sampling pipeline, while testing both attribution and reference separation at p<10^-6.

Abstract

from arXiv · show

Large-scale diffusion models have fueled numerous profitable downstream applications for AI-related businesses, including visual editing and content creation. Meanwhile, due to the huge amount of resource consumption (e.g., computation and high-quality data) during training, such diffusion models are deemed valuable intellectual property (IP) for tech companies like OpenAI and Google. Yet, the IP assets are vulnerable to various unauthorized uses by adversaries seeking to steal models for customized, usually commercial applications. Some existing approaches have explored IP protection for AI models; however, they mostly face structural limitations in common --- using a training-time watermarking by injecting artifacts in the model, which can impose a measurable utility cost and can be weakened by post-hoc fine-tuning. To address these challenges, this work investigates IP protection (i.e., model ownership verification) for diffusion models in a realistic commercial scenario with minimal model utility loss. Specifically, the proposed method builds a framework for model ownership verification, termed ``{Membership is Ownership} (MiO)'', based on a population-level hypothesis test on a private member evidence dataset. MiO verifies ownership using two criteria: model attribution through membership inference and model separation from public references. Both are tested at $p<10^{-6}$. We evaluate MiO on DDIM and Stable Diffusion models without modifying the owner model or its sampling pipeline, and report ROC-AUC and true-positive rates at fixed nominal false-positive targets. Furthermore, MiO stays stable under different post-theft fine-tuning and weight perturbation in adversarial scenarios, reflecting better robustness compared to the watermarking methods.

I. INTRODUCTION

Diffusion models are valuable but vulnerable IP, while existing watermarking and per-sample memorization methods have important limitations. MiO addresses these issues with population-level testing over private evidence while preserving utility and improving verification robustness.

  • Motivation: Diffusion models require substantial computation and high-quality data, making them valuable but vulnerable intellectual property.Adversaries may steal and redeploy models through fine-tuning without contributing to their development.
  • Limitations of Existing Methods: Training-time watermarking can impose measurable utility costs and weaken under post-hoc fine-tuning.These methods inject signatures into model parameters or output behavior for later identification.
  • Limitations of Existing Methods: Per-sample memorization verification cannot certify ownership because easy-to-reconstruct samples may produce misleading evidence.Its binary sample-level decision is insufficient for an ownership claim.
  • MiO: MiO reframes ownership verification as a population-level hypothesis test using a private member evidence set as a non-invasive watermark.The framework extracts robust memorization signals from the private set and uses Gaussian quantile regression to derive thresholds for selected false-positive targets.
  • MiO: MiO introduces minimal utility impact, reporting 0 change in FID score for original models.The private evidence dataset leaves model parameters and generation quality intact.
  • Evaluation: MiO is evaluated on naive and commercialized diffusion models across multiple datasets and configurations, with detection performance reported at fixed false-positive targets.The evaluation includes DDIM models trained from scratch and Stable Diffusion models fine-tuned from production-scale datasets.
  • Related Work: MiO differs from prior membership-inference work by aggregating evidence and comparing suspect models with calibrated references for statistically interpretable ownership decisions.The framework targets model ownership rather than data ownership and uses a closed-form threshold for a specified false-positive rate.

III. PROBLEM FORMULATION

This section distinguishes diffusion-model reconstruction and sampling from ownership verification. It formalizes the gap between single-sample membership inference and population-level testing against independent reference models.

  • 1) Diffusion Models.:: Diffusion models corrupt clean samples through forward noising and train a network to predict the added noise.The denoising objective samples timesteps uniformly from the diffusion schedule.
  • 1) Diffusion Models.:: DDIM sampling iteratively denoises from Gaussian noise and provides a closed-form clean-signal estimate used to construct reconstruction scores.The estimate is obtained from a noised sample and its timestep through the denoising network.
  • 1) Diffusion Models.:: Stable Diffusion performs the diffusion process in compressed latent space and conditions denoising on text embeddings through U-Net cross-attention.Its reconstruction error is therefore computed in latent space.
  • 1) Diffusion Models.:: Diffusion models memorize portions of their training data, creating the memorization gap exploited by MiO.This gap supports distinguishing training members from non-members.
  • 2) Membership Inference vs. Ownership Verification:: Membership inference is a binary classifier that predicts whether a query sample belongs to a target model’s training data.The classifier takes the model and query sample as inputs and returns a member or non-member decision.
  • 2) Membership Inference vs. Ownership Verification:: Ownership verification instead requires significant memorization on a designated evidence set relative to independent reference models.The claim is evaluated at the population level rather than through isolated sample predictions.
  • 2) Membership Inference vs. Ownership Verification:: The formal ownership protocol tests whether a suspect model derives from an owner model using private evidence and a controlled false-positive rate.Reference models provide the comparison set for the hypothesis test.
  • 2) Membership Inference vs. Ownership Verification:: Reference models must use data disjoint from the private evidence, have independent provenance, and support the same reconstruction-score procedure.These conditions make the comparison suitable for ownership verification.

B. Threat Model

The framework assumes a stolen owner model may be adapted through weight-level operations, while excluding independent retraining from scratch. Verification relies on confidential evidence data and combines reconstruction-based membership scoring with MiO’s ownership protocol.

  • Threat model: The adversary obtains the owner model through theft and may produce a derivative by fine-tuning.The verifier has white-box access to the owner, suspect, and public reference models.
  • Threat model: The threat model includes fine-tuning, weight perturbation, quantization, and pruning, but excludes independent retraining from scratch.Distillation is examined only as a boundary case.
  • Threat model: The owner’s confidential evidence set W is inaccessible to the adversary and is central to verification.The paper argues that selecting the exact subset W from leaked training data is computationally infeasible in the stated CIFAR-10 setting.
  • Threat model: MiO quantifies memorization with reconstruction errors, calibrates membership scores across sampled timesteps, and applies two ownership criteria.The reconstruction procedure corrupts samples, predicts noise, recovers x̂0, and aggregates errors into s(x).

B. Gaussian Quantile Regression for Adaptive Threshold

MiO calibrates adaptive membership thresholds by modeling log-transformed reconstruction scores conditionally on each sample. It stabilizes the score across timesteps and reduces predictor variance with an ensemble.

  • Score construction: MiO aggregates reconstruction errors over K uniformly sampled timesteps using percentiles to obtain a more stable score s(x).The 25th percentile is selected empirically, and K = 50 is used in experiments.
  • Conditional modeling: A learned predictor models log(1 + s(x)) with a conditional Gaussian whose parameters depend on x and summary statistics of its timestep-error sequence.The predictor outputs µ(x) and log σ(x), recovering σ(x) through exponentiation.
  • Conditional modeling: The Gaussian learner is trained on auxiliary non-member data disjoint from the owner’s evidence and training sets to calibrate the non-member score distribution.The auxiliary data may be a held-out split or public data from the same domain.
  • Adaptive thresholding: For target FPR α, MiO uses the modeled conditional α-quantile as an adaptive membership threshold.Members tend to produce lower scores, so likely membership is assigned in the lower tail.
  • Adaptive thresholding: A bagging ensemble of B = 50 predictors averages the estimated thresholds at test time to reduce predictor variance.Each predictor is trained on an 80% bootstrap resample of the auxiliary data.

C. Two-Point Verification Criteria

MiO declares ownership only when a suspect model shows memorization of the private evidence set and the owner separates from public references. The criteria use distinct hypothesis tests with stringent decision thresholds.

  • Criterion 1: Memorization detection: Criterion 1 tests whether the suspect model memorizes the private evidence set beyond the calibrated per-sample false-positive rate.The hit count over W is evaluated with an exact one-sided binomial test.
  • Decision rules: Criterion 1 passes when its p-value is below 10^-6.The criterion evaluates whether the suspect’s memorization exceeds the calibrated false-positive rate.
  • Criterion 2: Reference separation: Criterion 2 tests whether the owner reconstructs evidence samples better than each public reference model.It aggregates paired score differences into Cohen’s d and tests the corresponding null hypothesis.
  • Decision rules: Criterion 2 passes when its one-sided test has p < 10^-6 and |d|C2 > 2.The algorithm rejects ownership when the separation condition fails.
  • Decision rules: The protocol returns VERIFIED only if both criteria pass and REJECTED otherwise.The joint type-I error is at most 10^-6 without requiring independence between the tests.

A. Experimental Setup

The experiments cover controlled DDIM models and production-oriented Stable Diffusion fine-tuning scenarios. They vary datasets, private evidence sets, adversarial data, baselines, and public reference checkpoints.

  • Evaluation scenarios: The evaluation uses DDIMs trained from scratch and Stable Diffusion models fine-tuned on production-scale data.The two scenarios respectively provide controlled protocol measurements and a commercial threat surface.
  • Datasets: DDIM experiments span CIFAR-10, CIFAR-100, STL-10, and CelebA at multiple resolutions and dataset scales.Reserved evidence sets contain 5,000 images for CIFAR-10, CIFAR-100, and CelebA, and 1,000 for STL-10.
  • Datasets: Stable Diffusion uses 1,000 private COCO 2014 images, while adversaries fine-tune on a disjoint 1,000-image COCO subset plus synthetic images.This setup separates owner evidence from adversarial fine-tuning data.
  • Baselines: MiO is compared with WDM, Zhao et al., and CDI, with SleeperMark added for Stable Diffusion.The baselines include training-time watermarking, fingerprint embedding, inference-time MIA aggregation, and trigger-prompt coupling.
  • Reference models: Public HuggingFace checkpoints serve as references and are trained on data disjoint from the evidence sets.They represent the null hypothesis in the two-point verification protocol.

B. RQ1: How Reliable is MIA in Criteria 1 at Restricted FPR?

Criteria 1 uses calibrated per-sample membership inference decisions at restricted false-positive rates and aggregates them into a population-level test. Across DDIM and Stable Diffusion owners, the signal remains useful at FPR targets as low as 10^-4.

  • Membership-inference calibration: TPR is evaluated at fixed FPR targets α ∈ {10^-3, 10^-4}, alongside ROC-AUC, using a calibrated Gaussian quantile regression decision rule.The rule flags a sample when its log reconstruction error falls below the calibrated quantile.
  • Observed reliability: Every evaluated owner model achieves at least 0.879 AUC across four DDIM and three Stable Diffusion configurations.The target operating points still show substantial variation in TPR across owner models.
  • Observed reliability: The detection rate remains far above chance at restricted-FPR operating points, indicating a reliable t-error signal in the low-FPR regime.This supports the use of the per-sample signal for the subsequent aggregate test.
  • Population-level test: MiO retains usable true-positive rates at false-positive budgets as low as 10^-4, after which Criterion 1 aggregates decisions through a hit-rate binomial test.The corresponding null is HC1_0: r ≤ α, whose interpretation depends on the per-sample rule achieving the target FPR.

C. RQ2: Is Memorization Unique to the Owner?

Criterion 2 tests whether the owner model memorizes a private evidence set more strongly than public reference models. Across DDIM and Stable Diffusion configurations, the measured separations are significant and consistently support owner-specific memorization.

  • Reference separation: Criterion 2 compares owner-model reconstruction of private evidence samples against semantic-similar, cross-domain, and other public reference models.Semantic-similar references share data type or backbone but have not seen the owner’s private evidence set.
  • DDIM results: DDIM separations span |d|C2 from 16.3 to 87.3 across all 12 owner-reference cells, exceeding the Criterion 2 threshold of 2.0.The tested DDIM owners were trained from scratch, producing stronger memorization of W than the references.
  • Cross-family conclusion: All owner models are separated from their corresponding reference models across both DDIM and Stable Diffusion architecture families.The result reflects the uniqueness of the owner model’s memorization on the private evidence set W.

D. RQ3: Post-Theft Robustness

MiO’s ownership signal persists after several post-theft modifications, while a fresh model trained through distillation is rejected when its membership signal collapses. The framework also avoids owner-model modification and moves verification costs to offline auditor-side processing, though its comparison with baselines is constrained by incompatible native metrics.

  • Post-theft modifications: Across MMD and SGD fine-tuning, two weight-noise scales, and LoRA updates, the residual AUC remains at least 0.976.For the CIFAR-10 owner, the un-attacked control has TPR@10^-4 of 0.629, while the latent-space LoRA attack retains TPR@10^-4 of 0.614.
  • Distillation attack: The protocol rejects a distilled student whose per-sample MIA falls toward chance after training from 50,000 synthetic images sampled from the stolen model.Its hit rate on W stays at the α floor, so Criterion 1 fails to reject HC1_0 and returns REJECTED.
  • Baseline comparison: Baseline verification scores cannot be collapsed into one comparable number because methods use different statistical instruments, operating points, and sample-size regimes.The paper therefore reports each method under its native metric and compares protocols along cost axes instead.
  • Baseline comparison: MiO requires no owner-weight modification, adds no training-pipeline overhead, and performs verification offline without a deployment-time auxiliary component.Its ownership signal is supplied by a private dataset rather than embedded into the model.
  • Computational cost: MiO’s offline verifier costs approximately 100 V100-hours for one-time ensemble training and about 11 minutes to score and predict for a 5,000-sample evidence set.The ensemble training is parallelizable across B=50 models and reusable across queries.

F. Hyperparameters

The ablations select MiO hyperparameters that improve separation, stabilize quantile prediction, and preserve strong Stable Diffusion scoring performance.

  • t-error Aggregation Strategy: Q25 achieves the strongest separation with |d|C2 = 23.93, exceeding mean, median, and Q10 aggregation.The same pattern holds for latent-space diffusion, motivating adoption of Q25.
  • The Size of Ensemble B: B = 50 provides a favorable stability–cost trade-off for the Gaussian quantile-regression ensemble.Prediction variance drops sharply through B = 20, continues falling through B = 50, and plateaus thereafter.
  • The Number of Sampled Timesteps: K = 50 sampled timesteps is used because |d| plateaus once K ≥25 and changes negligibly beyond K = 50.The ablation varies K from 10 to 100.
  • Gaussian QR vs. Pinball-loss QR: Gaussian QR and pinball-loss QR yield comparable TPR at the reported extreme low-FPR operating points.Gaussian QR is preferred for flexibility because one fitted model returns multiple quantiles.
  • Latent vs. Pixel Space in Stable Diffusion: Latent-space scoring on Stable Diffusion reaches AUC = 0.9956 versus 0.959 for pixel-space scoring.The paper attributes the difference to VAE decoding diluting the membership signal in pixel space and uses latent space thereafter.
  • Framework Context: MiO’s hyperparameter study is conducted alongside a non-invasive ownership-verification framework that leaves the owner model and sampling pipeline unchanged.MiO uses the private member set as a non-invasive watermark based on natural memorization behavior.
Loading 2608.28929v1…