PaperlayerView exact revision on arXiv

Source-linked AI summary

Dataset Inference: Ownership Resolution in Machine Learning

Pratyush Maini, Mohammad Yaghini, Nicolas Papernot

arXiv:2104.10706v1stat.MLcs.CRcs.LG

TL;DR

Model stealing threatens valuable machine learning intellectual property, while watermarking-based defenses can require retraining and remain vulnerable to adaptive attacks. The paper introduces dataset inference, which tests whether a suspected copy contains knowledge from the victim’s private training data, and reports robust ownership claims across stealing settings with limited revealed data.

  • Problem

    Publicly exposed machine learning models can be stolen, while watermarking defenses may require retraining, reduce accuracy, and be weakened by adaptive attacks.

  • Method

    Dataset inference combines training-set knowledge with prediction-margin measurements, statistical testing, and white-box or black-box model-behavior analysis.

  • Results

    Dataset inference remains robust to strong and adaptive stealing techniques, claiming theft with at least 95% confidence for most threat models using only 10 samples.

  • Takeaways & Limitations

    Model developers can resolve ownership conflicts using knowledge of their training set without changing their existing models.

  • Takeaways & Limitations

    Ownership claims become less likely after an adversary fine-tunes a stolen copy on different data, although this concealment can cause accuracy degradation and catastrophic forgetting.

Abstract

from arXiv · show

With increasingly more data and computation involved in their training, machine learning models constitute valuable intellectual property. This has spurred interest in model stealing, which is made more practical by advances in learning with partial, little, or no supervision. Existing defenses focus on inserting unique watermarks in a model's decision surface, but this is insufficient: the watermarks are not sampled from the training distribution and thus are not always preserved during model stealing. In this paper, we make the key observation that knowledge contained in the stolen model's training set is what is common to all stolen copies. The adversary's goal, irrespective of the attack employed, is always to extract this knowledge or its by-products. This gives the original model's owner a strong advantage over the adversary: model owners have access to the original training data. We thus introduce $dataset$ $inference$, the process of identifying whether a suspected model copy has private knowledge from the original model's dataset, as a defense against model stealing. We develop an approach for dataset inference that combines statistical testing with the ability to estimate the distance of multiple data points to the decision boundary. Our experiments on CIFAR10, SVHN, CIFAR100 and ImageNet show that model owners can claim with confidence greater than 99% that their model (or dataset as a matter of fact) was stolen, despite only exposing 50 of the stolen model's training points. Dataset inference defends against state-of-the-art attacks even when the adversary is adaptive. Unlike prior work, it does not require retraining or overfitting the defended model.

1 INTRODUCTION

Machine learning models are valuable but exposed interfaces enable diverse model-stealing attacks, challenging ownership resolution. Dataset inference instead identifies stolen models through knowledge inherited from the victim’s private training data.

  • Motivation: Increasing model size and resource requirements make models valuable intellectual property, while public services expose them to extraction attacks.Adversaries can query prediction vectors or use models as labeling oracles to train copies.
  • Motivation: Watermarking defenses require retraining, can reduce accuracy, and may be weakened by adaptive extraction attacks.They verify responses on inputs outside the true training distribution.
  • Core idea: Dataset inference identifies stolen models by testing whether they contain direct or indirect knowledge from the victim’s training set.The approach exploits the owner’s access to private training data, which creates an information advantage over the adversary.
  • Core idea: Dataset inference distinguishes training-set points from other task-distribution points using prediction certainty, defined through a point’s margin to neighboring classes.Stolen models are expected to be more confident on points from the victim’s training set.
  • Approach: The framework combines targeted adversarial attacks or Blind Walk with confidence regression and hypothesis testing to produce an ownership claim.It supports white-box and black-box settings and is presented as a general ownership-resolution framework.
  • Broader relevance: Dataset inference also raises a possible use beyond ownership disputes: checking whether sensitive data was reused for purposes different from those stated at collection.The paper gives gender-classification models learning ethnicity-predictive features as an example motivating this concern.

2 RELATED WORK

Related work spans model extraction, watermarking, membership inference, and out-of-distribution detection. Dataset inference differs by identifying which dataset trained a model even when candidate datasets share the same distribution.

  • Model Extraction: Model extraction steals remotely deployed models using labels or logits and can be adapted after the defender’s strategy becomes known.Extraction may also support reconnaissance for later attacks such as finding adversarial examples.
  • Watermarking: Watermarking embeds information unrelated to the true data distribution and later uses it as a signature for ownership verification.The cited discussion notes that watermarking requires new training procedures and can decrease model accuracy.
  • Membership Inference: Membership inference predicts whether individual examples were used to train a model, using confidence-based distinctions between training and testing data.Prior work studies both white-box and black-box attacks and relates vulnerability to overfitting.
  • Out of Distribution Detection: Out-of-distribution detection asks whether inputs belong to a distribution, whereas dataset inference asks which of similarly distributed datasets supplied the training data.The distinction is that dataset inference separates training sources even when the dataset distribution is the same.

3 THREAT MODEL AND DEFINITION OF DATASET INFERENCE

The threat model considers a victim’s private dataset and an adversary’s model trained using a subset of the victim’s private knowledge. Dataset inference tests whether that model was trained on the victim’s data under a false-positive constraint.

  • Threat model: A victim trains classifier fV on private data SV, while an adversary may use a subset of the victim’s private knowledge to train fA*.The victim’s goal is to determine whether the adversary’s model is a copy of the victim’s model.
  • Victim definition: A dataset-inferring victim receives a classifier, significance level α, and sample budget m, then may reveal at most m private samples.It either proves that private knowledge was used with Type-I error below α or returns an inconclusive result.
  • Experiment: The dataset inference experiment compares classifiers trained on the victim’s private dataset with classifiers drawn from the broader data-distribution classifier set.The experiment parameterizes the comparison by m, α, the victim dataset SV, and distribution D.
  • Experiment: The experiment records success when the victim claims ownership and the sampled classifier came from the victim-trained set; otherwise it returns failure.The outcome is defined by the ownership decision and the hidden random label indicating the classifier source.

4 THEORETICAL MOTIVATION

Dataset inference distinguishes models trained on a victim’s data from models trained on comparable external data by aggregating differences in prediction margins. In the linear theoretical setting, this signal remains detectable for dataset inference even when single-point membership inference approaches chance.

  • Prediction margin: Dataset inference measures the difference between a model’s responses to training-set and non-training-set inputs using prediction margins.For the theoretical analysis, prediction margin is defined as y · f(x), the distance of a point from the decision boundary.
  • Prediction margin: Dσ2 is the expected prediction-margin gap between training samples and samples from the data distribution for the analyzed linear classifier.Here, σ2 is the variance of the Gaussian noise dimensions.
  • Dataset inference versus membership inference: Membership inference makes a single-point decision without access to the victim’s training set, whereas dataset inference aggregates evidence across multiple points.The aggregate signal is available through expectations over multiple samples, even when individual points are difficult to classify as members.
  • Dataset inference versus membership inference: As training-set size increases, membership-inference success approaches a coin flip because its success is tied to overfitting.The theoretical setup models membership inference as distinguishing points sampled from the training set or the overall data distribution.
  • Dataset inference versus membership inference: Dataset-inference success is high and independent of training-set size in the stated linear setting.The paper’s example with 50K training points reports dataset-inference probability approximately 1.0, versus 0.526 for membership inference.

5 DATASET INFERENCE

Dataset inference tests whether an adversary model contains private knowledge from a victim’s training set by comparing prediction-margin embeddings and confidence scores. It supports white-box and black-box settings, followed by statistical hypothesis testing.

  • Dataset inference determines whether a suspected model incorporates the victim’s private knowledge directly or indirectly.The approach measures how the adversary model responds to the victim’s training data.
  • Embedding generation: Prediction-margin embeddings characterize each data point’s distance from the model’s decision boundaries.Deep networks require embeddings rather than the simple linear-network margin y · f(x).
  • Embedding generation: White-box MinGD estimates distances to target classes through gradient-based optimization and uses those distances as embedding features.The method minimizes perturbation distance subject to changing the model’s target class, using ℓ1, ℓ2, or ℓ∞ distances.
  • Embedding generation: Black-box Blind Walk estimates prediction margins from robustness to random noise when only label-query access is available.This avoids gradients and is designed to reduce the cost of querying a deployed model.
  • Ownership testing: A confidence regressor is trained on private and public embeddings, then a one-sided test compares equal-sized private and public confidence samples.Rejecting H0: µ < µV supports the conclusion that the adversary model is stolen.

6 EXPERIMENTAL SETUP AND IMPLEMENTATION OF DATASET INFERENCE

The experiments evaluate dataset inference on standard image-classification benchmarks against multiple model-stealing threats, including adaptive attacks. The implementation combines confidence regression, statistical testing, repeated trials, and black- or white-box embeddings.

  • Datasets and models: Experiments use CIFAR10, CIFAR100, SVHN, and ImageNet, with WideResNet victim models for CIFAR10 and CIFAR100.The CIFAR victim architecture is WRN-28-10 with dropout rate 0.3; stealing attacks use smaller architectures in the CIFAR settings.
  • Threat models: The evaluation covers six attackers, including query-based extraction, direct model access, fine-tuning, and data-free distillation.An adaptive zero-shot attack specifically targets dataset inference, while an independently trained model serves as a control.
  • Threat models: Training uses unlabeled TinyImages or SVHN extra data for extraction and fine-tuning, while zero-shot learning uses data-free adversarial distillation.Student models are trained for 20 epochs for extraction and 5 epochs for fine-tuning.
  • Implementation: The confidence regressor is a two-layer linear network with tanh activation trained to distinguish public from victim-private data embeddings.Its loss assigns b = 1 to public-model training points and b = −1 to victim-private points.
  • Statistical testing: Hypothesis tests compare confidence distributions from private and public samples, aggregate repeated p-values with the harmonic mean, and construct bootstrap 99-percentile intervals.Each statistical test is repeated 100 times, while confidence intervals use 40 repetitions.

7 RESULTS

Dataset inference is robust across strong and adaptive stealing attacks, while requiring relatively few private samples and no retraining or overfitting. It also remains practical for black-box API assessment.

  • Robustness: DI is robust to strong model-stealing techniques and an adaptive zero-shot attack, with concurring results on SVHN and ImageNet.The evaluation reports at least 95% confidence for most threat models using only 10 samples.
  • Attack comparison: Fine-tuned models are consistently flagged as stolen, whereas label-query and zero-shot attacks challenge DI more strongly.The latter attacks leak less private knowledge or use synthetic queries, while requiring greater query complexity.
  • Sample efficiency: 40, and often as few as 20, private samples achieve false positive rate α ≤ 1%.Figure 3 relates the number of revealed private points to the p-value used for ownership claims.
  • Query efficiency: For 100 data points, DI requires fewer than 30,000 API queries in the black-box setting.The authors note that embedding-generation optimizations could reduce this further.
  • Practicality: DI applies post hoc without overfitting or retraining the defended model.This allows owners to assess models that have already been publicly deployed.

8 DISCUSSION AND CONCLUSION

The discussion frames dataset inference as an ownership-resolution method that uses the owner’s training-set knowledge without modifying deployed models. The theoretical analysis studies prediction margins in a linear classification setting, while the authors identify adaptation and privacy-preserving training as boundaries for future work.

  • Discussion and conclusion: Dataset inference lets developers resolve ownership conflicts without changing existing models.It leverages training-set knowledge to identify copies produced through direct access or indirect distillation.
  • Limitations: Ownership claims degrade as adversaries spend more resources fine-tuning stolen models with different data.The paper links this boundary to model divergence, accuracy degradation, and catastrophic forgetting.
  • Future work: The interaction between dataset inference and ε-differentially private training remains future work.The authors hypothesize that DI’s use of multiple private samples may require lower ε values to become ineffective than membership inference does.
  • Theoretical analysis: The theoretical analysis derives learned linear-classifier weights and compares expected margins for training points with those of non-training points.The proof assumes fixed-step gradient descent and analyzes points included in training or not.

A.3 FAILURE OF MEMBERSHIP INFERENCE (THEOREM 2)

Theorem 2 shows that single-point membership inference becomes ineffective as training sets grow, while dataset inference exploits the victim’s access to its private data to resolve ownership.

  • Membership decision rule: The adversary classifies a point using its prediction margin, comparing behavior on training-set samples with behavior on samples from the task distribution.The decision threshold is tuned to balance true positives and false positives.
  • Theorem 2 conclusion: For a single queried point, the adversary’s optimal membership decision cannot reliably distinguish training from test membership.The theorem formalizes the limitation under the stated linear-model analysis.
  • Failure of membership inference: As the training-set size increases, membership inference approaches coin-flip performance because overfitting decreases.The analyzed success probability tends to 0.5 as m grows.
  • Dataset inference contrast: Dataset inference reverses the game by comparing mean prediction margins on the victim’s private training set and an independently sampled dataset.The victim has access to its private training data and uses this information asymmetry to test a suspected stolen model.
  • Dataset inference contrast: The dataset-inference margin distribution is independent of training-set size, so its success is not curtailed by reduced overfitting.The analysis gives t ∼ N(Dσ2, 2Dσ4) and shows success approaching 1 as D grows.

C EMBEDDING GENERATION

The paper generates embeddings by estimating how far inputs are from neighboring decision regions, using either gradient-based white-box attacks or randomized black-box walks.

  • White-box embedding generation: MinGD generates embeddings by minimizing an ℓp perturbation that changes an input’s predicted class.The optimization uses gradient descent over targeted misclassification distances.
  • Black-box embedding generation: Blind Walk estimates prediction margins by repeatedly moving in random directions until the model changes its predicted label.Multiple random directions provide distance features describing a point’s local robustness.
  • Design rationale: Blind Walk generally outperforms the white-box MinGD method despite MinGD’s more accurate nearest-target distance estimates.The result motivates using prediction-margin characterization rather than exact adversarial distances.
  • Design rationale: The method’s goal is to characterize the local prediction landscape rather than compute the exact minimum distance to a target class.The paper argues that adversarially optimized distances may not represent prediction confidence as well as randomized exploration.

D EFFECT OF EMBEDDING SIZE

Richer distance embeddings reduce the number of private samples needed for dataset inference, but adding features beyond a moderate size often yields less benefit than revealing new samples.

  • Embedding-size trade-off: Richer embeddings reduce the number of revealed samples needed across the evaluated models.The reported analysis examines p-value performance as embedding size changes.
  • Embedding-size trade-off: Using 10 embedding features is sufficient for most threat models to reach the required p-value threshold of 0.01.This can reduce the number of black-box queries by one-third without loss of prediction confidence.
  • Embedding-size trade-off: When only 15 samples are revealed, adding more embedding features has little advantage over querying fresh samples.The paper reports that this pattern is not consistent in the zero-shot learning threat model.
  • Datasets: The evaluated benchmarks include CIFAR10, SVHN, CIFAR100, and ImageNet.The section’s dataset context spans small- and large-scale image-classification benchmarks.

E.2 ADDITIONAL DATASETS

Additional experiments extend dataset inference to SVHN and ImageNet, including random-query and large-scale settings, and report confident ownership detection with few private examples.

  • SVHN: Dataset inference remains resilient to completely random queries on SVHN.The extracted model reaches 90.2% test accuracy using random queries alone, making this threat model evaluable on SVHN.
  • ImageNet: The ImageNet experiment evaluates dataset inference under complete data theft, where the adversary directly obtains the victim’s training dataset.The paper presents this as a large-scale validation setting because prior extraction demonstrations had not established efficacy on ImageNet.
  • ImageNet: On ImageNet, 10 victim training examples produce p-values below 10^-30 for the evaluated Wide ResNet-50-2 setting.The confidence regressor generalizes to other points in the victim’s training and test sets.
  • ImageNet: With 10 ImageNet training examples, dataset inference reports p-values below 10^-4 for AlexNet and Inception V3 models.These models were trained through disjoint procedures, supporting evaluation across architectures.

F EXTENT OF OVERLAP

Dataset inference tests how much overlap between an adversary’s dataset and the victim’s private dataset is needed to claim theft. It succeeds when overlap is present, while avoiding a theft claim as overlap approaches zero.

  • Experimental setup: The experiment varies the fraction of the victim’s private training set included in the adversary’s dataset.The adversary trains on a union containing its own data and varying fractions of the victim’s private data.
  • Results: For fractional data overlap, dataset inference reaches p-value < 0.01 in under 10 revealed training samples.With no dataset overlap, the p-value increases as more samples are revealed, reducing confidence in a theft claim.
  • Mechanism: Training on shared points affects their prediction margins, enabling dataset inference when the victim queries those points.The adversary optimizes prediction margin over all training points, including points private to the victim.
  • Results: p-value = 10^-3 for overlap fractions 0.3, 0.5, 0.7, and 1.0 when the victim queries 50 private train and test samples.The validation uses SVHN, with the adversary trained on extra images plus varying fractions of the train set.
Loading 2104.10706v1…