PaperlayerView exact revision on arXiv

Source-linked AI summary

PRADA: Protecting against DNN Model Stealing Attacks

Mika Juuti, Sebastian Szyller, Samuel Marchal, N. Asokan

arXiv:1805.02628v5cs.CR

TL;DR

The paper addresses model extraction through prediction APIs, which threatens model confidentiality and can enable transferable adversarial examples. It develops stronger extraction attacks and PRADA, reporting improved attack performance and complete detection of prior attacks without false positives.

  • Problem

    Prediction APIs leak information that adversaries can use to extract confidential DNN models, while stolen models can support transferable adversarial examples.

  • Method

    The paper develops extraction attacks using synthetic-query generation and optimized training hyperparameters, then proposes PRADA to analyze successive API-query distributions.

  • Results

    The attacks outperform prior methods by +29-44 pp in adversarial-example transferability and up to +46 pp in prediction accuracy, while PRADA detects prior attacks with no false positives.

  • Takeaways & Limitations

    Effective extraction does not require the target’s training hyperparameters, and stateful query-distribution analysis can detect prior extraction attacks without model or training-data knowledge.

Abstract

from arXiv · show

Machine learning (ML) applications are increasingly prevalent. Protecting the confidentiality of ML models becomes paramount for two reasons: (a) a model can be a business advantage to its owner, and (b) an adversary may use a stolen model to find transferable adversarial examples that can evade classification by the original model. Access to the model can be restricted to be only via well-defined prediction APIs. Nevertheless, prediction APIs still provide enough information to allow an adversary to mount model extraction attacks by sending repeated queries via the prediction API. In this paper, we describe new model extraction attacks using novel approaches for generating synthetic queries, and optimizing training hyperparameters. Our attacks outperform state-of-the-art model extraction in terms of transferability of both targeted and non-targeted adversarial examples (up to +29-44 percentage points, pp), and prediction accuracy (up to +46 pp) on two datasets. We provide take-aways on how to perform effective model extraction attacks. We then propose PRADA, the first step towards generic and effective detection of DNN model extraction attacks. It analyzes the distribution of consecutive API queries and raises an alarm when this distribution deviates from benign behavior. We show that PRADA can detect all prior model extraction attacks with no false positives.

I. INTRODUCTION

The paper frames DNN model confidentiality as important because models carry business value and stolen models can support transferable adversarial examples. It introduces stronger extraction attacks and PRADA, a generic detection technique for API-based extraction.

  • Motivation: MLaaS and protected-device deployments expose models through prediction APIs while preserving their business value.Owners can monetize API access, but the model itself remains a valuable confidential asset.
  • Model extraction: Prediction APIs leak enough information for adversaries to iteratively train substitute models from crafted queries and returned predictions.The substitute can guide later queries, progressively refining its approximation of the target.
  • Research gap: Prior extraction attacks were narrowly scoped or demonstrated only on simple models, leaving generic DNN extraction defenses underdeveloped.The paper identifies this gap as a motivation for studying both more effective attacks and generic detection.
  • Contributions: +29-44 pp transferability and up to +46 pp prediction accuracy are reported for the proposed attacks versus prior attacks.These improvements cover targeted and non-targeted adversarial-example transferability and prediction accuracy.
  • Contributions: PRADA analyzes successive API-query distributions and reports 100% detection with no false positives on prior extraction attacks.The method is presented as a generic defense applicable without requiring the target model’s internals or training data.

C. Goals

The section describes why adversaries extract models: to reproduce predictions or transfer adversarial examples, while minimizing queries and natural samples. It also outlines the black-box attack setting and iterative extraction process.

  • Adversary goals: A substitute model seeks to reproduce the target’s predictions over either the whole input space or a relevant subset.Agreement can be measured by random sampling of the input space or by a held-out test set.
  • Adversary goals: Transfer attacks use minimally modified inputs to change a model’s classification to a chosen class or any other class.These are respectively targeted and non-targeted adversarial examples.
  • Adversary goals: Adversaries also seek to minimize prediction queries to reduce detection risk, API costs, and natural-sample requirements.Query efficiency is therefore a secondary objective alongside model reproduction and adversarial-example transfer.
  • Adversary model: The attack model assumes an isolated target accessible through an API that may return labels only or full class probabilities.The adversary may know the meaning of the classes and the input form without knowing the target training distribution.
  • Extraction process: Extraction iteratively labels seed and synthetic samples, trains a substitute, and repeats duplication rounds until the query budget is consumed or the process terminates.New synthetic samples are generated using information accumulated in the labeled set and current substitute model.

1) TRAMER attack

The section presents Tramer et al.’s extraction attacks as baselines for simple models and contrasts them with neural-network extraction procedures using seed data, selected architectures, and synthetic queries.

  • TRAMER attack: The baseline extraction process initializes a substitute, labels samples through the target, trains on the labeled data, and repeats synthetic-data rounds.Algorithm 1 represents this process through LABEL, TRAIN, synthetic-sample creation, and repeated retraining steps.
  • TRAMER attack: The neural-network baseline uses uniformly random input points, assumes knowledge of the target architecture, hyperparameters, and training strategy, and uses no natural samples.Its main query strategies include random selection, line-search retraining near the substitute boundary, and adaptive retraining.
  • TRAMER attack: The paper contrasts this baseline with settings using a small, balanced set of natural seed samples and strategies that do not require knowledge of target hyperparameters or training strategy.Expert knowledge is still used to choose an architecture appropriate for the classification task.
  • TRAMER attack: The paper investigates hyperparameter selection, synthetic sample generation, and whether probability outputs improve extraction outcomes.These dimensions extend beyond the narrowly scoped prior techniques.

A. Hyperparameters

The section motivates hyperparameter search for extraction and describes CV-SEARCH using five-fold validation and Bayesian optimization, alongside gradient-based methods for synthetic samples and adversarial examples.

  • A. Hyperparameters: Learning rate and training epochs affect neural-network predictive performance, so poor choices can prevent or overshoot useful solutions.The paper identifies rule-of-thumb, copying target settings, and cross-validated search as three selection strategies.
  • A. Hyperparameters: CV-SEARCH evaluates hyperparameter combinations by five-fold cross-validation on the initial labeled seed samples.The selected combination maximizes average validation accuracy across the five folds.
  • A. Hyperparameters: Bayesian optimization chooses later hyperparameter trials using predicted validation value and uncertainty after initial samples are evaluated.This makes the search more efficient than testing every combination when time is limited.
  • B. Adversarial Example Crafting: Jacobian-based methods modify samples according to feature impacts on classification loss to generate adversarial or synthetic queries.Targeted changes use a target-class gradient direction, while non-targeted changes move against the genuine-class likelihood.
  • B. Adversarial Example Crafting: FGSM, I-FGSM, and MI-FGSM differ in whether they use one step, iterative steps, or accumulated momentum, with perturbations bounded by L∞ distance ϵ.The figure contrasts non-targeted FGSM’s failure to generate novel points with T-RND I-FGSM’s feature variation and random-class targeting.

C. Synthetic Sample Generation

The paper develops synthetic-query strategies that either use the partially trained substitute model or operate independently of it. Targeted and iterative crafting methods are designed to move samples toward classification boundaries while reducing overlap and increasing new information.

  • Synthetic samples are generated either with the partially trained substitute model F′ or independently of it.These are called Jacobian-based Synthetic Sample Generation and Random Synthetic Sample Generation, respectively.
  • The number of newly generated samples grows exponentially across duplication rounds, controlled by the expansion factor k.The expansion factor denotes the rate at which synthetic samples increase.
  • Targeted and non-targeted crafting methods move synthetic samples progressively closer to perceived classification boundaries over successive duplication rounds.The methods use adversarial-example crafting algorithms to generate new samples.
  • Iterative FGSM methods reduce sample overlap more effectively than non-targeted FGSM or targeted random-direction crafting in the toy-data illustration.I-FGSM can vary the contribution of different feature components, helping preserve distinct information about the target model.
  • Targeted variants can use expansion factors up to the number of classes C, whereas non-targeted methods use k = 2.The paper sets k = 4 for its targeted variants.

2) Random Synthetic Sample Generation:

The evaluation compares prior attacks and hyperparameter choices on MNIST and GTSRB, showing that CV-SEARCH improves transferability while query volume drives eventual agreement. Iterative crafting is stronger for transferability, and PAPERNOT is the fastest evaluated attack.

  • 2) Random Synthetic Sample Generation:: The study evaluates model-extraction attacks on MNIST and GTSRB using prior methods and new hyperparameter configurations.The experiments replicate prior techniques to assess parameter choices and develop more effective attacks.
  • 2) Random Synthetic Sample Generation:: CV-SEARCH produces the highest initial agreement and transferability among the compared training setups.Its transferability is higher than SAME in the reported MNIST experiment.
  • 2) Random Synthetic Sample Generation:: RU-agreement rises with additional synthetic queries but stagnates after the sixth duplication round, at 6400 samples.The authors suspect limitations of FGSM contribute to this stagnation.
  • 2) Random Synthetic Sample Generation:: CV-SEARCH is the only method whose transferability begins improving exponentially after the seventh duplication epoch.The authors verified that this effect is due to dropout training, while other models may be limited by overfitting.
  • 2) Random Synthetic Sample Generation:: PAPERNOT completes querying and training in 4.5 minutes on average, compared with 26 minutes for CV-SEARCH and 18 minutes for SAME.These times exclude network latency.
  • 2) Random Synthetic Sample Generation:: Iterative I-FGSM and MI-FGSM produce stronger transferability than the FGSM used to craft PAPERNOT’s synthetic samples.After synthetic queries, non-targeted transferability is 2× and targeted transferability up to 3× higher for iterative variants.

C. Impact of seed samples

More natural seed samples generally improve extraction, while probability outputs help transferability more than Test-agreement. Synthetic crafting effects depend on the dataset and objective, with larger perturbations creating a trade-off between transferability and agreement on GTSRB.

  • C. Impact of seed samples: Test-agreement trends are similar across MNIST and GTSRB, but GTSRB achieves lower Test-agreement.The authors attribute this to greater dissimilarity between attacker-set samples and target-model training samples.
  • C. Impact of seed samples: Increasing seeds from 5 to 50 samples per class raises Test-agreement by 23 pp on MNIST and 29 pp on GTSRB.The resulting Test-agreement reaches 93% on MNIST and 47% on GTSRB.
  • C. Impact of seed samples: Probability outputs improve Targeted transferability more than labels-only outputs, especially with more seed samples.On MNIST, Targeted reaches 16% with labels-only and 20% with probabilities at 50 seeds per class.
  • C. Impact of seed samples: With 50 samples per class on MNIST, Non-targeted transferability reaches 48% with labels-only and 65% with probabilities.On GTSRB, Non-targeted transferability reaches 98%, with probabilities adding at most 4 pp over labels-only.
  • D. Synthetic sample generation: On MNIST, T-RND raises Targeted transferability by 4.5 pp on average and T-RND I-FGSM raises Non-targeted transferability by 5.6 pp over baseline.COLOR decreases Test-agreement and transferability, while targeted methods improve Test-agreement by nearly one pp.
  • D. Synthetic sample generation: On GTSRB, targeted methods increase Test-agreement and Targeted transferability, while COLOR has the largest impact on Test-agreement.Non-targeted transferability is already 100% for the baseline and is not reduced by Jacobian-based methods.
  • D. Synthetic sample generation: Larger λ improves both Test-agreement and transferability on MNIST but improves transferability while reducing Test-agreement on GTSRB.The paper therefore separates settings for optimizing transferability from those for optimizing Test-agreement.

E. Comparative evaluation to prior work

The proposed extraction techniques outperform prior methods on agreement and adversarial-example transferability across MNIST and GTSRB. Architecture matching or greater substitute complexity generally improves agreement, whereas lower complexity can undermine extraction and transferability.

  • Comparative performance: +29.3 pp and +31.5 pp: the techniques improve Targeted and Non-targeted transferability on MNIST.With 102,400 queries, T-RND I-FGSM reaches 97.9% Test-agreement, while Targeted and Non-targeted reach 39.9% and 87.7%.
  • Comparative performance: +46 pp and +44 pp: the techniques improve Test-agreement and Targeted transferability on GTSRB.T-RND reaches 47.6% Test-agreement and Targeted reaches 84.8%, compared with 41.1% Targeted for PAPERNOT; Non-targeted is 100.0% in both cases.
  • Comparative performance: CV-SEARCH is crucial for GTSRB Test-agreement, outperforming PAPERNOT even without synthetic samples.PAPERNOT with 110,800 queries does not reach the same Test-agreement as the proposed techniques without synthetic samples.
  • Architecture mismatch: Matching architectures form the baseline for architecture-mismatch experiments, with five target complexities and substitute models trained using SAME.The evaluation varies target and substitute complexity and reports average Test-agreement over ten runs.
  • Architecture mismatch: Matching or higher substitute complexity is almost always beneficial, while lower complexity can cause Test-agreement to drop below its initial level.The paper relates this pattern to the impossibility of perfectly reproducing a high-complexity classifier with a classifier of too low VC dimension.
  • Architecture mismatch: 19 out of 20 architecture deviations significantly decrease Non-targeted transferability, while matching architectures improve transferability.The transferability results are summarized for duplication round 7 using I-FGSM with step size ϵ = 64/255.

G. Takeaways

The paper identifies practical factors that improve model extraction and introduces PRADA, which detects attacks from the distribution of successive query distances. PRADA uses class-aware minimum distances and normality testing rather than assumptions about the target model or training data.

  • Takeaways: Cross-validated hyperparameter search can yield similar or better agreement and transferability without matching the target model’s learning rate or training epochs.The paper treats CV-SEARCH as preferable to copying the target’s training hyperparameters.
  • Takeaways: Natural seed samples are necessary for high Test-agreement, while synthetic samples improve agreement and transferability less efficiently than natural samples.Exploring several synthetic-generation directions with T-RND yields better agreement and transferability.
  • PRADA: PRADA detects attacks spanning several queries by analyzing how successive queries relate, without modeling the content of benign or malicious queries.It makes no assumptions about the model or its training data.
  • PRADA: PRADA computes class-aware minimum distances from each query to retained earlier samples and stores these distances for distribution analysis.For each class, it incrementally maintains a growing set containing samples whose minimum distance exceeds a class-specific threshold.
  • PRADA: Benign-query distances are close to normal, whereas adversarial queries create spikes and skewed distributions that PRADA can detect.The paper presents this contrast for MNIST and GTSRB examples.
  • PRADA: After at least 100 queries, PRADA removes values beyond three standard deviations, computes Shapiro-Wilk W, and flags an attack when W(D′) falls below δ.The method uses a domain-specific distance metric; experiments use the L2 norm for images.

B. Evaluation

PRADA is evaluated for detection success, detection speed, and false positive rate across attacks, models, and query distributions. Its threshold must be selected carefully because detection and false positives vary by model and attack.

  • Evaluation: PRADA is evaluated using detection speed and false positive rate, with speed defined by the number of adversarial samples queried before detection.False positive rate is the ratio of false alarms to benign query sequences.
  • Evaluation: The evaluation tests TRAMER, PAPERNOT, T-RND, and COLOR against the MNIST and GTSRB target models.The attacks use samples generated during the earlier attack-performance evaluation.
  • Evaluation: False positives are assessed across MNIST, USPS, GTSRB, BTS, and randomly generated queries using 30 simulated benign clients.Each client produces 6,000 successive queries, divided into 120 chunks of 50 queries.
  • False positives: δ = 0.96 yields no false positives for MNIST, while GTSRB requires δ = 0.87 for the same result.Benign query distributions differ across GTSRB users and datasets.
  • Detection speed: Most attacks are detected shortly after changing query strategy, including after 100 MNIST queries and 430 GTSRB queries for synthetic-query attacks.TRAMER is detected later, after 5,000 queries, but is itself slow at extracting DNNs.
  • Evaluation boundary: A low δ can miss T-RND against GTSRB, while δ = 0.94 detects it with a 0.1% false positive rate but creates many false positives for GTSRB sequence queries.The paper concludes that δ selection may not transfer across deployment scenarios.
  • Overhead: Before detecting PAPERNOT and T-RND, PRADA stores around 55kB for MNIST and 3.2MB for GTSRB.These estimates use 98 MNIST samples and 343 GTSRB samples in the growing set.

C. Discussion

PRADA’s detection can be evaded by shaping query distributions or distributing queries across clients, but these strategies impose practical costs and constraints.

  • Carefully selecting λ can evade PRADA by making attack-query distances resemble a normal distribution.The authors report that changing λ may degrade the stolen model’s performance.
  • Dummy-query evasion requires queries that complement useful queries so the distance distribution satisfies W(D) ≥δ.The dummy queries are not useful for substitute-model construction and must be compared against a selected subset of the growing set.
  • +3× to +10× more queries are required to circumvent PRADA, depending on the target model and attack.The reported overhead applies to the adaptive dummy-query strategy evaluated by the authors.
  • The query-overhead estimates are lower bounds because the evaluation controlled dmin values without generating the required dummy queries.Creating suitable dummy queries was found to be nontrivial.
  • Random noise, perturbed natural samples, constrained-distance variants, and conditional submission did not fool PRADA.Conditional submission became too costly after approximately 50 samples, requiring thousands of samples to find one valid query.
  • A Sybil attack can circumvent PRADA because detection analyzes queries from a single client.Distributing queries among sufficient numbers of clients avoids the per-client detection scope.
  • The detection threshold δ depends on the benign query distribution defined by the use case.The authors propose selecting δ during a benign-only training period to avoid false positives.
  • PRADA required a few megabytes per client and a few gigabytes for several hundred simultaneous clients.The authors characterize these storage requirements as reasonable for local and cloud-based deployments.

VI. RELATED WORK

Related work includes extraction attacks for simple models, targeted substitute-DNN extraction, and defenses that modify outputs or analyze inputs. PRADA instead targets generic detection from client query behavior.

  • Model Extraction Attacks: Tramer et al. extracted simple models efficiently, but their attacks are ineffective against DNNs.Their methods target logistic regression and decision trees through equation solving and path finding.
  • Model Extraction Attacks: Papernot et al. introduced JbDA for extracting substitute DNNs to generate transferable non-targeted adversarial examples.JbDA generates additional samples near decision boundaries using Jacobian-based augmentation.
  • Model Extraction Attacks: PRADA introduces generic DNN extraction with higher performance for targeted adversarial-example transfer and predictive-behavior reproduction.The paper describes the approach as multipurpose relative to the closest prior works.
  • Model Extraction Attacks: Prior model-stealing studies differ in their assumptions about natural samples, training data, model architecture, and accessible hyperparameters.The paper considers an adversary with limited natural samples, while other work assumes training data or targets architecture and hyperparameter inference.
  • Defenses: Output modification defenses are weakened because model extraction remains effective without prediction probabilities.Quantizing or perturbing probabilities is therefore ineffective against attacks using labels only in the reported evaluation.
  • Defenses: Input-distribution defenses may produce false alarms because they assume benign inputs follow the training-data distribution.Such methods may not generalize to attacks that do not craft adversarial examples.
  • Defenses: PRADA instead tracks how a client’s submitted-sample distribution evolves without assuming the training-data distribution.The paper reports low or no false positives on benign data from diverse distributions.
  • Conclusion: Hiding hyperparameters does not protect against model extraction, whereas hiding model architecture and limiting natural seed samples can help.The conclusion reports different effects for output probabilities, architecture confidentiality, and seed-sample access.

APPENDIX

The appendix situates the paper’s black-box adversary among surrogate-learning and finite-difference attacks, emphasizing different assumptions and query costs.

  • The paper defines a black-box adversary with surrogate data that uses a substitute model for adversarial attacks.This setting is distinguished from surrogate-learning work that may use the same training data without knowing the target classifier type.
  • Surrogate-learning attacks can produce highly transferable adversarial examples for ImageNet models.The cited work demonstrates this possibility without establishing that it extracts the target model.
  • Finite-difference methods can create targeted adversarial examples without substitute models but may require thousands of queries per sample.They generally do not extract models and mostly require access to target-model probabilities.
Loading 1805.02628v5…