PaperlayerView exact revision on arXiv

Source-linked AI summary

DAWN: Dynamic Adversarial Watermarking of Neural Networks

Sebastian Szyller, Buse Gul Atli, Samuel Marchal, N. Asokan

arXiv:1906.00830v5cs.CRstat.ML

TL;DR

Model extraction threatens the intellectual property embodied in ML models, while training-time watermarks do not protect surrogates trained by adversaries. DAWN instead dynamically alters a small subset of prediction-API responses to watermark extracted models. It reliably demonstrated ownership across two state-of-the-art extraction attacks while preserving model utility with negligible accuracy loss.

  • Problem

    Model extraction lets adversaries train surrogate models from prediction-API outputs, making existing training-time watermarking ineffective for this form of IP theft.

  • Method

    DAWN dynamically changes responses for a small subset of prediction-API queries, creating a trigger set that is embedded when a client trains a surrogate.

  • Results

    DAWN reliably demonstrated ownership of several surrogate models from two state-of-the-art extraction attacks with confidence >1 − 2−64 and a 0.03-0.5% accuracy decrease.

  • Takeaways & Limitations

    DAWN enables reliable ownership demonstration for extracted surrogate models while preserving the utility of the original model.

  • Takeaways & Limitations

    An adversary willing to incur the maximal allowable accuracy loss can weaken watermark embedding, and strengthening DAWN against this adversary remains open.

Abstract

from arXiv · show

Training machine learning (ML) models is expensive in terms of computational power, amounts of labeled data and human expertise. Thus, ML models constitute intellectual property (IP) and business value for their owners. Embedding digital watermarks during model training allows a model owner to later identify their models in case of theft or misuse. However, model functionality can also be stolen via model extraction, where an adversary trains a surrogate model using results returned from a prediction API of the original model. Recent work has shown that model extraction is a realistic threat. Existing watermarking schemes are ineffective against IP theft via model extraction since it is the adversary who trains the surrogate model. In this paper, we introduce DAWN (Dynamic Adversarial Watermarking of Neural Networks), the first approach to use watermarking to deter model extraction IP theft. Unlike prior watermarking schemes, DAWN does not impose changes to the training process but it operates at the prediction API of the protected model, by dynamically changing the responses for a small subset of queries (e.g., <0.5%) from API clients. This set is a watermark that will be embedded in case a client uses its queries to train a surrogate model. We show that DAWN is resilient against two state-of-the-art model extraction attacks, effectively watermarking all extracted surrogate models, allowing model owners to reliably demonstrate ownership (with confidence $>1- 2^{-64}$), incurring negligible loss of prediction accuracy (0.03-0.5%).

1 INTRODUCTION

ML models embody substantial business value because their development depends on costly data collection and training. Protecting this intellectual property is therefore necessary.

  • ML models support a wide variety of applications and are increasingly deployed by major enterprises.
  • Training data collection and model training form part of model owners’ business advantage.
  • Protecting the intellectual property embodied in ML models is necessary.

N. Asokan University of Waterloo & Aalto University

API-based model access enables extraction attacks because adversaries can use returned predictions to train surrogate models. DAWN addresses this threat by watermarking API responses and linking extracted models to querying clients.

  • Prediction APIs let adversaries send inputs and use returned predictions to train surrogate models with similar functionality.
  • Existing training-time watermarking is ineffective against extraction because the adversary trains the surrogate model.
  • DAWN dynamically changes responses for a tiny fraction of client queries, creating a trigger set embedded in a surrogate trained from those responses.
  • DAWN watermarks are client-specific, enabling surrogate detection and identification of the client whose queries trained the model.
  • DAWN resisted two state-of-the-art extraction attacks, demonstrating ownership with confidence >1 − 2−64 and reducing accuracy by 0.03-0.5%.

2 BACKGROUND

Model extraction uses black-box API queries and their predictions to train a surrogate model. DNN watermarking instead embeds incorrect behavior on trigger inputs while preserving the primary task elsewhere.

  • In black-box extraction, adversaries query a victim API and iteratively train a surrogate toward the victim’s accuracy.
  • Adversaries may guess the victim architecture or use publicly available high-capacity pretrained models.
  • Limited access to in-distribution natural data leads adversaries to use alternate-domain, widely available, synthetic, or mixed data.
  • Extraction alternates between querying the victim and training the surrogate, using either broad datasets or deliberately crafted inputs.
  • DNN watermarking uses model capacity to encode a secondary task that assigns incorrect classes to a trigger set.
  • A watermark consists of trigger inputs and their backdoor outputs; matching outputs on a copied model can demonstrate ownership.

3 PROBLEM STATEMENT

DAWN targets black-box model extraction by altering a small subset of prediction-API responses so the extracted surrogate learns a verifiable watermark. Its requirements combine ownership evidence, client linkability, utility, and resistance to evasion.

  • Problem setting: Extraction seeks a high-accuracy surrogate by querying the victim API and training on the returned predictions.
  • Adversary model: The adversary controls queried samples, surrogate architecture, hyperparameters, and training process, while expecting repeated inputs to receive consistent predictions.
  • Adversary model: The paper focuses on adversaries who expose extracted surrogates through prediction APIs rather than keeping them private.
  • Problem statement: Existing defenses are difficult to apply because extraction is hard to prevent and prior watermarks are vulnerable when the adversary trains the surrogate.
  • DAWN overview: DAWN modifies selected API responses into incorrect predictions that become a backdoor trigger set in the adversary’s surrogate.
  • DAWN overview: Verification checks whether enough trigger inputs receive the watermark predictions while differing from the victim’s predictions.
  • System requirements: The requirements include unremovability, reliability, non-ownership piracy resistance, client linkability, utility, indistinguishability, and collusion resistance.

4 DYNAMIC ADVERSARIAL WATERMARKS

DAWN dynamically alters a small, client-specific subset of prediction-API responses so extracted surrogates acquire identifiable watermarks without modifying the victim’s training process. Its design balances watermark reliability, prediction indistinguishability, surrogate embedding, and ownership verification.

  • Watermark generation: DAWN defines watermarked inputs as queries receiving incorrect responses, whose trigger set and labels become a backdoor in a surrogate trained on those responses.The watermark is represented as (T_A, B̂_V(T_A)).
  • Watermark generation: The deterministic functions W_V and B_V use model-specific keyed randomness so clients cannot predict watermarking decisions or distinguish altered from ordinary responses.DAWN derives randomness from HMAC-SHA-256 and splits the digest between watermark selection and backdoor responses.
  • Watermark generation: r_w controls trigger-set size, with larger values improving ownership reliability while smaller values reduce incorrect predictions returned to benign clients.The trigger-set size is |T_A| = floor(r_w × N).
  • Watermark generation: The backdoor response applies a keyed pseudo-random permutation to the victim’s prediction, preserving consistency with correct outputs while preventing recovery of the original prediction from observed altered responses.Only high-probability positions need to be permuted when the class count is large.
  • Indistinguishability: A mapping function smooths watermark decisions for nearby inputs, making small perturbations receive the same watermarking outcome and backdoor response.The paper considers autoencoders, masking and binning, and embeddings from an intermediate victim-model layer.
  • Watermark embedding: High-capacity surrogates are expected to memorize incorrectly labeled watermark samples as a backdoor; if they do not, their primary-task accuracy should be too low for practical use.Different clients’ query datasets produce distinctive trigger sets and watermarks, linking extracted models to clients.
  • Watermark verification: Verification succeeds when the disagreement ratio L between the watermark labels and suspected surrogate predictions is below tolerated error rate e.At most floor(e × |T|) trigger samples may disagree; trivial verification success decreases exponentially with watermark size, and e must scale with the number of classes.
  • Demonstrating ownership: Ownership demonstration requires a trusted judge, confidential watermark information, prediction-API access, and publication ordering that establishes the original model’s prior commitment.A surrogate claim is valid only when the suspected model’s commitment is later than the victim’s, or absent.

5 EXPERIMENTAL SETUP

The evaluation uses image-recognition datasets and low- and high-capacity DNNs to study DAWN’s watermark embedding during simulated extraction. Surrogate utility and watermark fidelity are measured separately throughout training.

  • Datasets: DAWN is evaluated on MNIST, GTSRB, CIFAR10, Caltech256, and a 100,000-sample ImageNet subset spanning 10 to 1,000 classes.The datasets cover handwritten digits, traffic signs, miscellaneous objects, and natural or man-made objects.
  • Models: The study compares low-capacity DNNs with fewer than 10M parameters against high-capacity models with more than 20M parameters.The evaluated models are listed in Table 2.
  • Models: The experiments reproduce prior extraction settings using the same architectures and training processes, including ImageNet-pretrained ResNet34 models for high-capacity evaluations.Caltech-RN34, GTSRB-RN34, and CIFAR10-RN34 are fine-tuned on their respective datasets.
  • Extraction simulation: The DAWN-enhanced API returns either the victim’s correct prediction or a backdoor response according to W_V, with the mapping function varied only in the dedicated mapping-function experiments.The identity mapping is used for Sections 6.1 and 7, while an intermediate victim-model embedding is used for Section 6.2.
  • Extraction simulation: Simulated adversaries train surrogates on their complete queried datasets, including incorrectly labeled trigger samples, without knowing which samples were watermarked.The setup evaluates extraction as an unaware training process.
  • Metrics: Test accuracy measures the adversary’s primary-task objective, while watermark accuracy measures the victim’s ability to embed the watermark in the surrogate.Both metrics are computed every five epochs to track their progress during training.

6 ROBUSTNESS OF WATERMARKING

DAWN remains difficult to remove during surrogate training because high-capacity models learn the watermark, while attempts to suppress it through capacity reduction or regularization often damage task accuracy. Its mapping function also preserves most watermarked samples under small input perturbations.

  • Training robustness: High-capacity surrogate models retain high watermark and test accuracy, enabling ownership demonstration with tolerated error rate e = 0.3.Plain training often yields watermark accuracy close to 100%.
  • Model capacity: Reducing model capacity can prevent watermark embedding, but watermark and test accuracy may both approach random performance near 10%.This makes the surrogate model ineffective for the primary task.
  • Regularization: Dropout or weight decay lowers watermark accuracy in selected low-capacity settings, but dropout can also reduce test accuracy by up to 50%.Dropout completely prevents watermark embedding in MNIST-5L while substantially reducing utility.
  • Regularization: No tested regularization method removes the watermark from high-capacity GTSRB-RN34 and CIFAR10-RN34 models.Their overcapacity for the primary task prevents regularization from limiting capacity enough to suppress the watermark.
  • Adversarial constraints: An adversary needs relevant test data and ground truth to optimize regularization parameters, but extraction attacks typically use available data for surrogate training.This prevents reliable optimization of regularization and early stopping.
  • Perturbation resistance: 85% of the watermark remains preserved when perturbations satisfy δ ≤ 0.1, because M_V provides consistent mappings in over 85% of cases.The evaluation queried 10 perturbed versions of each of 10,000 samples, including 121 watermarked samples.

7 PROTECTING AGAINST MODEL EXTRACTION ATTACKS

DAWN protects prediction APIs against PRADA and KnockOff extraction by embedding client-specific watermarks into surrogate models through selectively changed API responses. Across tested models, it supports reliable ownership verification while keeping victim-model utility loss negligible, although distributed collusion increases the required watermark ratio.

  • Evaluation setup: DAWN is evaluated against PRADA for low-capacity models and KnockOff for high-capacity models, using synthetic or natural query data respectively.The experiments cover MNIST-5L, GTSRB-5L, CIFAR10-9L, GTSRB-RN34, CIFAR10-RN34, and Caltech-RN34.
  • Verification design: 1 − 2^-64 confidence is targeted for ownership demonstration with tolerated error rate e = 0.5 across the evaluated victim models.The watermark size is computed from the required attack queries, confidence, tolerated error rate, and number of classes.
  • Victim utility: r_w < 0.5% suffices to reach 1 − 2^-64 confidence with 1M registered clients, causing negligible degradation of victim-model accuracy.The setting requires probability of trivial verification below 5.4 × 10^-26.
  • Extraction defense: All tested surrogate models achieve watermark accuracy A_ccwm ≥ 50%, allowing successful ownership demonstration against both PRADA and KnockOff.DAWN has little impact on victim utility, although MNIST-5L surrogate test accuracy decreases significantly.
  • Distributed attacks: DAWN deterministically preserves the same trigger-set responses across clients, so colluding clients contribute subwatermarks that remain individually linkable.The judge can verify each colluding client’s subwatermark and retain linkability W4.
  • Distributed attacks: Distributed collusion requires increasing r_w by the number of colluding clients, and Sybil accounts must be mitigated through stronger registration and detection.The paper assumes the number of real colluders is limited to a few tens.

8 WATERMARK REMOVAL

The section evaluates attacks that remove DAWN watermarks, showing that watermark removal often requires substantial accuracy loss, especially when the adversary has limited data. With unlimited natural data, some extraction-based attacks can preserve utility while removing the watermark.

  • Prior backdoor-detection methods are ineffective against DAWN because its trigger set uses unmodified samples with incorrect labels rather than static input patterns.
  • Double extraction and stealing+fine-tuning can remove DAWN watermarks, but they incur 20-80% additional test-accuracy degradation.
  • KnockOff-based double extraction can remove the watermark while preserving utility, with final accuracy only 1-10 percentage points below the victim model.
  • PRADA-based extraction produces final models that are often less than half as accurate as the victim, making them too inaccurate to be useful.
  • Pruning removes the watermark only at large pruning levels, which sacrifices model utility; test accuracy and watermark accuracy also do not decline proportionally.
  • Training or inference noise generally fails to defeat verification without unacceptable utility loss, while perturbed-query recognition imposes substantial search overhead.

9 DISCUSSION

The discussion argues that DAWN meets its intended reliability, utility, unremovability, and linkability requirements, while identifying adversarial data access and mapping circumvention as scope boundaries. Several evasion strategies can work only by sacrificing utility, increasing data needs, or accepting additional attack capabilities.

  • 9.1 Meeting system requirements: DAWN meets unremovability because training manipulations, pruning, and fine-tuning either fail to remove the watermark or substantially reduce model utility.
  • 9.1 Meeting system requirements: DAWN reliably demonstrates ownership of surrogate models with confidence 1-2^-64 while decreasing the victim model’s utility by only 0.03-0.5%.
  • 9.2 Limitations: Discarding enough training data can make watermark verification fail, but it reduces surrogate utility and requires more relevant data and API queries.
  • 9.2 Limitations: The authors do not consider data-discarding evasion realistic because it demands greater adversarial capabilities and conflicts with minimizing victim-model queries.
  • 9.2 Limitations: Mapping-function circumvention remains a potential limitation, although synthetic-sample behavior is undefined and experiments found mapped predictions consistent under minor perturbations.
  • 9.2 Limitations: Noise-based evasion can succeed in some cases when the adversary accepts a fixed accuracy-loss budget, including a 10 pp loss in the reported examples.

10 RELATED WORK

Prior watermarking methods either require white-box access or depend on owner-controlled training and trigger-set selection. DAWN instead embeds client-specific watermarks dynamically at the prediction API and targets extracted surrogate models.

  • Weight-embedded watermarks require white-box verification and can be removed through minimal retraining.These methods embed information into DNN weights after training.
  • Black-box watermarking improves robustness but requires a carefully selected trigger set and owner-controlled training process.These approaches use backdooring and verify watermarks through a prediction API.
  • DAWN targets victims who neither control DNN training nor select a trigger set from the whole input space.It dynamically embeds the watermark in queries made to the model prediction API.
  • DAWN is presented as a defense that enables model owners to identify surrogates of their models.
  • Unlike prior defenses, DAWN assumes a surrogate can be extracted and aims to identify surrogate DNNs produced by any extraction attack.Earlier defenses protect only simple models or specific extraction attacks.

A DATASETS AND MODELS

The experiments use image datasets divided into training and testing sets, with images resized for the evaluated architectures. Low-capacity model architectures support experiments on perfect-knowledge and PRADA attacks.

  • The evaluated image datasets are divided into training and testing sets.Different sample sizes are input to different models.
  • Images are resized to fit the corresponding model architectures used in prior work.
  • Low-capacity model architectures are used for perfect-knowledge experiments and reproduction of the PRADA attack.

B DETECTING WATERMARKED INPUTS

The evaluated clustering defense projects last-hidden-layer activations into three dimensions and applies two-cluster k-means to detect watermarked inputs. It fails because DAWN watermarks mix with clean inputs, causing substantial clean-data loss and low watermark detection.

  • The defense projects last-hidden-layer activations into three dimensions with ICA and clusters them into two groups using k-means.The intended groups are benign training data and poisoned, watermarked inputs.
  • Watermarked inputs cannot be reliably isolated into a single cluster.Figure 3 shows watermarked and correctly labeled inputs mixed together, preventing cluster-based isolation.
  • 26.4% of clean data is wrongly discarded while only 67 of 250 watermarked samples are detected.The detected fraction is 26.8% of the watermark set.
  • Silhouette scores do not distinguish watermarked from plain models, and plain models are also falsely detected as watermarked.The scores remain above the recommended detection threshold.
  • DAWN watermarks resemble the training-data distribution, so models learn individual exceptions rather than one generalized watermark activation.This differs from prior solutions whose watermarked inputs come from a distant single manifold.
Loading 1906.00830v5…