PaperlayerView exact revision on arXiv

Source-linked AI summary

Entangled Watermarks as a Defense against Model Extraction

Hengrui Jia, Christopher A. Choquette-Choo, Varun Chandrasekaran, Nicolas Papernot

arXiv:2002.12200v2cs.CRstat.ML

TL;DR

Model extraction can remove conventional watermarks because they are learned separately from task behavior, creating a tension between ownership protection and model utility. EWE entangles watermark and task representations so extraction retains the watermark, achieving 95% confidence ownership claims in fewer than 100 queries with below 0.81 percentage points average utility cost.

  • Problem

    Conventional watermarks are outliers distinct from the task distribution, so extraction through task-distribution queries can remove them while preserving task behavior.

  • Method

    EWE jointly learns task classification and watermark outputs by entangling their representations with a soft nearest neighbor loss.

  • Results

    Across vision and audio datasets, EWE achieves over 38% average watermark retention after extraction and enables 95% confidence ownership claims with fewer than 100 queries, while average utility loss stays below 0.81 percentage points.

  • Takeaways & Limitations

    Removing entangled watermarks requires harming the stolen classifier’s generalization on legitimate task data, making watermark robustness compatible with extraction resistance.

  • Takeaways & Limitations

    EWE’s robustness depends on keeping the trigger and watermarking parameters secret, and scaling to complex tasks without large accuracy loss remains open.

Abstract

from arXiv · show

Machine learning involves expensive data collection and training procedures. Model owners may be concerned that valuable intellectual property can be leaked if adversaries mount model extraction attacks. As it is difficult to defend against model extraction without sacrificing significant prediction accuracy, watermarking instead leverages unused model capacity to have the model overfit to outlier input-output pairs. Such pairs are watermarks, which are not sampled from the task distribution and are only known to the defender. The defender then demonstrates knowledge of the input-output pairs to claim ownership of the model at inference. The effectiveness of watermarks remains limited because they are distinct from the task distribution and can thus be easily removed through compression or other forms of knowledge transfer. We introduce Entangled Watermarking Embeddings (EWE). Our approach encourages the model to learn features for classifying data that is sampled from the task distribution and data that encodes watermarks. An adversary attempting to remove watermarks that are entangled with legitimate data is also forced to sacrifice performance on legitimate data. Experiments on MNIST, Fashion-MNIST, CIFAR-10, and Speech Commands validate that the defender can claim model ownership with 95\% confidence with less than 100 queries to the stolen copy, at a modest cost below 0.81 percentage points on average in the defended model's performance.

1 Introduction

Model extraction threatens valuable ML intellectual property, while conventional defenses trade away utility. EWE addresses watermark fragility by entangling watermark and task representations, preserving utility while improving extraction robustness.

  • Model extraction uses victim-model queries to label substitute data and train a stolen copy, while benign and attack queries may come from the same task distribution.
  • Existing watermarking overfits defender-known outlier input-output pairs, enabling ownership claims without reducing task-distribution accuracy.
  • Naive watermarks are removable because attackers querying only task-distribution inputs retain the task decision surface while ignoring the watermark surface.
  • EWE jointly learns task classification and defender-specified watermark outputs by entangling their representations, forcing watermark removal to harm task generalization.
  • 95% confidence ownership claims require fewer than 100 queries to stolen copies, while average defended-model utility costs remain below 0.81 percentage points across evaluated datasets.

2 Background

The paper frames watermarking as an ownership-verification alternative to preventing extraction. It reviews supervised classification, query-based extraction, and the utility trade-offs of extraction countermeasures and watermarking.

  • Supervised classification learns a decision function mapping inputs to discrete classes from examples representing the task distribution.
  • Model extraction collects or synthesizes substitute inputs, queries the victim for predictions, and trains a copy using the resulting labeled dataset.
  • The threat model excludes differential-querying and side-channel attacks, focusing on prediction-query attacks seeking roughly the victim’s accuracy.
  • Restricting prediction information can impede extraction, but countermeasures create an inherent trade-off between benign-user utility and extraction efficiency.
  • Watermarking embeds defender-known outlier input-output pairs, often triggered inputs, so unusual predictions can support ownership claims.

3 Difficulties in Watermarking

Naively trained watermarks and task behavior can occupy separable functional components, allowing extraction from task-distribution queries to remove watermark behavior. The paper’s analytical and empirical examples motivate entanglement.

  • The paper’s toy analysis studies a binary task with 2D inputs and a watermark that forces output 1 when x2 = -1.
  • Figure 1 contrasts the watermark model with an extracted copy, illustrating independent task and watermark distributions despite shared neurons.
  • After extraction, the copy retains high task utility but outputs 0 rather than the watermark target 1 on the OOD input x2 = -1.
  • Task-distribution extraction cannot selectively update the small neuron groups used only for watermarking, so those watermark behaviors are not retained.
  • Empirically, baseline watermarking activates different and fewer neurons for watermark data, supporting a rough partition into task and watermark sub-models.

4 Entangling Watermarks

EWE targets the separability underlying watermark loss by jointly representing task and watermark data. It uses SNNL-based training and assumes attackers lack the secret trigger and calibration parameters.

  • Threat Model: The threat model grants attackers training-data and architecture knowledge, while withholding watermarking parameters and the trigger.
  • Entangled Watermark Embedding: EWE uses SNNL to make task and watermark representations, and therefore their activation patterns, similar.
  • Soft Nearest Neighbor Loss: SNNL measures entanglement by comparing distances between points from different groups with within-group distances in representation space.
  • Entangled Watermark Embedding: The method is presented as Algorithm 1, taking task and watermark data, labels, temperature, class parameters, training controls, a model, and a trigger as inputs.
  • Entangled Watermark Embedding: Optional FGSM-based gradient ascent perturbs watermarked inputs to reduce confidence in the target class and improve transfer across models.

6 FGSM(Xw,SNNL([Xw,XcT ],Y ′,T))/* optional

EWE generates and trains entangled watermarks so watermark features overlap with legitimate task representations, improving robustness to model extraction while preserving task accuracy. Across evaluated datasets and training analyses, EWE supports ownership verification with substantially higher post-extraction watermark success than the baseline.

  • Watermark generation and training: EWE generates watermarks from a source distribution, adds a trigger, and trains them jointly with target-class data using entanglement objectives.The trigger should preserve semantic distinction from target-class data; optional gradient ascent further reduces target-class confidence while SNNL encourages entanglement.
  • Ownership verification: Fewer than 100 queries suffice for ownership claims with 95% confidence at the lowest observed EWE success rate of 18.74% on CIFAR-10.For success rates above 23%, the required number of queries is 30, the minimum used for the CLT-based analysis.
  • Increased entanglement: EWE represents watermarked and legitimate target-class data similarly, whereas cross-entropy-only training pushes watermarks into a separate cluster.Penultimate-layer visualizations, neuron activation patterns, and CKA measurements all support increased representation similarity under EWE.
  • Utility and training trade-off: 0.81 percentage points is the average validation-accuracy degradation for EWE, with a maximum of 3 percentage points for a ResNet on Fashion MNIST.The reported trade-off improves with continued SNNL maximization until it plateaus near 60 epochs.
  • Robustness against extraction: 38.39% average post-extraction watermark success is achieved by EWE, compared with 5.77% for the baseline.Across datasets, EWE reaches 18%–60% success while the baseline reaches 0.3%–9%.
  • Scalability to deeper architectures: Comparable ResNets cannot entangle the evaluated out-of-distribution watermarks, although EWE still achieves sufficient watermark success for ownership claims.Residual connections pose a greater entanglement challenge because the outlier watermarks share few features with legitimate data.

5 Calibration of Watermark Entanglement

EWE is calibrated across vision and audio datasets to characterize the trade-off between task accuracy, watermark robustness, and parameter choices. The experiments find that entanglement resists defenses such as pruning, fine-pruning, and Neural Cleanse, although reducing watermark success can substantially damage legitimate-data accuracy.

  • Experimental setup: EWE is evaluated on four vision datasets and Google Speech Commands to study robustness, accuracy, configuration, and backdoor defenses.The evaluation includes MNIST, Fashion MNIST, CIFAR-10, CIFAR-100, and Google Speech Commands.
  • Watermark-utility trade-off: Watermark success above 40% is achieved on MNIST, Fashion MNIST, and Speech Commands with less than 1 percentage point test-accuracy loss.For CIFAR datasets, watermark success above 18% is reached with accuracy loss below 1.5 percentage points.
  • Parameter calibration: The watermark-to-legitimate-data ratio, source-target class pair, and selected watermark points significantly affect EWE performance, whereas temperature does not.The weight factor controls the trade-off between watermark robustness and task accuracy, while temperature is automatically optimized during training.
  • Watermark-utility trade-off: As test accuracy increases, robust watermarking becomes harder, with the relationship varying across datasets and hyperparameter choices.Fashion MNIST shows an exponential tendency, while Speech Commands contains both near-zero watermark-success models and high-success, low-accuracy-cost settings.
  • Defenses against backdoors: Pruning reduces watermark success below 20% only after legitimate-data accuracy drops by more than 40 percentage points, making pruning ineffective against EWE.Entangled watermarks activate neurons frequently on legitimate data, so removing rarely activated neurons preserves watermark behavior despite significant pruning.
  • Defenses against backdoors: Neural Cleanse fails to identify EWE watermarks on Fashion MNIST, where EWE has average anomaly index 1.24 versus 8.84 for the baseline.Anomaly indices above 2 indicate detection in this evaluation.

6 Robustness to Adaptive Attackers

EWE remains resistant to adaptive attacks when attackers know the scheme but not its configuration, although complete watermark removal is possible if configuration secrets are exposed. Across piracy, anomaly detection, and transfer learning, removing owner watermarks requires costly sacrifices in utility, additional data, or compute.

  • 6.2 Knowledge of EWE only: Knowledge of EWE’s configuration defeats watermarking, so robustness depends on keeping triggers and watermarking parameters secret.The paper treats these parameters similarly to cryptographic keys; knowing the trigger also enables denial-of-service responses.
  • 6.2 Knowledge of EWE only: Fine-pruning removes a pirate watermark with less than 10% of neurons pruned while preserving the owner’s watermark.The attacker cannot similarly remove the owner’s watermark without a dataset labeled by another source, whose acquisition cost defeats the purpose of model stealing.
  • 6.2 Knowledge of EWE only: Incorrect source-target guesses reduce watermark success but still leave enough evidence for ownership claims with about 30 queries on vision datasets and nearly 100 on Speech Commands.Correct guesses require training models for K(K −1) possible source-target pairs, imposing significant compute that undermines model extraction.
  • 6.2 Knowledge of EWE only: Anomaly detectors identify more than 90% of watermarked inputs but reduce validation accuracy by 7.0 and 8.64 percentage points for LOF and Isolation Forest, respectively.The detectors also classify portions of the validation dataset as outliers.
  • 6.2 Knowledge of EWE only: Transfer learning from GTSRB to LISA leaves watermarks unaffected at 98.25% accuracy when only fully connected layers are fine-tuned, while larger learning rates weaken watermarks after pretrained knowledge is lost.Fine-tuning all layers reaches 98.56% accuracy but begins weakening the watermark as the learning rate increases.
  • 6.2 Knowledge of EWE only: Overall, attackers cannot remove EWE watermarks through disentanglement, piracy, anomaly detection, or transfer learning without sacrificing stolen-model utility.This is the section’s stated no-free-lunch conclusion for adaptive attacks.

7 Discussion

The discussion identifies utility, computation, scalability, and design-selection trade-offs that constrain EWE beyond the demonstrated attack robustness. It also outlines possible improvements to activation functions, entangled layers, trigger design, and watermark configurations.

  • Hyperparameter Selection: EWE incurs about 0.81% accuracy degradation on average, but the cost varies with dataset and training hyperparameters.The relationship between dataset properties and this variation remains future work.
  • Computational Overheads: Using a watermarked dataset half the size of the legitimate dataset increases training data and computational overhead by 1.5−2×.The paper argues this trade-off may be advantageous for proving ownership, but its behavior on larger, more complex datasets requires further analysis.
  • Improving Utility: Replacing ReLU with smoother Sigmoid activations is proposed as a way to reduce EWE’s nominal utility cost while preserving entangled activation patterns.This is presented as a possible improvement rather than an evaluated result.
  • Algorithmic Efficiency: Applying SNNL only in later layers may preserve the desired guarantees and slightly improve utility compared with entangling every layer.The paper notes that choosing the optimal layers remains insufficiently understood.
  • Scalability and Future Research Directions: EWE’s performance-versus-robustness trade-off worsens for deeper architectures and more complex datasets, with CIFAR-100 indicating limited scalability.More clusters make arbitrary entanglement harder, motivating further study of triggers and representation similarity.
  • Scalability and Future Research Directions: The work focuses on 1-to-1 watermarking, while m-to-n watermarking is proposed as a possible extension when watermarked behavior differs sufficiently from a clean model.The proposed extension would watermark m classes and entangle them with n other classes.

8 Conclusions

The paper concludes that EWE entangles legitimate and watermark representations using an SNNL-based loss, making watermarks robust across several attacks. This robustness comes with nominal accuracy loss and 1.5−2× computational overhead, while scaling to complex tasks remains open.

  • 8 Conclusions: EWE forces legitimate task data and watermarks to share entangled representations through a loss involving Soft Nearest Neighbors Loss.The evaluation covers vision and audio tasks and tests extraction, piracy, anomaly detection, transfer learning, and backdoor-mitigation attacks.
  • 8 Conclusions: EWE preserves watermarking accuracy while incurring nominal classification-accuracy loss and a 1.5−2× increase in computational overhead.These costs are reported alongside robustness across the evaluated attack types.
  • 8 Conclusions: Scaling EWE to complex tasks without substantial accuracy loss remains an open problem.

A.1 Finetuning the hyperparameters of EWE

EWE’s hyperparameters control the trade-off between task accuracy and watermark robustness, while source-target class similarity can affect robustness differently across datasets. The paper uses these analyses to identify calibration choices and dataset-dependent behavior.

  • Temperature: Temperature has little influence on watermark robustness overall, although some initializations produce better task-accuracy and watermark-success trade-offs.Temperature is evaluated on a logarithmic scale.
  • Weight Factor: Larger absolute values of κ increase watermark robustness but reduce task accuracy.Near κ = 10, accuracy resembles an un-watermarked model and watermark success is about 40%; at larger magnitudes, watermark success approaches 100% while accuracy declines significantly.
  • Ratio of task data to watermarks: Lower ratios r of task data to watermarks produce more robust watermarks, but ratios below 1 significantly reduce task accuracy.On Fashion MNIST, extraction can remove the watermark when r is greater than 3.
  • Source-Target classes: Source-target classes are selected using cosine similarity between class centers, with similar structures enabling more robust watermarks without affecting task accuracy.The paper uses MNIST data as out-of-distribution watermarks for Fashion MNIST.
  • Source-Target classes: Higher class similarity correlates with stronger watermark robustness on Fashion MNIST, whereas Speech Commands shows no clear trend.The paper attributes the dataset difference to how easily the relevant classes can be entangled.

A.2 Evasion Attacks for Detection

This section examines whether mechanisms used to generate adversarial examples can also detect watermarks. It introduces adversarial samples and distinguishes targeted from untargeted misclassification.

  • Adversarial samples start with source-class examples and add carefully crafted perturbations to induce misclassification.The intended error may be specified or left unrestricted.
  • Targeted attacks force a chosen incorrect class, whereas untargeted attacks accept any incorrect class.
  • Some adversarial-example attacks use gradients to construct samples with minimum perturbation.
  • The section investigates whether adversarial-sample generation mechanisms can be repurposed for watermark detection.

A.3 Additional Figures

The additional figures illustrate trigger ambiguity, watermark entanglement, cross-dataset behavior, scaling effects, source-target extraction settings, and audio watermark representations. Together, they provide visual comparisons of EWE with baseline watermarking and un-watermarked models.

  • A trigger that changes a digit-3 to a digit-5 can indicate watermarking, but a poorly designed rectangle trigger may cause the same change in an un-watermarked model.
  • Increasing convolutional layers is evaluated using both validation accuracy and watermark success for an in-distribution watermark on Fashion-MNIST.
  • EWE makes legitimate and watermarked data representations more similar in deeper layers than baseline watermarking.The activation figures compare legitimate and watermarked data across convolutional and fully connected layers.
  • The figures also show MNIST and Speech Command trigger configurations and provide an example watermarked audio signal with its corresponding Mel Spectrogram.The Speech Command example uses audios of people saying “one” as out-of-distribution watermarks.
  • On CIFAR-100, both baseline and EWE have substantially lower accuracy than an un-watermarked model as the number of classes increases, while EWE achieves higher watermark success than baseline.
  • Extraction experiments evaluate all 90 source-target class pairs on MNIST, Fashion-MNIST, and Speech Command under the same setting.A source-target pair specifies the source class of watermarked data and the target class predicted by the model.
Loading 2002.12200v2…