Source-linked AI summary
Have You Stolen My Model? Evasion Attacks Against Deep Neural Network Watermarking Techniques
Dorjan Hitaj, Luigi V. Mancini
TL;DR
Owners need ways to detect theft of costly neural-network models, and watermarking is one proposed approach. This paper presents evasion attacks against DNN watermarking and reports that an ensemble attack allowed owners to verify at most 25.5% of a model’s watermark on average in experiments with seven models.
Problem
The cost of building high-quality neural networks motivates protection against model theft, while owners need a way to detect unauthorized use of their models.
Source: §I · pp. 1–2Method
The paper presents two evasion attacks against DNN watermark verification, including a detector that identifies possible watermark triggers and returns random predictions for them.
Source: A. Attack Overview · p. 5, §V · p. 6Results
25.5% of its watermark was the most an owner could verify on average across seven models in the ensemble experiment.
Source: C. Ensemble experiments · p. 4Takeaways & Limitations
The experiments demonstrate that current DNN watermarking techniques can be evaded, allowing stolen models to be used without detection by their owners.
Source: §V · p. 6
Abstract
from arXiv · showhide
Deep neural networks have had enormous impact on various domains of computer science, considerably outperforming previous state of the art machine learning techniques. To achieve this performance, neural networks need large quantities of data and huge computational resources, which heavily increases their construction costs. The increased cost of building a good deep neural network model gives rise to a need for protecting this investment from potential copyright infringements. Legitimate owners of a machine learning model want to be able to reliably track and detect a malicious adversary that tries to steal the intellectual property related to the model. Recently, this problem was tackled by introducing in deep neural networks the concept of watermarking, which allows a legitimate owner to embed some secret information(watermark) in a given model. The watermark allows the legitimate owner to detect copyright infringements of his model. This paper focuses on verifying the robustness and reliability of state-of- the-art deep neural network watermarking schemes. We show that, a malicious adversary, even in scenarios where the watermark is difficult to remove, can still evade the verification by the legitimate owners, thus avoiding the detection of model theft.
I. INTRODUCTION
Building high-quality neural networks requires substantial data, computation, and expense, making model theft a threat to owners’ business interests. The paper presents two evasion attacks that let an adversary use stolen models without detection by watermark verification.
I. INTRODUCTION
- Building a high-quality model is costly because it requires large quantities of data, human effort to assemble datasets, and substantial GPU computation.Finding a suitable architecture and parameters can also consume time training models that perform poorly.Source: §I · p. 1
- Stolen models can expose proprietary training information, enable unlimited predictions without usage fees, or support a competing prediction service.Companies that sell models also risk customers reselling them or insiders leaking them.Source: §I · p. 1
- Watermarks embed secret information in a model so owners can test suspected models for unauthorized copying or redistribution.The paper frames watermarking as a way to detect copyright violations.Source: §I · pp. 1–2
- Two novel evasion attacks let an adversary run a service using stolen proprietary models while avoiding detection by their legitimate owners.The attacks target current state-of-the-art DNN watermarking techniques.Source: §I · p. 2
II. BACKGROUND AND RELATED WORK
The section introduces neural-network backdoors and explains their intended behavior and how training-set poisoning can implant them. It also describes their potential for harm and their use in a model-watermarking approach.
A. Backdoors in Neural Networks
- The section introduces backdoors as a distinct neural-network concept alongside their traditional description as trap doors.It distinguishes neural-network backdoors from the traditional computer-security usage.Source: A. Backdoors in Neural Networks · p. 2
- A neural-network backdoor makes one or more trigger instances map to a preset target label, while the model continues classifying clean instances.The trigger instances and target class are added to training through training-set poisoning.Source: A. Backdoors in Neural Networks · p. 2
- Backdoors are presented as a security risk, but the section also notes their use in a technique for watermarking neural-network ownership.The cited example describes a trigger placed over a speed-limit sign in an image classifier for a self-driving vehicle.Source: A. Backdoors in Neural Networks · p. 2
B. Watermarking Neural Networks via Backdooring
The section describes a backdoor-based watermark that uses a distinctive trigger set and verifies ownership by querying those triggers. It then identifies limits on public verification and asks whether verification can be evaded even when the watermark remains.
B. Watermarking Neural Networks via Backdooring
- The watermark uses distinct trigger instances, each assigned a random class, and adds them to the model’s training alongside the original training set.The instances are chosen to be sufficiently different so that a leaked portion does not reveal the rest of the trigger set.Source: B. Watermarking Neural Networks via… · p. 2
- Ownership verification queries the model with watermark triggers and accepts a claim when at least an ϵ-sized fraction of the trigger set receives the expected outputs.The tolerance parameter accounts for changes such as fine-tuning or parameter clipping.Source: B. Watermarking Neural Networks via… · p. 2
- Public verification is limited because an adversary who obtains the trigger set after verification can fine-tune the model to remove the watermark.Repeated public verification requires multiple embedded watermarks, constrained by the maximum number of backdoors a network can hold.Source: B. Watermarking Neural Networks via… · p. 3
- The described watermark is robust to removal and overwriting, yet the paper asks whether its verification can still be evaded in a black-box setting.The section says fine-tuning and embedding a new watermark do not necessarily remove the original watermark or let an adversary claim ownership.Source: B. Watermarking Neural Networks via… · pp. 2–3
III. ENSEMBLE ATTACK
The ensemble attack combines stolen models that perform the same task and returns the class receiving the most votes. Because models assign outputs even to unfamiliar inputs, their combined predictions can interfere with watermark verification while preserving service quality on regular inputs.
A. Attack Overview
- Figure 1 presents the ensemble setup, in which an API layer queries the participating models and collects their responses.The API layer returns the class with the most votes.Source: Figure 1 · p. 3, A. Attack Overview · p. 3
III. ENSEMBLE ATTACK
- A voting ensemble of stolen models predicts the class receiving the most votes, reducing the portion of any one model’s watermark that can be verified as the number of models increases.If no class receives a majority, the service returns a random class.Source: A. Attack Overview · p. 3, §III · p. 3
A. Attack Overview
- Neural networks still output a class for unfamiliar inputs, which the paper models as a random event over the task’s classes.The section illustrates this with a handwritten-digit classifier receiving an abstract image that contains no number.Source: A. Attack Overview · p. 3
- The attack is described as preserving service quality on regular inputs because the stolen models are high quality.The section also notes that ensembles of good models can be better predictors.Source: A. Attack Overview · p. 3
B. Ensemble set-up
The ensemble attack combines stolen or purchased models behind a programmatic API that returns a prediction by voting across the models.
B. Ensemble set-up
- Models can be stolen from MLaaS providers or bought more cheaply on the DarkWeb.The paper cites Tram`er et al as an example of a model-stealing attack.Source: B. Ensemble set-up · p. 3
- The API queries each model and returns the class with the most votes.If there is no majority, it randomly selects among the predicted classes.Source: B. Ensemble set-up · p. 4
- Users, including legitimate model owners, have only black-box access through the API.They can submit queries but cannot inspect the ensemble models.Source: B. Ensemble set-up · p. 4
C. Ensemble experiments
The ensemble evades watermark verification across the tested settings: owners verify only a portion of the watermark, including in larger-output tasks and with smaller ensembles.
C. Ensemble experiments
- The seven models were MNIST digit classifiers, each with a 10-instance watermark and over 99% test accuracy.Each model classified all watermark triggers correctly; trigger sets were assumed not to intersect.Source: C. Ensemble experiments · p. 4
- At most 25.5% of each model's watermark was verified on average in the seven-model experiment.Figure 2 reports the percentage verified for each model in the seven-model ensemble.Source: Figure 2 · p. 4, C. Ensemble experiments · p. 4
- Close to 1/7 or around 15% of each watermark was verified for ImageNet models in a seven-model ensemble.The passage notes that ImageNet has an output space 100 times bigger than MNIST.Source: C. Ensemble experiments · p. 4
- 34% of the watermark was verified on average with three models, and the smaller ensemble lowers the attacker's cost.The watermark verification was still successfully evaded.Source: C. Ensemble experiments · p. 4
- The attack relies on neural networks' uncertain predictions for unseen inputs far from their training distribution.The authors report that owners cannot decide with high confidence whether their model was stolen and included in the adversary's service.Source: C. Ensemble experiments · p. 4
IV. DETECTOR ATTACK
The detector attack aims to evade watermark verification while lowering the adversary's costs by reusing one stolen high-quality model to build a detection mechanism.
IV. DETECTOR ATTACK
- The detector attack seeks both to evade watermark verification and to reduce the adversary's attack costs.Figure 3 is titled as depicting watermark evasion via the detector mechanism.Source: §IV · p. 4, Figure 3 · p. 5
- The adversary needs to steal only one high-quality model for this attack.The detector mechanism is built using that same stolen model.Source: §IV · p. 4
- Reusing the stolen model is intended to make construction of the detection mechanism minimally costly.Source: §IV · p. 4
A. Attack Overview
The attack routes suspicious watermark-like queries away from the stolen model, using a detector trained from features extracted by that model.
A. Attack Overview
- The service checks each query with a binary detector, forwarding clean instances and returning a random class for suspected triggers.The random response is drawn from the stolen model's output labels.Source: A. Attack Overview · p. 5
B. Detector Build-up
- The detector's balanced training set pairs clean images with abstract images, under an assumption of access to similar clean examples.Figure 4 shows a clean image on the left and an abstract image on the right.Source: B. Detector Build-up · p. 5, Figure 4 · p. 5
- Features extracted by passing dataset images through the stolen network train a fully connected softmax classifier.The output layer is removed for feature extraction; the classifier may be single- or multi-layer.Source: B. Detector Build-up · p. 5
C. Experiments
The experiments evaluate detector-based watermark evasion using image-classification models and report detector accuracy above 90% across the tested stolen-model backbones. The detector is trained quickly, and the authors also consider non-abstract trigger images.
C. Experiments
- Less than 2 seconds per epoch on a commodity laptop was required to train the detector, whose test accuracy exceeded 90%.The detector has three fully connected layers with 512, 256, and 2 neurons; the authors state that fewer than 10% of watermark triggers would bypass it.Source: C. Experiments · pp. 5–6
- The experiments also consider non-abstract, computer-tweaked real-image triggers and classify images using features extracted from a pretrained model.The image is preprocessed, passed through the pretrained model to obtain a feature vector, then classified by the detector.Source: C. Experiments · p. 6
- Above 90% detector accuracy was reported across the tested stolen-model backbones, supporting evasion of watermark verification.The evaluated backbones include ResNet, InceptionV3, Xception, VGG16, and VGG19; the detector uses a 3-layer fully connected architecture.Source: Table I · p. 6, C. Experiments · p. 6
V. CONCLUSION
The conclusion presents two evasion attacks: an ensemble voting service and a detector-based service. Both aim to evade watermark verification while retaining prediction quality for clean inputs.
V. CONCLUSION
- The paper concludes that current deep-neural-network watermarking techniques are susceptible to evasion attacks.The two attacks target techniques presented in.Source: §V · p. 6
- Around 30% of a watermark was verifiable for ensemble models with an output space of cardinality 10.The ensemble combines stolen models trained for the same task; watermark triggers unique to one model often fail to receive a majority vote.Source: §V · p. 6
- The ensemble attack is described as improving clean-instance predictions over using one predictor, while the detector attack retains the stolen model's high quality.The paper attributes the ensemble comparison to prior research on ensembles of good models.Source: §V · p. 6
- For the detector-based attack, the owner has probability 1/l of verifying each watermark trigger when the output space has cardinality l.The detector identifies most possible watermark instances, and the service returns a random class prediction for them.Source: §V · p. 6
VI. FUTURE WORK
Future work includes improving the detector and investigating neural-network backdoors. The authors report preliminary evidence of differing activation patterns for backdoored and legitimate instances.
VI. FUTURE WORK
- The authors plan to improve the detector mechanism, possibly using Generative Adversarial Networks.This is presented as future work rather than a reported evaluation.Source: §VI · p. 6
- They also intend to investigate detecting and removing backdoors in neural networks.Source: §VI · p. 6
- Preliminary work indicates that backdoored instances exhibit different activation patterns, motivating a possible real-time classifier for queried instances.The proposed classifier would distinguish possible backdoors from legitimate instances.Source: §VI · p. 6