Source-linked AI summary
A survey of deep neural network watermarking techniques
Yue Li, Hongxia Wang, Mauro Barni
TL;DR
Protecting DNN intellectual property matters because training demands costly data, computation, and expertise. The paper synthesizes DNN watermarking through a new taxonomy and representative algorithm review, concluding that the field has practical opportunities but unresolved robustness, security, and dynamic-watermarking questions.
Problem
DNN watermarking must adapt multimedia watermarking concepts to neural networks because DNNs differ in how watermarks are embedded and read.
Method
The paper introduces a taxonomy of DNN watermarking and reviews representative algorithms, requirements, attacks, challenges, and opportunities.
Results
The reviewed methods include schemes that preserve accuracy while demonstrating robustness to fine-tuning or pruning, alongside weaknesses against overwriting or other attacks.
Takeaways & Limitations
DNN watermarking offers a framework for protecting model intellectual property, but deployment requires attention to attacks and unresolved design questions.
Takeaways & Limitations
The survey identifies unresolved high-capacity multibit watermarking, deliberate-attack security, and the potentialities of dynamic watermarking.
Abstract
from arXiv · showhide
Protecting the Intellectual Property Rights (IPR) associated to Deep Neural Networks (DNNs) is a pressing need pushed by the high costs required to train such networks and the importance that DNNs are gaining in our society. Following its use for Multimedia (MM) IPR protection, digital watermarking has recently been considered as a mean to protect the IPR of DNNs. While DNN watermarking inherits some basic concepts and methods from MM watermarking, there are significant differences between the two application areas, calling for the adaptation of media watermarking techniques to the DNN scenario and the development of completely new methods. In this paper, we overview the most recent advances in DNN watermarking, by paying attention to cast it into the bulk of watermarking theory developed during the last two decades, while at the same time highlighting the new challenges and opportunities characterizing DNN watermarking. Rather than trying to present a comprehensive description of all the methods proposed so far, we introduce a new taxonomy of DNN watermarking and present a few exemplary methods belonging to each class. We hope that this paper will inspire new research in this exciting area and will help researchers to focus on the most innovative and challenging problems in the field.
1. Introduction
DNNs are valuable intellectual property because they require costly data, computation, and expertise to train. This survey adapts watermarking concepts to DNNs, introducing a taxonomy, reviewing representative algorithms, and highlighting practical challenges and opportunities.
- Motivation: Training high-performance DNNs requires massive often proprietary datasets, extensive computation, and expert topology and hyper-parameter tuning.These costs support treating high-performance DNNs as the model owner's intellectual property.
- Motivation: DNN watermarking adapts multimedia watermarking for protecting the intellectual property associated with deployed and commercialized neural networks.The paper frames DNN watermarking as an emerging application inspired by multimedia IPR protection.
- DNN-specific differences: Unlike multimedia watermarking, DNN watermark embedding is performed during training by modifying the loss function rather than directly changing model weights.Direct weight modification makes the watermark's effect on network performance difficult to estimate.
- DNN-specific differences: DNN watermarks may be read statically from network weights or dynamically from model outputs or intermediate activations for specific inputs.Dynamic watermarking therefore changes model behavior, whereas static watermarking reads information from the parameters.
- Survey goals: The paper introduces a taxonomy based on similarities and differences between multimedia and DNN watermarking, combining conventional classifications with new perspectives.It also interprets essential watermarking requirements across the different technique classes.
- Survey goals: The survey reviews popular and high-performing DNN watermarking algorithms to clarify the field's practical challenges and opportunities.It is selective rather than a comprehensive description of every proposed method.
2. A taxonomy of DNN watermarking
The paper develops a taxonomy of DNN watermarking techniques by combining established multimedia classification criteria with criteria arising from DNN-specific properties.
- Taxonomy construction: The taxonomy compares DNN watermarking with conventional multimedia watermarking to identify which established classification criteria apply to DNNs.It begins by reviewing the main criteria used to categorize classic watermarking algorithms.
- Taxonomy construction: The taxonomy also introduces unique classification criteria based on the peculiar characteristics of DNN watermarking.The paper discusses relationships among the resulting watermarking classes.
2.1. Classical watermarking models
Classical watermarking embeds messages into host content, often using transformations and secret keys, and organizes schemes by payload, recovery mode, and robustness. These concepts motivate corresponding distinctions for DNN watermarking.
- Embedding model: Classical watermark embedding optionally transforms host content, injects a keyed watermark, and returns the result to the original domain.Direct and inverse transforms are optional, while a secret key can parameterize embedding and recovery.
- Payload type: Multi-bit watermarking recovers an N-bit message, whereas zero-bit watermarking detects whether a known watermark is present.The distinction applies to DNN watermarking as well as multimedia watermarking.
- Payload type: Multi-bit schemes provide greater application flexibility, while zero-bit schemes generally offer higher robustness and suit copyright-presence detection.The cited applications include fingerprinting, source tracking, labeling, and copyright-protection platforms.
- Robustness: Robust watermarking survives intended or adversarial host manipulations, while fragile watermarking is altered by modifications and supports tamper detection.Robustness concerns non-malicious and malicious manipulations; fragility is mainly associated with data authentication.
- Robustness: DNN watermarking applications generally require robustness to operations such as fine-tuning and node pruning, with security against deliberate removal needed in some settings.The distinction between ordinary robustness and security reflects whether an adversary is actively attempting watermark removal.
- Recovery mode: Blind watermarking recovers the watermark without comparing marked and original content, whereas non-blind watermarking uses the original content during extraction.For DNNs, the notion of an original non-watermarked content does not directly apply.
- Informed watermarking: Informed coding associates each message with a pool of codewords and selects the codeword producing minimum distortion during informed embedding.This paradigm is connected to watermarking as channel coding with side information at the transmitter.
2.2. DNN watermarking models
DNN watermarking uses learned network parameters to encode information beyond the primary task and introduces white-box/black-box and static/dynamic recovery distinctions. These modalities differ according to whether extraction accesses internal model information, outputs, or behavior under specific inputs.
- DNN watermarking models: DNNs map inputs to outputs through a parameterized function whose parameters determine network behavior and can encode additional information.Training optimizes these parameters through a loss function, commonly using back-propagation.
- White-box vs. black-box: White-box watermark recovery accesses internal model parameters, including weights or neuron activations, to decode or detect the watermark.Multi-bit methods recover message bits, whereas zero-bit methods decide whether a specific watermark exists.
- White-box vs. black-box: Black-box watermarking accesses only model outputs by querying the DNN with properly chosen inputs, without access to its architecture or internal parameters.The queried inputs may function like decoding or detection keys when kept secret.
- Static vs. dynamic: Static watermarking embeds information in fixed network weights, while dynamic watermarking associates it with network behavior on specific triggering or key inputs.Dynamic watermark recovery can inspect final outputs or intermediate activations, depending on the method.
- Relationship among classes: Dynamic watermarking supports both white-box and black-box recovery, whereas static watermarking requires white-box access to internal parameters.White-box methods have supported multi-bit and zero-bit watermarking, while black-box methods are more often used for zero-bit watermarking.
3. Requirements
DNN watermarking must balance robustness, fidelity, and capacity while also addressing integrity, security, generality, and efficiency. Robustness concerns survival under fine-tuning and pruning, whereas fidelity concerns preserving task performance and integrity concerns reliable recovery or detection.
- Robustness: Robustness requires recovering the watermark after host-model modifications, especially fine-tuning and network pruning.Fine-tuning adapts a pretrained model to a new task, while pruning removes small-magnitude weights to simplify deployment without significantly reducing accuracy.
- Security: Security requires watermark removal to impose significant degradation on the host model, including resistance to overwriting and surrogate-model attacks.These attacks respectively add a competing watermark or train a functional replica from model queries and outputs.
- Fidelity: Fidelity requires the watermarked model to retain performance close to that of an otherwise similarly trained non-watermarked model.The paper measures fidelity using Test Error Rate (TER), and watermark training can sometimes reduce overfitting risk.
- Capacity: Capacity is the number of payload bits, applies to multi-bit watermarking, and conflicts directly with robustness as payload increases.Zero-bit watermarking conveys no payload.
- Integrity: For multi-bit watermarking, integrity is evaluated with Bit Error Rate (BER), ideally zero without processing or attacks; zero-bit methods instead face false-detection and missed-detection errors.Informed embedding can enable error-free recovery when the model is unmodified.
- Other requirements: Generality concerns applicability across architectures and tasks, while efficiency concerns avoiding unaffordable computational overhead during joint task and watermark training.Both requirements extend beyond the core robustness–fidelity–capacity trade-off.
4. Static Watermarking Algorithms
Static watermarking embeds messages directly into DNN weights, using training-time objectives and extraction from weight-derived representations. Exemplary methods illustrate trade-offs among payload, classification fidelity, robustness, and detectability.
- Static watermarking: Static watermarking embeds a message directly into network weights and later extracts it from those weights.The section presents static methods as distinct from behavior-based dynamic watermarking.
- Uchida et al.’s algorithm: Uchida et al. flatten averaged convolutional weights and add a regularization term that promotes correct watermark decoding.The training objective combines the original classification loss with a watermark regularizer controlled by λ; decoding projects weights onto pseudorandom spreading directions.
- Uchida et al.’s algorithm: Uchida et al.’s method preserves classification accuracy while supporting payloads larger than the number of selected-layer weights, although zero BER limits payload size.For convolutional levels 2, 3, and 4, 576, 1152, and 2304 weights support maximum zero-BER payloads of 1024, 2048, and 4096 bits, respectively.
- Uchida et al.’s algorithm: Uchida et al.’s watermark remains at zero BER after fine-tuning and is robust to pruning, but additional watermarking causes a large BER.The reported results use 20 fine-tuning epochs and describe overwriting with an additional 256-bit watermark as a weakness.
- ST-DM DNN watermarking: ST-DM replaces conventional spread-spectrum embedding with dither modulation, using a smooth approximation to enable back-propagation during training.The approximation’s smoothness and periodicity are controlled by α and β, and ST-DM generally provides larger capacity with similar robustness.
- Tartaglione et al.: A static zero-bit method by Tartaglione et al. uses a robustness-oriented loss and achieves robustness against fine-tuning and weight quantisation with negligible TER influence.The example uses 0.4% of model weights to convey the watermark.
5. Dynamic Watermarking
Dynamic watermarking encodes messages through a DNN’s behavior on selected inputs or internal activations, with methods spanning activation maps, outputs, and image-processing outputs. Exemplary approaches demonstrate fidelity, robustness, detection, and low false-positive performance, while payload or tuning constraints remain.
- Dynamic watermarking: Dynamic watermarking reads a message from model behavior on selected key-inputs, whose labels form key input-label pairs and whose task relation may be entangled or non-entangled.The watermark may be observed in intermediate activation maps or only at the network output.
- White-box activation-map watermarking: DeepSigns forces activation maps into class-specific Gaussian mixture distributions whose means encode watermark bits, using additional loss terms during training.The original loss and two added terms are balanced by λ1 and λ2; one term matches target distributions while the other separates class-associated Gaussians.
- White-box activation-map watermarking: 64 bits for MNIST and 128 bits for CIFAR 10-CNN and CIFAR 10-WRN are supported at zero BER by DeepSigns, alongside good fidelity and robustness.These payload limits are reported for the marked models and datasets evaluated in the survey.
- Natural key images: DAWN shows good watermark detection when surrogate models achieve good test performance and resists the PRADA surrogate attack.The reported evaluation varies surrogate architectures and considers epochs optimized either for watermark accuracy or test accuracy.
- Hand-crafted key images: Guo et al.’s invisible signatures mark training images with predefined labels, then fine-tuning teaches the DNN to recognize the marked behavior.The signature is formed by adding a pseudorandom bit sequence at random pixel locations, with strength selected by binary search.
- Image-processing networks: Dynamic watermarking also applies to image-processing networks, where the watermark appears as a signature in generated images rather than classification labels.The survey relates this setting to classical image watermarking because produced images carry the signature.
6. Specific Attacks against DNN Watermarking Algorithms
Specific attacks target watermark removal rather than ordinary DNN operations, including weight-based attacks, surrogate-model attacks, and attacks exploiting key-image selection. These results expose security challenges beyond robustness to fine-tuning and pruning.
- Attack landscape: Specific attacks deliberately remove watermarks while preserving the DNN, extending evaluation beyond routine operations such as fine-tuning and model pruning.The paper distinguishes deliberate watermark-removal attacks from common lifecycle manipulations that may incidentally affect watermarks.
- Static-watermark attacks: The variance-based attack relies on watermark regularization increasing projections of weights onto embedding-matrix rows, with standard deviation scaling approximately linearly with watermark dimension.The approximation invoking the law of large numbers is stated to hold when the matrix dimension N is sufficiently large.
- Static-watermark attacks: Weight-based attacks detect watermark presence by increased weight variance, estimate watermark length, and overwrite the existing watermark with a new one.This attack class applies specifically to static watermarking systems.
- Key-image vulnerabilities: Non-entangled key-images can be removed by fine-tuning on even a small amount of data, and backdoor-based watermarking is vulnerable to existing backdoor defenses.A learning-rate schedule favoring watermark forgetting and elastic weight consolidation can further improve fine-tuning attacks.
- Surrogate-model attacks: Surrogate-model attacks optimize query generation and training hyperparameters, including learning rate and training epochs, to improve watermark-removal effectiveness.The survey identifies these as the two main steps refined in stronger surrogate attacks.
7. Final remarks and suggestion for future research
The survey identifies unresolved challenges for DNN watermarking, including capacity, robustness, security, and the behavior of dynamic schemes. These challenges are expected to drive substantial future research because suitable protection against DNN misuse remains urgently needed.
- The survey reviews DNN watermarking categories while highlighting their advantages, drawbacks, and distinctive open challenges.The taxonomy and algorithm review are used to identify priorities for future work.
- High-capacity multibit watermarking with robustness against common DNN manipulations remains insufficiently addressed.Open questions include reliable capacity for a given model size and task, and differences between static and dynamic schemes.
- Robustness against fine tuning, pruning, transfer learning, and deliberate watermark-removal attacks remains a major research challenge.The survey notes that most existing watermarks are weak against fine tuning and overwriting, while transfer-learning robustness has not been proven.
- Dynamic watermarking raises unresolved questions about trigger count, post-manipulation behavior, and whether triggers should resemble task inputs.These choices must be clarified without impairing the network’s intended task capability.
- The identified challenges are expected to occupy researchers because suitable means to protect DNNs from misuse are urgently needed.