Source-linked AI summary
Watermarking Images in Self-Supervised Latent Spaces
Pierre Fernandez, Alexandre Sablayrolles, Teddy Furon, Hervé Jégou, Matthijs Douze
TL;DR
Watermarking must hide messages while preserving imperceptibility, payload, and robustness under image distortions. This paper embeds marks and binary messages in pre-trained self-supervised latent spaces, using augmentation during marking. The method improves zero-bit performance over a baseline and achieves multi-bit results comparable to state-of-the-art deep watermarking, while remaining computationally expensive because embedding is not a single forward pass.
Problem
Watermarking must balance imperceptibility, payload, and robustness when the decoder lacks the original image.
Method
The method uses pre-trained self-supervised networks as latent embedding spaces and applies gradient-based marking with data augmentation.
Results
The method improves zero-bit performance over a baseline, remains robust across transformations, and achieves multi-bit results comparable to state-of-the-art deep data hiding.
Takeaways & Limitations
Self-supervised networks naturally provide excellent watermarking spaces for robust zero-bit and multi-bit image watermarking.
Takeaways & Limitations
Compared with encoder-decoder techniques, the method is expensive because watermarking is not a single forward pass.
Abstract
from arXiv · showhide
We revisit watermarking techniques based on pre-trained deep networks, in the light of self-supervised approaches. We present a way to embed both marks and binary messages into their latent spaces, leveraging data augmentation at marking time. Our method can operate at any resolution and creates watermarks robust to a broad range of transformations (rotations, crops, JPEG, contrast, etc). It significantly outperforms the previous zero-bit methods, and its performance on multi-bit watermarking is on par with state-of-the-art encoder-decoder architectures trained end-to-end for watermarking. The code is available at github.com/facebookresearch/ssl_watermarking
1. INTRODUCTION
The paper adapts watermarking to pre-trained self-supervised latent spaces, using augmentation during marking to improve robustness while supporting both marks and binary messages.
- Watermarking must jointly satisfy imperceptibility, payload, and robustness, with blind decoding performed without the original image.
- Self-supervised networks provide embedding spaces designed to be invariant to augmentations without requiring explicit image semantics.
- The method uses pre-trained networks as embedding spaces, avoiding the heavy training required by end-to-end watermarking architectures.
- Gradient descent over image pixels embeds watermarks while data augmentation and preprocessing at marking time improve robustness and imperceptibility.
- The algorithm encodes both marks and binary messages in latent spaces of pre-trained networks.
- The paper experimentally finds that self-supervised networks provide excellent embedding spaces.
2. RELATED WORK
Prior watermarking methods use transform, supervised-neural, or encoder-decoder embedding spaces, while this work extends supervised-feature watermarking with self-supervision and marking-time augmentation.
- Traditional watermarking commonly embeds information in transform spaces such as DFT, DCT, or DWT, while zero-bit detection checks for a hidden mark.
- Encoder-decoder methods jointly train networks to embed and decode messages, often simulating perturbations or optimizing against attacks.
- These encoder-decoder methods can make embedding distortion difficult to control and are designed more for decoding than detection.
- Earlier supervised-feature methods push image features into a hypercone and use augmentation during pre-training to improve robustness.
3. WATERMARKING WITH SSL NETWORKS
The method uses self-supervised feature extractors as watermarking spaces, combining secret-key regions, augmentation during marking, and low-distortion image optimization for zero-bit and multi-bit watermarking.
- Approach: The method adopts a supervised-network watermarking framework, replacing its feature representation with self-supervised networks and adding augmentation during marking.It also extends the framework to multi-bit watermarking.
- Feature space: The feature extractor should remain stable under geometric and valuemetric transformations while remaining alterable through invisible image perturbations.These properties define the desired marking space.
- Feature space: DINO is selected because self-supervised learning trains augmentation-invariant features without the semantic collapse associated with supervised classification.DINO uses teacher-student self-distillation on different views of the same image.
- Feature space: PCA-whitening transforms the output features into centered vectors with unit covariance so watermark embeddings can reach arbitrary secret-key regions.The normalization addresses feature concentration away from the target region.
- Marking: Marking minimizes a watermark loss on augmented images together with an image loss favoring low distortion, producing a visually similar watermarked image.The watermark loss pushes transformed features into the target region, while the image loss controls perceptual change.
- Marking: Stochastic gradient descent samples augmentations during optimization, and admissible perturbations are shaped by SSIM attenuation and a target PSNR.The final image is the rounded result after K iterations.
- Detection and decoding: The system supports zero-bit detection with a secret-key hypercone and multi-bit decoding using orthogonal carriers whose projection signs encode the message.Zero-bit detection has an author-stated false-positive-rate formulation; multi-bit marking uses a hinge loss with margin.
4. EXPERIMENTS & RESULTS
Experiments evaluate zero-bit and multi-bit watermarking across datasets, resolutions, and image attacks. The method shows strong robustness, near-invisible embedding, and competitive multi-bit accuracy while scaling to higher resolutions.
- Evaluation setup: Experiments use YFCC, CLIC, and MS-COCO, covering varied content, professional high-resolution images, and smaller images for state-of-the-art comparisons.The evaluation includes 1,000 YFCC images, 118 CLIC images, and 1,000 MS-COCO images.
- Evaluation setup: The method uses a self-supervised ResNet-50 backbone, Adam-based pixel optimization, SSIM attenuation, PSNR control, and randomized transformations during marking.Transformations include identity, rotation, blur, crop, and resize.
- Zero-bit watermarking: At FPR=10−6, the method produces no false positives on 100k YFCC images and one false positive among 1,281,167 ImageNet images.Higher FPR widens the detection region and improves robustness against severe attacks, but increases false positives.
- Zero-bit watermarking: Rotation augmentation is needed during both pre-training and marking to achieve high robustness against rotation.The ablation evaluates True Positive Rate at PSNR=40 dB and FPR=10−6.
- Qualitative results: The watermark is almost invisible at PSNR=40 dB because SSIM normalization places the signal in textured regions.Figure 4 shows an 800×600 image, detected alterations, and the scaled watermark amplitude.
- State-of-the-art comparison: Against prior zero-bit methods, the approach improves especially on large rotations, crops, and Gaussian blur, with almost perfect detection on 118 CLIC images.On CLIC, higher image resolutions provide more pixels for conveying the mark.
- Multi-bit data hiding: For 30-bit watermarking, decoding achieves low BER and WER across geometric and valuemetric attacks, while rotation and Gaussian blur are particularly harmless.Images with little texture are harder to watermark, and error-correcting codes can reduce WER further.
- Multi-bit data hiding: Multi-bit performance is comparable to state-of-the-art deep data hiding, except for center crops, while JPEG robustness is achieved without specific training.The method also scales to higher-resolution images and achieves lower BER for a fixed payload.
5. CONCLUSION & DISCUSSION
The method robustly and invisibly embeds information in latent spaces, improving zero-bit performance over a baseline and achieving competitive multi-bit results. Its main limitation is computational expense because marking is not a single forward pass.
- Contributions: The method embeds information into off-the-shelf self-supervised latent spaces while maintaining robustness and invisibility.It incorporates data augmentation and constraints during marking.
- Results: Zero-bit watermarking greatly improves performance over the baseline and remains robust to a wide range of transformations with high fidelity and a very low false positive rate.
- Results: Multi-bit watermarking achieves results comparable to state-of-the-art deep data hiding and performs better under some transformations, including JPEG compression and blur.
- Limitation: Compared with encoder-decoder deep watermarking techniques, the method is expensive because marking is not a single forward pass.The authors suggest that adapting the network specifically for watermarking could improve performance and efficiency.
Appendix
The appendix derives the distribution used for a spherical random-vector calculation and illustrates watermarking at low and high image resolutions. It connects the sampling construction to a Fisher-distribution expression and a tail probability formula.
- Distribution derivation: A uniform spherical vector U can be sampled by normalizing a white Gaussian vector G drawn from N(0, I) in R^d.The construction is U = G/∥G∥.
- Distribution derivation: Y follows a Fisher distribution F(1, d −1), whose cumulative density function is expressed using the regularized incomplete beta function.
- Distribution derivation: The tail probability P(|U^T a| > cos θ) is rewritten as a regularized incomplete beta function involving sin^2 θ and parameters (d −1)/2 and 1/2.
- Watermarked-image demonstrations: The demonstrations include 128 × 128 watermarked images and images at approximately 2048 × 1536 resolution.The low-resolution example uses a 30-bit payload and target PSNR values of 52dB, 40dB, and 32dB.