PaperlayerView exact revision on arXiv

Source-linked AI summary

Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advances

Shilin Lu, Zihan Zhou, Jiayou Lu, Yuanzhi Zhu, Adams Wai-Kin Kong

arXiv:2410.18775v2cs.CVcs.AIcs.CR

TL;DR

Large-scale generative image editing can disrupt embedded watermarks, while evidence about watermark robustness across editing types is limited. The paper introduces W-Bench and VINE, combining blurring-based surrogate attacks with an adapted SDXL-Turbo encoder. The authors report that VINE outperforms existing methods in image quality and robustness, while noting limits for image-to-video generation and increased resource requirements.

  • Problem

    Watermarking methods remain vulnerable to large-scale generative-model editing, and their robustness across varied editing techniques needs evaluation.

    Source: §1 · pp. 1–2

  • Method

    The paper benchmarks eleven watermarking methods in W-Bench and proposes VINE, using blur distortions as surrogate attacks and adapting pretrained SDXL-Turbo as a watermark encoder.

    Source: §1 · pp. 2–3

  • Results

    VINE achieves outstanding performance across image editing techniques, outperforming prior methods in image quality and robustness.

    Source: §5 · p. 10

  • Takeaways & Limitations

    The results suggest one-step pretrained models can serve as versatile watermarking backbones and that a generative prior can enhance quality and robustness.

    Source: §5 · p. 10

    VINE remains limited in image-to-video generation effectiveness and is larger than baselines, requiring more memory and slightly slower inference.

    Source: §5 · p. 10

Abstract

from arXiv · show

Current image watermarking methods are vulnerable to advanced image editing techniques enabled by large-scale text-to-image models. These models can distort embedded watermarks during editing, posing significant challenges to copyright protection. In this work, we introduce W-Bench, the first comprehensive benchmark designed to evaluate the robustness of watermarking methods against a wide range of image editing techniques, including image regeneration, global editing, local editing, and image-to-video generation. Through extensive evaluations of eleven representative watermarking methods against prevalent editing techniques, we demonstrate that most methods fail to detect watermarks after such edits. To address this limitation, we propose VINE, a watermarking method that significantly enhances robustness against various image editing techniques while maintaining high image quality. Our approach involves two key innovations: (1) we analyze the frequency characteristics of image editing and identify that blurring distortions exhibit similar frequency properties, which allows us to use them as surrogate attacks during training to bolster watermark robustness; (2) we leverage a large-scale pretrained diffusion model SDXL-Turbo, adapting it for the watermarking task to achieve more imperceptible and robust watermark embedding. Experimental results show that our method achieves outstanding watermarking performance under various image editing techniques, outperforming existing methods in both image quality and robustness. Code is available at https://github.com/Shilin-LU/VINE.

1 INTRODUCTION

The paper introduces W-Bench to assess watermarking across four editing settings and uses its findings to motivate VINE, which combines editing-inspired training distortions with a generative watermark encoder. Benchmark results show that most prior methods struggle after editing, while VINE targets robustness and image quality together.

1 INTRODUCTION

  • W-Bench evaluates eleven representative watermarking methods across image regeneration, global editing, local editing, and image-to-video generation.The benchmark covers seven distinct editing models and algorithms.Source: §1 · pp. 1–3
  • Most prior watermarking models struggle to extract watermarks after editing; StegaStamp and MBRS retain them in some cases at the expense of image quality.Figure 1(b) summarizes performance across editing tasks.Source: §1 · p. 2, Figure 1 · p. 2
  • VINE uses blurring distortions as surrogate attacks because editing and blur share a tendency to remove high-frequency patterns while leaving low-frequency patterns less affected.The authors report that adding blurring distortions to training noise layers enhances robustness to editing.Source: §1 · pp. 2–3
  • VINE adapts pretrained SDXL-Turbo as a generative watermark encoder to improve perceptual image quality and robustness.The encoder conditions on the input image and watermark.Source: §1 · pp. 2–3

2 RELATED WORK

Prior watermarking benchmarks and methods cover only parts of the challenge posed by generative-model editing. The paper positions W-Bench as a broader evaluation and discusses recent robustness methods alongside their constraints.

Watermarking benchmark

  • W-Bench expands beyond WAVES by covering global and local editing and image-to-video generation as well as regeneration, and evaluates methods intended for any image type.WAVES covers only regeneration and evaluates three methods, two of which are limited to generated images.Source: §2 · p. 3

Robust watermarking

  • Recent editing-robust methods include EditGuard, Robust-Wide, and JigMark, but the paper notes practical constraints involving detector inputs, message decoding, generalization, and code availability.EditGuard may require the unedited watermarked image; JigMark does not decode free-form messages, while neither JigMark nor Robust-Wide is open-sourced.Source: §2 · p. 3

3 METHOD

VINE treats watermarking as embedding and recovering a message despite image editing, and develops surrogate attacks and a generative encoder to improve robustness and image quality. Its training combines image-quality and extraction objectives with staged optimization and resolution scaling.

3 METHOD

  • VINE embeds a watermark into an input image and trains a decoder to recover it even after editing; the method uses frequency analysis and a one-step text-to-image encoder.The method section is organized around surrogate attacks, adapting the encoder, and training strategy.Source: §3 · pp. 3–4

3.1 FREQUENCY NATURE OF IMAGE EDITING

  • Image editing typically removes mid- and high-frequency patterns while leaving low-frequency patterns relatively unaffected, motivating low-frequency watermark embedding.The analysis inserts patterns into frequency bands and compares edited watermarked images with edited originals.Source: §3.1 · p. 4
  • Pixelation and blur distortions resemble editing in the frequency domain, so VINE uses efficient blurring attacks during training; JPEG compression and saturation do not show the same pattern.The noise layer also includes common distortions to counter transmission-related degradation.Source: §3.1 · pp. 4–5, Figure 3 · p. 5

3.2 GENERATIVE PRIOR FOR WATERMARK ENCODING

  • VINE adapts one-step SDXL-Turbo as a conditional generative encoder, fusing the image and watermark before the VAE and adding skip connections to improve perceptual similarity.The VAE reconstruction capability is important for keeping the watermarked image perceptually close to the input.Source: §3.2 · pp. 5–6, Figure 4 · p. 6

3.3 OBJECTIVE FUNCTION AND TRAINING STRATEGY

  • Training balances watermarked-image quality and extraction, initially prioritizing extraction at α=10 and βMSE=βLPIPS=βGAN=0.01 while SDXL-Turbo UNet, VAE decoder, and added zero-convolution layers are frozen; above 0.85 bit accuracy, all parameters unfreeze.The first two stages produce VINE-B, which is fine-tuned in a third stage by incorporating Instruct-Pix2Pix into the noise layer; gradients are backpropagated through a straight-through estimator.Source: §3.3 · p. 6

4 EXPERIMENTS

The experiments benchmark watermarking across image-editing tasks, compare VINE with existing methods, and ablate its design. VINE-B and VINE-R offer different image-quality and robustness trade-offs, while performance varies by editing task.

4.1 IMAGE EDITING METHODS

  • The evaluation covers stochastic and deterministic image regeneration, prompt-guided global editing, mask-guided local editing, and image-to-video generation.The experiments use VP stochastic regeneration, DPM-solver deterministic regeneration, three global-editing models, two local-editing models, and SVD for video generation.Source: §4.1 · p. 7

4.3 BENCHMARKING RESULTS AND ANALYSIS

  • VINE-B offers superior image quality, while VINE-R delivers greater robustness at some image-quality cost; together they provide the best trade-off.The comparison covers watermarked-image quality and detection accuracy across editing methods at 512 × 512 resolution.Source: Table 1 · p. 8, §4.3 · p. 9

4.2 EXPERIMENTAL SETUP

  • VINE is trained on OpenImage at 256 × 256 and evaluated on 10,000 randomly sampled UltraEdit instances; it is compared with eleven watermark baselines, focusing primarily on TPR@0.1%FPR and TPR@1%FPR.Source: §4.2 · p. 8

4.3 BENCHMARKING RESULTS AND ANALYSIS

  • VINE-R and MBRS maintain high TPR@0.1%FPR across regeneration difficulty levels, while VINE-B, StegaStamp, and PIMoG perform well under minor perturbations.SSL has higher bit accuracy and AUROC than TrustMark but lower TPR@0.1%FPR; its high false-positive rate shows why those metrics alone are insufficient.Source: §4.3 · p. 9
  • VINE-B, VINE-R, and StegaStamp achieve high TPR@0.1%FPR under global and local editing, but methods effective against local UltraEdit editing fare poorly against global editing with that model.The authors suggest global editing more significantly degrades watermarks.Source: §4.3 · pp. 9–10
  • Image-to-video generation has low overall detection, with nearly all watermark patterns reduced across frequency bands; the authors infer that layout changes from object or camera motion contribute.Figure 5 presents results for stochastic regeneration, global editing, and local editing; additional results are in Figure 18.Source: Figure 5 · p. 9, §4.3 · p. 10

4.4 ABLATION STUDY

  • Ablations show blurring boosts robustness at image-quality cost; the condition adaptor restores quality, skip connections improve both, and Instruct-Pix2Pix fine-tuning trades quality for robustness.Compared with Config C, Config G's larger model and generative prior significantly improve image quality with modest robustness gains; random initialization lowers image quality without changing robustness.Source: §4.4 · p. 10, Table 2 · p. 10

5 CONCLUSION

W-Bench evaluates watermark robustness across four generative-model-based editing types, and VINE outperforms prior methods in image quality and robustness. The method remains limited on image-to-video generation and requires more memory and slightly slower inference than baseline models.

5 CONCLUSION

  • VINE achieves outstanding performance against image editing, outperforming prior methods in both image quality and robustness.The conclusion also suggests one-step pretrained models can serve as versatile watermarking backbones, with generative priors supporting less visible, robust embedding.Source: §5 · p. 10
  • VINE remains limited on image-to-video generation and is larger than baseline models, requiring more memory and slightly slower inference.The authors describe its performance against common generative-model-powered editing tasks as exceptional.Source: §5 · p. 10

Appendix

  • Figure 6 compares Tree Ring, RingID, and Gaussian Shading on real images after each image is inverted into noise using 50 steps before watermark injection.The appendix describes this as an inversion-based route to embedding in real images.Source: Figure 6 · p. 19

A.1 IN-GENERATION IMAGE WATERMARKING

  • In-generation watermarking alters image generation or fine-tunes generative models, but the benchmark excludes it because it applies to generated images rather than real ones.Applying these methods to real images via inversion produced images that differed significantly from the originals, undermining photographers’ intent to protect their work.Source: §A.1 · p. 19

A.2 GENERATIVE MODEL-BASED IMAGE EDITING

  • The benchmark covers image regeneration, global editing, and local editing, while excluding drag-based editing because of lengthy per-image optimization and limited datasets with valid drag points.The paper identifies extending the benchmark to drag-based methods as future research.Source: §A.2 · p. 20

B WATERMARKING FREQUENCY PATTERNS

Frequency analyses link watermark robustness to patterns concentrated in low-frequency bands, while editing affects some watermark patterns less than others. The authors also present the frequency-based observations that informed their design separately from a robustness correlation they describe as a byproduct.

B WATERMARKING FREQUENCY PATTERNS

  • The analysis averages absolute Fourier-spectrum differences over 1,000 watermarked and original image pairs, and the authors say the positive correlation between low-frequency pattern intensity and editing robustness emerged as a byproduct of training.The authors base their design on the observed frequency similarity between editing and blurring, and on treating watermarking as conditional generation with a generative prior.Source: B WATERMARKING FREQUENCY PATTERNS · pp. 20–22
  • StegaStamp, MBRS, SepMark, and PIMoG show prominent low-frequency patterns with a cross-shaped high-intensity pattern, while CIN and TrustMark have relatively low robustness.The authors infer that the cross pattern and its intensity may be related to robustness; StegaStamp has the strongest pattern intensity.Source: B WATERMARKING FREQUENCY PATTERNS · p. 21
  • Watermarks with impacts concentrated in low-frequency bands are more robust to editing; VINE-R’s brighter central pattern is associated with greater robustness than VINE-B.The authors infer that VINE’s dense patterns near the axes contribute to enhanced robustness.Source: B WATERMARKING FREQUENCY PATTERNS · p. 21
  • VINE-R, VINE-B, MBRS, and StegaStamp have frequency patterns less affected by editing than their original patterns, but band-specific effects are difficult to isolate visually.Figure 8 displays the absolute difference between the edited-watermarked and edited-original Fourier spectra.Source: Figure 8 · p. 21, B WATERMARKING FREQUENCY PATTERNS · p. 22

C STATISTICAL TEST

The statistical test evaluates watermark detection by thresholding matching bits, with false- and true-positive rates based on matching probabilities under the null and watermark-present hypotheses. The appendix also describes resolution scaling and reports that most methods show slight improvements in several image-quality metrics after scaling.

C STATISTICAL TEST

  • Detection flags an image when at least τ of the k extracted bits match the ground-truth watermark, testing watermark-present H1 against null H0.Under H0, the matching-bit count is modeled as binomial when extracted bits are i.i.d. Bernoulli with matching probability po; under H1, pw is the corresponding matching probability.Source: C STATISTICAL TEST · p. 22
  • High bit accuracy or AUROC alone does not ensure high TPR at low FPR; the supplied example gives TPR 0.99 at FPR 1.6 × 10^-5 for pw = 0.8 and po = 0.5.With pw = 0.9 and po = 0.7, matching the same FPR requires threshold τ = 87 and yields TPR 0.8.Source: C STATISTICAL TEST · pp. 22–23

D.1 METHOD

  • Resolution scaling normalizes an image, resizes it to the encoder’s training resolution, scales the encoder residual back to the input dimensions, and adds it to the original image.The algorithm describes the resulting watermarked image as its output.Source: §D.1 · p. 23

D.2 IMPACT ON WATERMARKED IMAGE QUALITY

  • Most methods show slight PSNR, SSIM, and FID improvements after resolution scaling, alongside a slight LPIPS increase.Table 3 compares image quality and detection accuracy under normal conditions at the methods’ original training resolutions.Source: §D.2 · p. 23, Table 3 · p. 23

D.3 IMPACT ON ROBUSTNESS AGAINST DISTORTIONS

Resolution scaling does not compromise the evaluated methods’ robustness and can sometimes improve it, with JPEG robustness increasing while several other distortion responses remain stable. Robustness to Gaussian blur also aligns with robustness to image editing, and the appendix describes the watermark encoder’s condition adaptor architecture.

D.3 IMPACT ON ROBUSTNESS AGAINST DISTORTIONS

  • At 512×512, several methods withstand Gaussian blur with kernel size 5, while that kernel has a greater impact at their original 128×128 or 256×256 resolutions.The authors explain that the same distortion affects larger images comparatively less; Figure 10 uses a halved kernel-size range relative to Figure 9.Source: §D.3 · pp. 23–24
  • Resolution scaling does not compromise robustness and can sometimes enhance it; it significantly improves JPEG robustness while maintaining robustness to brightness, contrast, and Gaussian noise.The authors note that comparisons at original training resolutions are somewhat unfair because the methods were trained at different resolutions.Source: §D.3 · p. 24
  • Methods that are robust to Gaussian blur also demonstrate robustness to image editing, consistent with the authors’ finding that editing and blurring have similar frequency characteristics.The comparison is described at 512×512 resolution.Source: §D.3 · p. 24

E CONDITION ADAPTER ARCHITECTURE

  • The watermark encoder’s condition adaptor consists of multiple fully connected and convolutional layers, each followed by ReLU activation.The architecture is shown in Figure 11.Source: E CONDITION ADAPTER ARCHITECTURE · p. 24

F.1 PRE-TRAINING

VINE-B is trained in two stages, and VINE-R is then fine-tuned on Instruct-Pix2Pix. The benchmark rationale covers global and local edits as well as regeneration, which can threaten watermarks; VINE detects watermarks even under very challenging regeneration.

F.1 PRE-TRAINING

  • VINE-B training uses two stages: first prioritizing extraction while key SDXL-Turbo components remain frozen, then unfreezing all parameters once bit accuracy exceeds 0.85.The second stage changes the loss weights; VINE-R is subsequently fine-tuned on Instruct-Pix2Pix using edited images and straight-through gradient estimation.Source: §F.1 · p. 24, §F.2 · p. 24

F.2 FINE-TUNING

  • Figures 9 and 10 assess methods under blur and common image distortions, with training resolutions varying by method; Figure 11 depicts the condition adaptor architecture.The figure captions list Gaussian blur, brightness, contrast, Gaussian noise, and JPEG compression as tested conditions.Source: Figure 9 · p. 25, Figure 10 · p. 26, Figure 11 · p. 27

G.1 RATIONALE FOR IMAGE EDITING SELECTION

  • The benchmark treats image regeneration and image-to-video generation as copyright threats because they can create similar digital content while removing watermarks.Global editing changes most pixels while preserving overall layout or semantics; local editing targets specific elements, semantics, or regions.Source: §G.1 · p. 27

G.2 IMAGE REGENERATION

  • VINE embeds and detects watermarks even at extremely challenging regeneration levels, although harder regeneration degrades reconstruction quality.Stochastic regeneration introduces more hallucinated details as difficulty rises, while deterministic regeneration smooths images.Source: §G.2 · p. 27

G.3 GLOBAL EDITING AND LOCAL EDITING

The global- and local-editing evaluation compares editing quality across models and examines whether watermarks alter editing outcomes. It also reports that watermarks do not perceptually affect image-to-video generation.

G.3 GLOBAL EDITING AND LOCAL EDITING

  • Global editing quality is assessed with CLIPdir, CLIPout, and CLIPimg, capturing prompt correspondence, target-caption alignment, and content preservation, respectively.The reported comparisons use image guidance scale 1.5 and text guidance scale 7.Source: §G.3 · pp. 27–28, Table 4 · p. 29
  • UltraEdit aligns global edits most effectively with prompts, while MagicBrush best preserves image layout; watermarks leave editing quality largely intact with minor fluctuations.For local edits, UltraEdit also excels in text-image alignment and content preservation.Source: §G.3 · p. 28, Table 4 · p. 29, Table 5 · p. 29

G.4 IMAGE-TO-VIDEO GENERATION

  • Watermarks do not perceptually affect image-to-video generation; detection accuracy is evaluated across 1,000 videos using TPR@0.1%FPR.The videos are generated from 1,000 watermarked CATER-GEN-v2 images, with detection assessed on frames 2, 4, 6, 8, and 10.Source: Table 6 · p. 29, §G.4 · p. 30

H INFERENCE SPEED AND GPU MEMORY EVALUATION

The evaluation compares watermarking methods by inference time and GPU memory, and qualitatively examines image and residual artifacts. The reported trade-off is that VINE uses more memory and time, but within what the authors call acceptable ranges.

H INFERENCE SPEED AND GPU MEMORY EVALUATION

  • VINE requires more GPU memory and longer inference time, though the authors describe both requirements as remaining within acceptable ranges.Inference speed and memory were measured on an NVIDIA Quadro RTX6000 GPU and averaged over 1,000 images.Source: H INFERENCE SPEED AND GPU MEMORY… · p. 30, Table 7 · p. 30
  • DWTDCT, DWTDCTSVD, and RivaGAN are omitted from the runtime and memory comparison because the employed implementations support CPUs exclusively.Table 7 compares running time per image and GPU memory usage.Source: Table 7 · p. 30

I QUALITATIVE COMPARISON

  • In qualitative examples of 512×512 images, MBRS and PIMoG watermarked images exhibit slight color distortions, StegaStamp images display black artifacts, while other methods appear identical to originals to the human eye.Source: I QUALITATIVE COMPARISON · p. 30

J ADDITIONAL BENCHMARKING RESULTS

Additional benchmark figures cover regeneration, global and local editing, and image-to-video generation. Their captions report only minor or imperceptible effects of different watermarks on editing and video-generation outcomes.

J ADDITIONAL BENCHMARKING RESULTS

  • Different watermarks have minimal impact on global and local editing outcomes, causing only slight changes, and have little effect on image-to-video generation.Figures 14–16 illustrate these outcomes; Figure 18 covers additional tests with DPM-Solver, UltraEdit, Instruct-Pix2Pix, ControlNet-Inpainting, and Stable Video Diffusion.Source: J ADDITIONAL BENCHMARKING RESULTS · p. 30, Figure 14 · p. 31, Figure 15 · p. 32
  • Figure 17 presents qualitative comparisons using magnified 40 × 40 central regions from watermarked images and residuals.The examples include global-edit prompts about snow and fireworks and local-edit prompts about changing pizza to a burger or adding flowers.Source: J ADDITIONAL BENCHMARKING RESULTS · pp. 31–32, Figure 17 · p. 34

Related papers

Loading 2410.18775v2…