What Breaks Local Watermarks? A Robustness Benchmark for Local Invisible Image Watermarking

1 Introduction

• The benchmark evaluates five localization-related methods across 55 transformations and four datasets, including signal distortions, alignment changes, indirect local edits, and direct watermark edits.The local edits include splicing, inpainting, and outpainting, which are underrepresented in prior evaluations despite their relevance to local watermarking.Source: §1 · pp. 1–2
• Evaluation covers payload recovery, clean-setting image fidelity, and localization accuracy, and tests each method with and without SyncSeal synchronization.SyncSeal adds a signal intended to correct geometric transformations and restore the coordinate frame expected by the decoder.Source: §1 · p. 2
• No evaluated method is robust across all transformations; MaskWM has the strongest clean payload recovery and localization but the lowest clean image fidelity.Geometric misalignment sharply reduces payload recovery, and generative local edits can impair recovery even without overlapping the watermark region.Source: §1 · p. 2
• Because the benchmark transformations are non-adaptive, a scheme’s failure under one implies a practical black-box watermark-removal attack.The authors conclude that the evaluated schemes are unlikely to provide provenance in realistic copyright scenarios with adversaries motivated to remove watermarks.Source: §1 · p. 2

2.1 Local watermarking

• Local watermarking embeds a payload in selected image regions to provide provenance for individual objects, unlike global watermarking, which spreads its signal across the full image.WAM predicts watermark presence and payload bits per pixel, while MaskWM uses mask-guided training and a localization module to identify watermarked regions.Source: §2.1 · p. 2
• Tamper-localization methods estimate where an image was manipulated; OmniGuard uses separate payloads for copyright protection and localization, designed to respond differently to local edits.Local edits can selectively corrupt its localization payload while the copyright payload is designed to remain recoverable.Source: §2.1 · p. 2

2.2 Robustness evaluations

• Prior local-watermark robustness evidence is limited: WAM’s local evaluation lacks additional distortions, while MaskWM’s evaluation uses a controlled setup and a limited set of conventional transformations.The MaskWM evaluation covers twelve region-size intervals, ten distortions, and three geometric transformations, but not diverse realistic transformations such as generative and local edits.Source: §2.2 · p. 3

3.1 Overview

• Each run embeds a method-specific binary payload in a mask, applies one transformation, then blindly decodes; with SyncSeal, rectification precedes decoding.The same payload is used across transformations with and without SyncSeal.Source: §3.1 · p. 3, §3.2 · p. 3

3 Evaluation Framework

• The framework benchmarks five localization-related watermarking methods across 55 transformations and four datasets, measuring payload recovery, localization, and clean-setting image fidelity.Source: §3.2 · p. 3, §3.4 · p. 4, §3.5 · pp. 4–5

3.2 Localization and synchronization

• MaskWM and WAM are native local methods; OmniGuard, TrustMark, and PixelSeal are adapted from global watermarking, while SyncSeal is a separate optional synchronization module.Source: §3.2 · p. 3

3.3 Image datasets and watermark masks

• COCO and SA-1B use supplied segmentation masks; DIV2K and MIRFLICKR use generated rectangular or irregular masks, with one region per image.Source: §3.3 · pp. 3–4

3.4 Transformation taxonomy

• The 55 transformations span signal distortions, alignment changes, indirect local edits, and direct watermark edits, with distinct protocols for inpainting and outpainting.Indirect edits sample without reference to the watermark mask; direct edits use the watermark region as the edit mask and replacement target.Source: §3.4 · p. 4

3.5 Evaluation metrics

• Payload recovery uses bit accuracy and a payload-length-aware evidence score; score 3 corresponds to random decoding probability 10^-3.The evidence score is −log10 of the probability that random decoding would produce at least the observed number of correct bits.Source: §3.5 · pp. 4–5
• Localization uses IoU after thresholding predictions at 0.5 and applying coordinate mapping, and is comparable only for MaskWM and WAM; clean-setting fidelity uses PSNR, SSIM, and LPIPS.Fidelity is measured only in the clean setting to isolate distortion introduced by watermark embedding.Source: §3.5 · p. 5

4.1 Payload recovery

• In the clean setting, MaskWM has the strongest recovery, while PixelSeal’s evidence score exceeds WAM’s; MaskWM also has the lowest clean-setting image fidelity.MaskWM’s bit accuracy is 0.973 ± 0.018 and evidence score is 17.08 ± 1.37; PixelSeal’s evidence score is 6.73, compared with WAM’s 3.39 ± 0.35.Source: §4.1 · pp. 5–7
• MaskWM alone stays above the recovery threshold across all five signal-distortion subcategories, while alignment effects vary and SyncSeal helps some cases but not vertical flip or expanded-canvas outpainting.For MaskWM, SyncSeal raises the mean evidence score from 5.14 to 10.91 for geometric transformations and from 0.48 to 12.41 for diagonal flip.Source: Figure 2 · p. 6, §4.1 · p. 7
• Blackout and splicing preserve recovery for MaskWM and PixelSeal, but inpainting and outpainting leave every method below threshold; direct watermark inpainting and splicing drive bit accuracy toward random.Restoring the pixels outside an inpainting edit mask brings mean evidence scores above threshold for MaskWM, PixelSeal, and WAM.Source: §4.1 · p. 7

4.3 Imperceptibility

• MaskWM combines the strongest clean-setting recovery and localization with the largest embedding distortion; SyncSeal adds a further fidelity cost.Mean clean-setting PSNR is 49.48 dB for MaskWM, versus 62.55 for WAM, 56.80 for PixelSeal, 54.09 for TrustMark, and 51.36 for OmniGuard; SyncSeal lowers MaskWM’s mean PSNR by 7.17 dB.Source: §4.3 · pp. 7–8

4.2 Localization accuracy

• Localization degrades most under alignment changes, and it can remain informative when payload decoding is near random, so the two outcomes capture distinct failure modes.Under vertical flip, MaskWM’s localization IoU remains high while bit accuracy stays near random; localization also persists under direct inpainting and splicing.Source: §4.2 · p. 7, Figure 3 · p. 8

4.4 Performance by watermark-region area

• Larger watermark regions are associated with higher payload-recovery evidence scores and better localization, with WAM’s post-transformation localization remaining below MaskWM’s.On COCO, MaskWM’s post-transformation evidence score rises from 3.84 in the 0–3% area bin to 9.71 in the 40–85% bin; its IoU rises from 0.634 to 0.825, while WAM rises from 0.052 to 0.359.Source: §4.4 · p. 8, Figure 4 · p. 9

4.5 A deeper analysis of generative edits

• For MaskWM and WAM, evidence scores remain close to zero across five overlap bins (<10% to ≥70%) with less overlap; increased edit–watermark distance does not restore MaskWM recovery.MaskWM’s evidence score is 0.451 when the edit touches the watermark region and 0.346 across non-touching distance bins.Source: §4.5 · p. 9
• Restoring pixels outside an inpainting mask recovers payload, while MaskWM recovery declines as preserved watermark-region edge density rises.MaskWM’s evidence score remains high across overlap bins after outside-pixel restoration, but falls from 1.53 in the smoothest quintile to 0.36 in the highest-edge quintile.Source: §4.5 · p. 9

SyncSeal

• Figure 6 plots SyncSeal's change in payload-length-aware evidence score for alignment transformations; positive changes indicate stronger payload recovery.Rows are grouped into affine and non-affine blocks, with horizontal bars showing sample standard deviation across four dataset-level changes.Source: Figure 6 · p. 10

Small regions

• Smaller regions perform poorly even without transformations, motivating designs that scale payload bits with region area or embed signals across multiple spatial resolutions.The authors identify tracing small objects in composited imagery as a key motivation for local watermarking.Source: §5 · pp. 10–11

Payload recovery and fidelity

• MaskWM has the strongest clean payload recovery but the lowest clean image fidelity, and SyncSeal further improves recovery for several transformations at an additional fidelity cost.The authors suggest varying added-signal strength to find where recovery gains no longer justify further distortion.Source: §5 · p. 11

Generative editing

• Inpainting can reduce payload recovery without visible quality loss in the watermarked region, while both outpainting transformations reduce recovery to random guessing.The authors identify generative edits as a serious robustness threat.Source: §5 · p. 11

Security implications

• Several benchmark transformations substantially reduce payload recovery, sometimes to random guessing; when applied deliberately, they become non-adaptive black-box removal attacks.The authors recommend evaluating adaptive attacks that may use method-design knowledge or decoder outputs.Source: §5 · p. 11

Conclusion

• Within the evaluated scope, none of the current methods is ready for reliable production deployment because recovery is weak on small regions, trades off against fidelity, and remains vulnerable to transformations including generative edits.The authors conclude that these challenges must be addressed before reliable deployment.Source: §5 · p. 11

Images

• Each method is evaluated on 200 fixed images per dataset, for 800 images per method; inputs are resized to 512 × 512 pixels before embedding.Watermark masks are generated in the same coordinate frame as the resized images.Source: §A.1 · p. 13

Masks

• COCO and SA-1B masks are sampled from segmentation masks covering at least 2% of the image, while DIV2K and MIRFLICKR use content-independent masks.Sampled target areas are 4%–14%; TrustMark uses rectangular masks, including the smallest enclosing rectangle for selected COCO and SA-1B masks.Source: §A.1 · p. 13

Transformations

• Each stochastic transformation is applied once per image using a fixed random seed.Examples include random cropping, Gaussian noise, and blackout applied to a randomly sampled region.Source: §A.1 · p. 13

Reproducibility

• The benchmark uses released pretrained weights for all watermarking methods and SyncSeal without further training, and fixes seeds for transformations, masks, and payload generation.The benchmark code is stated to be available at the cited repository.Source: §A.1 · p. 13

Reporting convention

• Unless stated otherwise, results are reported without SyncSeal, and dataset means are averaged equally to produce the overall mean.SyncSeal results and direct comparisons are labeled explicitly.Source: §A.1 · p. 13

Area distributions

• Mean watermark-region areas are 15.22% for COCO, 12.50% for SA-1B, and 8.80% for both DIV2K and MIRFLICKR in clean MaskWM examples without SyncSeal.COCO and SA-1B use native masks; DIV2K and MIRFLICKR use sampled masks.Source: Figure 7 · p. 13

Transformation protocols

• The examples show edit masks, transformed images, and absolute differences for inpainting and outpainting protocols.Outside-restored controls restore pixels outside the edit mask; fixed-canvas outpainting regenerates surrounding content, while expanded-canvas outpainting extends and rescales the canvas.Source: Figure 8 · p. 13

• Table 3 presents a taxonomy of 55 transformations, with the clean setting listed separately.Source: Table 3 · p. 13

• OmniGuard, TrustMark, and PixelSeal are adapted to embed within selected regions using released pretrained weights without additional training or fine-tuning.The watermark residual is the pixelwise difference between an image before and after embedding.Source: §A.5 · p. 14
• OmniGuard keeps its auxiliary signal across the image while restricting its binary payload signal to the selected region.Payload decoding resizes the complete received image to 256 × 256.Source: §A.5 · p. 14
• TrustMark adapts the selected mask to its smallest enclosing axis-aligned bounding box, then uses detected regions for decoding.The adaptation uses the Q variant with a 100-bit payload and error correction disabled.Source: §A.5 · p. 14
• PixelSeal confines its embedding changes to the selected region by masking the residual from its globally watermarked output.Unlike OmniGuard, it retains no separate auxiliary watermark outside the region and decodes with a whole-image decoder.Source: §A.5 · p. 14
• For all three methods, the selected mask is used only during embedding; decoding receives neither the ground-truth mask nor its bounding box.Source: §A.5 · p. 14

A.6 Transformation Parameters

• Table 4 defines the clean reference and 55 transformation conditions, sampling one predefined parameter configuration per image and condition with fixed random seeds.Brightness and Gaussian-blur intervals are uniform-sampling ranges, not exhaustive sweeps applied to every image.Source: Table 4 · p. 14, §A.6 · p. 14
• Alignment conditions include translation, grid warping, flips, row or column removal, a JPEG-crop-brightness chain, and expanded-canvas outpainting.Ground-truth masks undergo the same coordinate changes for the listed alignment transforms.Source: §A.6 · p. 15
• Signal conditions vary image intensity, color, filtering, noise, compression, quantization, and spatial or frequency-domain content.The listed conditions include DCT, DWT, and FFT coefficient perturbations.Source: §A.6 · pp. 15–16
• Local-edit conditions include independently sampled blackout, inpainting, and splicing, as well as watermark-region inpainting and splicing and outpainting.For independent-region splicing, pixels outside the edit are restored from the watermarked input; watermark-region edits target the embedded region.Source: §A.6 · p. 16

Use of Generative AI

• Generative AI assisted with grammar checking, language polishing, and coding; the authors reviewed the suggested edits and remain responsible for the manuscript.Source: Use of Generative AI · p. 16