PaperlayerView exact revision on arXiv

Source-linked AI summary

Where Does the Watermark Hide? Push-Pull Disentanglement for Invisible Watermark Removal

Jidong Yang, Huaike Yu, Qi Li, Chunpeng Wang, Yuantian Miao, Suo Gao, Xiao Chen

arXiv:2609.31722v1cs.CV

TL;DR

Fixed distortions do not fully characterize attackers trained on aligned clean and watermarked images. This paper trains a paired-supervised push–pull model for single-image deployment without the clean reference or watermark decoder. Its selected operating point and auxiliary-input intervention support decoder dependence in the evaluated setting, while the mechanism theory remains post-hoc and conditional.

  • Problem

    Fixed-distortion evaluations do not fully characterize an attacker that learns from aligned clean and watermarked images and later processes one image at a time.

    Source: §1 · p. 1

  • Method

    A paired-trained encoder-decoder uses a full auxiliary path to reconstruct the watermarked image and a zero-auxiliary path trained toward its paired clean image.

    Source: §4.1 · p. 3, §4.2 · p. 4

  • Results

    At k = 1.10, u = 0, the four-method sweep gives average BER 0.3958, PSNR 31.07 dB, and SSIM 0.9554.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Restoring u moves recovery away from chance, supporting decoder dependence on the auxiliary input in this model.

    Source: §9 · p. 10

    The finite (k, u) grid and one run cannot establish a continuous optimum or repeatability across seeds.

    Source: §8 · p. 10

Abstract

from arXiv · show

Fixed image distortions do not cover an attacker that learns from paired clean and watermarked images. We study this paired-training threat with single-image inference: deployment uses neither the clean reference nor the watermark key, payload, or decoder. An encoder maps each image to a structural latent $g$ and an auxiliary residual latent $u$. Push supervision reconstructs the watermarked image from $D(g_w,u_w)$. Pull supervision trains the zero-auxiliary output $D(A_g(g_w;k),0)$ toward the paired clean image. At $k=1.10,u=0$, the four-method sweep gives an average BER of $0.3958$, PSNR of $31.07$ dB, and SSIM of $0.9554$. Restoring $u$ from $0$ to $0.15$ moves average BER from $0.3893$ to $0.3357$, while PSNR falls from $30.99$ to $28.23$ dB. The intervention supports decoder dependence on the auxiliary input in the evaluated setting. The accompanying theory is a conditional, post-hoc account of this behavior rather than an experimentally verified information-relocation result.

1 Introduction

The paper targets paired-trained attackers that learn from aligned clean and watermarked images but remove watermarks from a single image at deployment. It introduces opposing push–pull reconstruction paths and reports a high-fidelity operating point, with an auxiliary-input intervention supporting decoder dependence in the evaluated setting.

1 Introduction

  • Aligned clean–watermarked pairs expose image-specific watermark changes and can train a removal model that later processes one image at a time.Prior evaluations cover compression, filtering, geometric edits, and generative reconstruction,,, but do not fully characterize this paired-training threat.Source: §1 · p. 1
  • The encoder separates each image into a one-channel structural latent g and a 16-channel auxiliary latent u; the push path D(gw, uw) reconstructs the watermarked input.The pull path zeroes u and uses Ag(gw; k) as the structural input, with paired clean targets supervising that output.Source: §1 · pp. 1–2
  • Training uses paired clean and watermarked images, while deployment uses only the watermarked image and no clean reference, key, payload, or decoder.The introduction frames this as a paired-training, single-image-inference threat model.Source: §1 · p. 2
  • At k = 1.10, u = 0, the four-method sweep gives average BER 0.3958, PSNR 31.07 dB, and SSIM 0.9554.The selected operating point is reported for the four-method sweep.Source: §1 · p. 2
  • Restoring u from 0 to 0.15 moves average BER from 0.3893 to 0.3357, while PSNR falls from 30.99 to 28.23 dB.The intervention supports decoder dependence on the auxiliary input without establishing that u exclusively contains watermark information.Source: §1 · p. 2
  • The paper evaluates seven watermarking methods and presents the selected four-method sweep point alongside preliminary evidence from the u intervention.Full scans and per-method results appear in the supplementary material.Source: §1 · p. 2

2 Related Work

Watermarking approaches expose different recovery interfaces, so removal evaluations distinguish payload decoding from detector decisions and report image fidelity alongside attack success. Learned removal includes fixed-distortion, regeneration, model-side, and paired-supervised approaches; this paper distinguishes its approach through an explicit G/U intervention rather than claiming paired supervision as new.

2.1 Watermarking and robustness evaluation

  • Multi-bit systems return a payload, whereas detector systems return a score or decision, so removal evaluation should preserve this distinction and report fidelity beside attack success.Watermarks may be embedded through transform-domain, neural, or generative approaches,,,,, [17].Source: §2.1 · p. 2

2.2 Learned watermark removal

  • The paired-supervised First-Place Solution trained on 1,000 StegaStamp pairs and removed watermarks from a single test image.Other learned approaches include fixed distortions, regeneration, controllable reconstruction, saliency-aware reconstruction, and model-side fine-tuning [21].Source: §2.2 · p. 2
  • Unlike the paired-supervised precedent, this paper trains one decoder to reconstruct xw with uw and approach paired xc when u is zeroed.A latent-frequency path then perturbs the remaining structural input; the distinction is a mechanism and representation distinction, not a priority claim.Source: §2.2 · p. 2

3 Threat Model

The threat model separates paired training from single-image deployment and distinguishes deployment inputs from evaluator access. It defines family-level seen and unseen groups and evaluates removal under both fidelity and watermark-specific metrics.

3 Threat Model

  • The attacker learns a single-image mapping from paired clean and watermarked images, while deployment excludes the clean reference, key, payload, and watermark decoder or detector.Offline evaluation uses the relevant decoder or detector to measure removal and select the reported (k, u) point; the deployed mapping is query-free, not the full selection procedure.Source: §3 · p. 2
  • Training includes DwtDctSvd, RivaGAN, StegaStamp, and Tree-Ring; DwtDct, SSL Watermarking, and Stable Signature are excluded families.“Unseen” means family exclusion only, not new image distributions, payload lengths, keys, resolutions, or implementations.Source: §3 · p. 2
  • The capability table separates attacker access by phase, including evaluator access versus deployment inputs.The caption identifies the phase distinction but does not specify further table encodings.Source: Table 1 · p. 3
  • For bit decoders, removal is stronger as BER approaches 0.5; Tree-Ring is evaluated separately with lower TPR at nominal 1%-FPR calibration preferred.Acc Removal is 1 − TPR, while PSNR and SSIM are measured against xw; the objective is high removal subject to visible fidelity.Source: §3 · p. 3

4 Paired Push–Pull Attack

The attack separates each image into structural and auxiliary latents, then trains paired reconstruction paths with opposing targets. At inference, a frequency intervention and zero auxiliary input produce the clean-target path from a watermarked image alone.

4.1 Latent representation and inference path

  • The one-channel structural latent g and 16-channel auxiliary latent u are full-resolution decoder inputs, but their channel asymmetry does not prove a factorization.The full path uses both latents; the deployed path uses the attacked structural latent with the auxiliary input set to zero.Source: §4.1 · pp. 3–4

4.2 Push and pull supervision

  • Push reconstructs the watermarked image from the joint code, while pull trains the attacked zero-auxiliary path toward its paired clean target.Paired supervision retains the corresponding scene, pose, color layout, and texture rather than using an unrelated clean reference.Source: §4.2 · p. 4
  • Clean reconstruction, perceptual, edge, and high-frequency terms constrain degenerate solutions, but they do not certify latent factorization.The staged schedule introduces the attacked zero-auxiliary pull target after initial clean and full-path reconstruction.Source: §4.2 · p. 4, Table 2 · p. 5

4.3 Latent-frequency operator and mechanism test

  • The frequency operator modifies structural latent g through low-, middle-, and high-frequency bands before decoding; the Tree-Ring ring notch is reserved for its separate radius experiment.A multiband magnitude loss aligns the attacked latent with the clean structural latent but does not constrain phase or imply pixel equality.Source: §4.3 · p. 5, Figure 3 · p. 5
  • The auxiliary-input intervention tests decoder sensitivity; the expansion isolates that sensitivity, while watermark-specific evidence comes from BER response rather than the expansion itself.The conditional-information interpretation and its premises are unverified.Source: §4.3 · p. 5

5 Experimental Setup

The experiment selects a shared operating point through coarse and fine sweeps, then evaluates transfer and baseline comparisons. Baseline rows differ in generation settings and available subsets, so those comparisons are not an identical-protocol benchmark.

5 Experimental Setup

  • The shared fine sweep covers DwtDct, DwtDctSvd, RivaGAN, and SSL Watermarking, with Table 3 presenting the candidate settings.Candidates require average PSNR of at least 25 dB and are ranked by average RR.Source: §5 · p. 5, Table 3 · p. 6
  • The selected k = 1.10, u = 0 point is applied across methods; Tree-Ring's radius scan is reported separately.The sweep uses 100 samples per method and attack setting; evaluation at the selected point adds StegaStamp, Stable Signature, and Tree-Ring.Source: §5 · pp. 5–6
  • Baseline comparisons are operating-point comparisons, not an identical-protocol benchmark, because generation settings and available subsets differ across rows.Baselines include distortions, reconstruction and regeneration attacks, and model-side fine-tuning.Source: §5 · p. 6

6 Results

The selected settings balance watermark removal and image fidelity, while the auxiliary-input intervention supports decoder branch dependence in the tested setting. Results vary across watermark families and baselines, and the ablations show tradeoffs rather than uniform component effects.

6.1 Selected operating point

  • At k = 1.10, u = 0, the selected point reaches average BER 0.3958 and PSNR 31.07 dB; changing k from 0.90 to 1.10 moves BER from 0.3867 to 0.3958.At k = 0.90, restoring u = 0.03 raises PSNR by 0.19 dB but moves BER to 0.3720.Source: §6.1 · p. 6

6.2 Auxiliary-input intervention

  • Average BER moves from 0.3893 at u = 0 to 0.3357 at u = 0.15, while PSNR falls from 30.99 to 28.23 dB.This supports decoder dependence on the auxiliary input in the evaluated setting, but does not locate the information or show that g is information-free.Source: §6.2 · p. 6

6.3 Seen and unseen families

  • At k = 1.10, u = 0, BER ranges from 0.4856 for DwtDctSvd to 0.2275 for Stable Signature, showing method-dependent transfer to excluded families.DwtDct, SSL Watermarking, and Stable Signature are excluded families; Tree-Ring is reported separately because its detector output is not a bit payload.Source: §6.3 · pp. 6–7, Table 5 · p. 7

6.4 Removal and fidelity relative to baselines

  • Our selected point gives mean RR 0.7464 and mean SSIM 0.9436, while SADRE-Cheng gives mean RR 0.7692 and mean SSIM 0.8792; CtrlRegen's mean PSNR is 21.28 dB.The comparison describes a measured fidelity–removal profile, not uniform dominance.Source: Table 6 · p. 7, §6.4 · p. 7
  • CtrlRegen and WatermarkAttacker achieve higher mean RR than the selected point at 21.28 and 24.13 dB PSNR, while regeneration generally has lower fidelity and JPEG leaves most learned payloads recoverable.The comparison separates removal from visible damage; Stable Signature remains a weak case.Source: Table 6 · p. 7, §6.4 · p. 7
  • Figure 4's absolute and signed residuals show broad changes for regeneration and diffusion methods, while the proposed output has weaker residuals on DwtDctSvd and StegaStamp.These are qualitative diagnostics, not measurements of watermark information or proof of frequency localization.Source: §6.4 · p. 7, Figure 4 · p. 8
  • A Tree-Ring ring notch at r = 0.50 raises Acc Removal from 0.3200 to 0.3900 as PSNR falls from 26.81 to 25.47 dB.Larger radii recover fidelity but lose the removal gain; these detector results are kept separate from BER averages.Source: §6.4 · p. 7, Table 7 · p. 8

6.5 Ablation

  • Removing the multiband magnitude loss changes mean BER by −0.044 and raises PSNR by 1.19 dB; removing scheduled drop0 changes mean BER by +0.038 and lowers PSNR by 3.37 dB.The drop0 variant retains Lrecov, so these are schedule and weight ablations, not a clean test of zero-auxiliary supervision; the spectral association is not proof of sole causation.Source: §6.5 · pp. 7–9, Table 8 · p. 9

7 Discussion

The discussion frames the attack as paired supervision followed by a single-image, zero-auxiliary intervention, and distinguishes that contribution from prior paired removal. Its interpretation is branch dependence under tested conditions, not demonstrated relocation of watermark information.

7 Discussion

  • Paired supervision supplies an image-aligned clean target: push reconstructs xw from D(gw, uw), while pull trains D(Ag(gw; k), 0) toward xc.Zeroing u selects the trained pull path; restoring u tests how much decoder recovery returns.Source: §7 · p. 9
  • Restoring a fraction of uw changes BER more sharply than changing k across u = 0 settings, supporting decoder response dependence on the auxiliary input.The experiment does not determine whether that response reflects payload-specific evidence, watermark-correlated texture, or another residual feature.Source: §7 · p. 9
  • The attack occupies a distinct removal–fidelity region from aggressive regeneration and high-fidelity but weak distortions, without evidence of universal superiority.CtrlRegen has higher mean RR but 21.28 dB mean PSNR; SADRE-Cheng has slightly higher mean RR at similar mean PSNR, while the proposed attack has higher SSIM.Source: §7 · p. 9
  • A stronger causal study would report all four combinations of attacked or unperturbed g and restored or zeroed u for every watermark family.This separates the direct branch intervention from the complete attack, which also applies a controlled structural perturbation.Source: §7 · p. 9
  • Deployment is query-free, but training uses paired data and operating-point selection uses evaluator decoding; neither is supplied when attacking one image.Transfer testing is limited to excluded families; broader evaluation should vary family, implementation, payload, key, and image distribution independently.Source: §7 · pp. 9–10

8 Limitations

The evaluation is bounded by finite samples, a finite parameter grid, and non-identical baseline generation settings. Several metrics and mechanism tests also leave important aspects of fidelity, stability, and representation behavior unresolved.

8 Limitations

  • The four-method sweep and seven-method selected-point evaluation use 100 samples per method, and the finite grid cannot establish a continuous optimum or repeatability across seeds.The authors call for repeated training seeds and per-image confidence intervals before making a stability claim.Source: §8 · p. 10
  • Baseline outputs come from different generation settings and subsets, so Table 6 is an operating-point comparison rather than a controlled benchmark.A controlled benchmark would regenerate attacks from a shared image manifest, preserve per-image pairing, report failure counts, and tune baselines under the same quality constraint.Source: §8 · p. 10
  • PSNR and SSIM are measured against the watermarked input and do not fully capture semantic or perceptual preservation.The authors suggest LPIPS or another learned perceptual metric and protocol-defined human inspection as complementary assessments.Source: §8 · p. 10
  • The mechanism account is post-hoc: mutual information is unestimated, g is not a proven bottleneck, and the u-probe result comes from one split.The authors recommend AUROC, repeated cross-validation, uncertainty and label-permutation tests, plus varying g channel count and quantization strength.Source: §8 · p. 10
  • The ablations summarize one run per variant without uncertainty, and duplicate Stage 3 residuals complicate interpreting the intended objective components.Reproducibility requires the data manifest, family split, seeds, configurations, evaluation commands, per-image scores, and checkpoints.Source: §8 · p. 10

9 Conclusion

The paper studies paired-training watermark removal with single-image inference and reports a fidelity-constrained operating point. Its explanation of latent branch dependence remains conditional and post-hoc, while diagnostic corners illustrate a removal–fidelity trade-off.

9 Conclusion

  • At k = 1.10, u = 0, the four-method sweep gives average BER 0.3958 and PSNR 31.07 dB; restoring u moves recovery away from chance in this model.The authors present this as support for decoder dependence on the auxiliary input, not exclusive storage of watermark information.Source: §9 · p. 10

A.1 Fidelity-constrained removal risk

  • The fidelity-constrained risk framing makes the attack class explicit; robustness to a hand-designed distortion subset does not bound risk over a larger class containing paired learned attacks.The passage cites [31], [32] for the broader attack-class point.Source: §A.1 · p. 13

A.2 Conditional-information interpretation

  • The information identity partitions payload information between g and u but does not determine either term; the proposed concentration account requires additional premises.The paper describes Equations (11)–(13) as a conditional, post-hoc explanation because the objective does not optimize mutual information and the experiment does not estimate εI.Source: §A.2 · p. 13

A.3 Constrained push–pull view

  • The push–pull constraints permit u to help reconstruct xw while the zero-auxiliary output follows xc, but this is not the unique solution or proof of disentanglement.The proposed checks include restoring u, removing spectral alignment, and removing zero-auxiliary supervision; only the first is addressed by the current residual-scale experiment, while the third remains unresolved.Source: §A.3 · p. 13

B.1 Coarse and fine grids

  • K1U0 is the only diagnostic corner combining high removal with PSNR above 30 dB; Table 9 reports the four corners.K0U0 is closest to BER 0.5 but has PSNR 22.51 dB; restoring the full auxiliary input at K0U1 yields PSNR 37.80 dB and recoverable payload.Source: §B.1 · p. 13, Table 9 · p. 14

B.2 Per-method results

The supplied passages describe per-method tables, representation probes, and baseline comparisons, but provide few numeric outcomes for these sections. The probe evidence is preliminary, and baseline generation settings and result subsets differ.

B.2 Per-method results

  • Table 12 presents per-method fine-sweep metrics, with BER judged by closeness to 0.5 and PSNR/SSIM by larger values; the supplied passage gives no cell results.The figure passage gives full PSNR and BER views across k and u, with BER read by proximity to 0.5.Source: Table 12 · p. 14, Figure 5 · p. 15

B.3 Tree-Ring radius and representation probes

  • On 399 matched representations, a single train/test linear probe reaches 98.75% accuracy using u, while the g probe gives 21.25%; neither establishes exclusive information storage.The authors call the u result preliminary and warn that the below-chance g result cannot be read as no information; the Tree-Ring scan table reports points recomputed from scan files.Source: §B.3 · pp. 14–15, Table 13 · p. 15

C Complete Baseline Comparisons

  • Tables 14 and 15 compare conventional, denoising, reconstruction, and regeneration attacks with the selected point, but attack-generation configurations and result subsets are not identical across rows.The outputs use method-specific decoders or detectors and common fidelity metrics.Source: C Complete Baseline Comparisons · p. 16, Table 14 · p. 16, Table 15 · p. 16

D Full Ablation Results

The appendices document ablation variants, residual visualizations, architecture, training losses and schedule, and evaluation choices. The supplied passages specify many settings but do not provide ablation cell outcomes or full training-schedule values.

D Full Ablation Results

  • The listed ablations remove or change multiband magnitude alignment, drop0, pooled-VGG constraint, high-frequency residual input, or Fourier-block group count.The no-drop0 variant still retains Lrecov on the same zero-auxiliary prediction, so it is not a complete removal of zero-auxiliary supervision.Source: D Full Ablation Results · p. 17, Table 16 · p. 17, Table 17 · p. 17

E Full Double Residual Visualization

  • Figure 6 shows broader bidirectional residuals for regeneration and diffusion methods, weaker residuals for several bit-decoder methods, and broader signed changes for Tree-Ring.These images describe spatial modification patterns, not watermark information or a causal frequency mechanism.Source: E Full Double Residual Visualization · p. 18, Figure 6 · p. 18

F Architecture Details

  • The reported model uses 512 × 512 inputs, 16 auxiliary channels, an encoder with two output heads, and a full-resolution decoder that does not downsample.The encoder input combines the image and its high-frequency grayscale component; the decoder receives g, u, and H(g).Source: F Architecture Details · p. 19, Table 19 · p. 19

G Full Loss Weights

  • The Stage 3 objective duplicates pixel and pooled-VGG computations, producing effective weights 16 and 10 after their respective warmups.The authors recommend consolidating each pair before causal ablations; Table 20 lists loss weights for stages spanning epochs 0–14, 15–79, and 80–139.Source: G Full Loss Weights · p. 21, Table 20 · p. 21

H Training Schedule

  • Table 21 records epoch ranges and quantities that change at each transition, but the supplied passages do not specify those transition values.The loss-weight table identifies the three stage ranges as epochs 0–14, 15–79, and 80–139.Source: Table 20 · p. 21, H Training Schedule · p. 22, Table 21 · p. 22

I.1 Parameter search

  • The parameter search refines k over 0.80–1.20 and u over selected values from 0 to 0.20, choosing a point under visual-quality constraints rather than maximizing BER alone.The stated motivation is to avoid destructive attacks that cause detector failure while making outputs visibly unusable.Source: §I.1 · p. 23

I.2 Metric interpretation

  • For bit-decoder systems, BER closer to 0.5 indicates stronger removal, while Tree-Ring uses lower TPR at nominal 1%-FPR; PSNR and SSIM are measured against xw.The passage warns that BER above 0.5 is not automatically stronger and recommends reporting realized empirical FPR when it differs from the nominal target.Source: §I.2 · p. 23

Related papers

Loading 2609.31722v1…