Disentangle and Drop: Robust Universal Removal of Image Watermarks via Reconstructive Grayscale Residual Decomposition

• The attack is designed for a setting where the remover lacks the watermark method, decoder, key, and payload, unlike watermark-specific adaptive attacks.Source: §1 · p. 2
• The simple drop operation is inspectable and controllable, but can be brittle under spectral filtering or diffusion regeneration.Source: §1 · p. 2
• DnD frames universal watermark removal as routing visible semantics into a grayscale-like structure latent while suppressing discardable residual evidence.Source: §1 · p. 2
• DnD trains a staged curriculum combining latent spectral perturbations, diffusion regeneration, and residual refinement through a straight-through path.Source: §1 · p. 2
• The evaluation suite analyzes removability through fidelity-constrained attacks, latent swaps, energy decomposition, decoder probes, spectral localization, and a Fano-type certificate.Source: §1 · p. 2
• One shared K/U/S operating point is evaluated across seven watermarking methods, including SSL, RivaGAN, Stable Signature, and Tree-Ring.Source: §1 · p. 2

2.1 Watermarking

• Classical frequency-domain and learned watermarking methods distribute payload evidence differently, so survival of standard image edits does not establish inseparability from image semantics.Source: §2.1 · p. 2

2.2 Removal and Regeneration Attacks

• Removal studies include regeneration and diffusion attacks, while DnD instead learns an image-specific representation split before regeneration.Source: §2.2 · p. 3

2.3 Representation Reconstruction

• Representation-learning and information-bottleneck ideas motivate preserving task-relevant visible content while routing nuisance evidence elsewhere.Source: §2.3 · p. 3

2.4 Adaptive Defenses

• Adaptive defenses test whether the carrier withstands spectral perturbation and whether residual dropping remains effective after image reconstruction.Source: §2.4 · p. 3

• The attacker receives only a watermarked RGB image and lacks the watermark method, decoder, embedding key, and payload.Source: §3 · p. 3
• Success means increasing BER for multi-bit watermarks or reducing detection at a fixed false positive rate for detector-based watermarks, subject to preserving visible content.Source: §3 · p. 3
• The model is trained against defenses that strengthen spectral robustness or use diffusion regeneration to purify the image.Source: §3 · p. 4

4.1 Overview

• DnD routes the semantic carrier into structure latent g and residual evidence into auxiliary latent u, then decodes with the residual branch suppressed.Source: §4.1 · p. 4
• Low-strength diffusion stress and residual refinement are applied after dropping as stress and repair components, not as the main source of watermark removal.Source: §4.1 · p. 4, Figure 1 · p. 5
• The inference pipeline encodes the watermarked image, spectrally perturbs the structure branch, drops the residual, and decodes the resulting image.Source: §4.1 · p. 4, Figure 1 · p. 5
• The strict algorithm sets U = 0, while reported K/U/S operating points use U as a controlled residual keep ratio; K controls spectral perturbation and S diffusion strength.Source: §4.1 · p. 5

4.2 Reconstructive Grayscale Residual Decomposition

• DnD routes visible structure into a grayscale-like carrier and residual evidence into an auxiliary branch, so dropping that branch moves reconstruction toward the clean image.The full pair reconstructs the watermarked image, while decoding with the auxiliary branch removed suppresses the routed payload evidence.Source: §4.2 · p. 6

4.3 Formal Analysis of Residual Channel Removability

• Removal under a fidelity constraint requires reduced verification signal while meeting the fidelity threshold; success is measured by BER for bit payloads or detector acceptance for detector-based watermarks.The definition excludes visibly damaging outputs that fail the fidelity constraint.Source: §4.3 · p. 6
• The information gate treats dropping the auxiliary branch as valid removal when the structure latent stays close to the clean carrier and the dropped output remains faithful to it but separates from the watermarked observation.The certificate is stated for the unperturbed latent pair; the deployed perturbed drop path is checked separately.Source: §4.3 · pp. 6–7
• If payload recoverability follows the auxiliary latent across swaps and falls when that branch is zeroed, the evaluated evidence is more strongly localized there than in the structure branch.The counterfactuals hold the decoder fixed and exchange one branch at a time.Source: §4.3 · p. 7
• As decoder bit error approaches 0.5, the Fano-type lower-bound certificate vanishes, but this does not establish that all payload information has been erased.The result concerns recoverability by the evaluated decoder.Source: §4.3 · p. 7
• Under a fixed fidelity constraint, useful removal comes from suppressing the auxiliary residual branch while preserving PSNR and SSIM; stronger image perturbation or diffusion can raise removal metrics at low fidelity.Diffusion serves as an adaptive stressor and low-strength refinement, not the core remover.Source: §4.3 · p. 7

4.4 Latent Spectral Perturbation for Carrier Stability

• Spectral perturbation is intended to make the drop path rely on content stable across frequency bands, not remove the watermark by corrupting the carrier.A clean-drop-only model can be brittle: small spectral changes may reintroduce residual evidence or damage the decoded image.Source: §4.4 · p. 7
• The latent operator partitions g into low, mid, and high radial bands and can apply band keep ratios, a ring notch, magnitude noise, and phase noise.Strength K interpolates between identity and stage-dependent targets; the perturbation acts only on g.Source: §4.4 · p. 7
• At inference, K perturbs the structure latent before the auxiliary branch is dropped, while S controls post-drop diffusion stress and refinement.With S = 0, the output is the strict decoded image; when S > 0, the diffusion and refiner path is applied.Source: §4.4 · pp. 7–8

4.5 Diffusion Regeneration and Straight Through Training

• The diffusion stressor regenerates a selected decoded image at strength S, step count N, and guidance G; a straight-through estimator keeps Stage 3 repair trainable.Its forward value is the regenerated image, while gradients flow as if regeneration were the identity.Source: §4.5 · p. 8

4.6 Residual Refiner

• The residual refiner restores the regenerated image using its high-frequency residual and re-encoded structure latent.The Stage 3 setting uses width 64, two Restormer blocks, four attention heads, and α = 0.5.Source: §4.6 · p. 8

4.7 Training Curriculum

• Training proceeds through reconstruction stabilization, robust latent dropping, and repair under diffusion exposure.The three stages first stabilize reconstruction, then learn latent dropping, and finally train repair under diffusion exposure.Source: §4.7 · p. 8

4.7.1 Stage 1: Reconstruction Warmup

• The 20-epoch warmup reconstructs clean and watermarked images without perturbation or the drop loss, emphasizing reconstruction consistency and grayscale conformity.Source: §4.7.1 · p. 8

4.7.2 Stage 2: Latent Drop Robustness

• During the 70-epoch latent-drop stage, spectral perturbation increases while the drop objective is warmed in and watermarked reconstruction retains a nonzero floor.The floor preserves decoder capacity without allowing the auxiliary branch to dominate the structure branch.Source: §4.7.2 · p. 8

4.7.3 Stage 3: Diffusion Repair

• The 50-epoch diffusion-repair stage freezes the Stage 2 encoder, decoder, and perturbation base by default, training only the repair path under gradually increasing perturbation.It begins with light regeneration at S = 0.10, 6 steps, and guidance 1.0.Source: §4.7.3 · p. 9

4.8 Loss Groups

• The objective groups fidelity, grayscale conformity, payload removal, perceptual and structural consistency, and latent regularization losses to balance reconstruction and watermark suppression.The groups include direct drop loss, attacked-latent alignment, recovery loss, VGG and frequency losses, edge loss, and auxiliary sparsity.Source: §4.8 · p. 9

5 Why the Drop Works

• The probes operationalize whether payload evidence is measurable in a discardable channel while the semantic carrier remains stable.The analysis connects this test to prior observations on benign-edit benchmarks, regeneration attacks, and representation disentanglement.Source: §5 · p. 9

5.1 Information Gate

• The information gate requires the structure latent to remain close to the clean carrier while the full latent pair reconstructs the watermarked image more faithfully than the dropped pair.Source: §5.1 · p. 9

5.2 Counterfactual Payload Swaps

• Counterfactual swaps test whether replacing the auxiliary latent reintroduces payload evidence and dropping it suppresses that evidence.The counterfactual images use the same decoders as the main BER measurements.Source: §5.2 · p. 9

5.3 Pixel Energy Decomposition

• The pixel-energy probe tests whether much of the watermark-induced energy is attributable to the auxiliary branch while cross energy remains controlled.Table 2 frames the probes as measurable predictions distinguishing targeted residual suppression from untargeted image degradation.Source: §5.3 · p. 9, Table 2 · p. 10

5.4 Decoder Sensitivity

• The Lipschitz probe compares decoder-output sensitivity to unit perturbations in the structure and auxiliary latents, distinguishing a branch that must remain stable from one that can be dropped.Source: §5.4 · p. 10

5.5 Spectral Localization

• The band-epsilon probe compares low-, mid-, and high-frequency residuals before and after spectral stress to test alignment with watermark-difference bands.Source: §5.5 · p. 10

5.6 Payload Information

• For the evaluated decoder, BER approaching 0.5 drives the bitwise Fano-type certificate of recoverable information toward zero, without establishing that all payload information is absent.Source: §5.6 · p. 10

6.1 Dataset and Methods

• Seven watermark families are evaluated on paired clean and watermarked images resized to 512 × 512, with equal family weighting in aggregate results.The families are SSL, DwtDct, DwtDctSvd, RivaGAN, StegaStamp, Stable Signature, and Tree-Ring.Source: §6.1 · p. 10

6.2 Metrics

• Bit-decoder methods use BER, PSNR, and SSIM, while Tree-Ring uses removal accuracy at a fixed low-FPR detector setting with PSNR and SSIM.BER near 0.5 indicates decoder performance approaching random guessing; values above 0.5 are not treated as stronger removal.Source: §6.2 · p. 10

6.3 Operating Curves

• The operating scans vary latent perturbation K, residual keep ratio U, and diffusion strength S, then report every family at K = 0.925, U = 0.02, S = 0.04.Figure 2 compares BER and PSNR: conventional edits reach high removal only at low PSNR, while the fixed DnD point lies in the high-fidelity region.Source: §6.3 · p. 10, Figure 2 · p. 11
• Table 3 compares conventional image-space removal attacks with DnD at the fixed controlled operating point.It reports BER for bit-decoder watermarks and Removal Acc. for Tree-Ring; BER above 0.5 is interpreted under the random-guessing ceiling.Source: Table 3 · p. 11

7 Results

• Mean BER 0.400, mean PSNR 28.98 dB, and mean SSIM 0.902 across six bit-decoder families; Tree-Ring removal accuracy is 0.4700 at 27.57 dB.The fixed DnD setting is not optimal for every metric, but occupies a more useful removal–fidelity trade-off than the compared endpoints.Source: §7 · p. 11, Figure 3 · p. 12

7.1 Operating and Ablation Evidence

• Figure 4 shows DnD preserving image content while altering decoder-relevant residual evidence, unlike severe brightness changes or high-noise attacks.The figure compares attacked-image and amplified-residual rows across seven watermark families, with the fixed DnD endpoint highlighted in green.Source: §7.1 · p. 12, Figure 4 · p. 13

7.2 Attack Operating Regime

• The coarse scan reaches mean BER 0.521 and Tree-Ring removal accuracy 0.90 only at PSNR 17.99 dB and SSIM 0.464; low U and small positive S instead keep BER near 0.41 and PSNR around 28–29 dB.The fixed shared setting comes from the practical regime rather than the maximum-removal boundary, without per-method tuning.Source: §7.2 · pp. 13–14, Table 5 · p. 14

7.3 Auxiliary Residual Reintroduction Under Diffusion

• Without diffusion, raising U from 0 to 1 improves PSNR from 26.09 dB to 31.21 dB but lowers mean BER from 0.373 to 0.098; under diffusion, reintroducing residuals mainly weakens removal.With coarse S > 0, PSNR shifts only from 21.45 dB to 21.37 dB as U rises, while Tree-Ring removal accuracy falls from 0.61 to 0.28.Source: §7.3 · p. 14, Figure 5 · p. 15

7.4 Component Contributions

• Replacing frequency-aware latent perturbation with latent noise raises mean BER to 0.495 but lowers PSNR to 24.40 dB; removing clean-drop supervision gives BER 0.449 and PSNR 22.80 dB.Unfreezing the Stage 2 base shifts toward reconstruction, while disabling the test-time refiner weakens both removal and fidelity; the refiner repairs structure after the residual branch is reduced.Source: §7.4 · pp. 14–15, Figure 6 · p. 15

8.1 Implications for Watermark Design

• A watermark may be vulnerable when its payload occupies a low-energy, discardable residual channel that can be removed without disturbing the semantic carrier.The discussion argues that persistence under standard edits does not imply resistance to adaptive reconstruction [15].Source: §8.1 · pp. 15–16

8.2 The Limited Role of Strong Diffusion in Removal

• Strong diffusion can weaken watermark evidence by overwriting residuals, but the resulting image may leave the fidelity-constrained regime.The discussion treats diffusion as an adaptive training stressor and low-strength inference refinement, while useful removal comes from preserving g and suppressing u.Source: §8.2 · p. 16

8.3 Diffusion as Semantic Regeneration Rather Than Reconstruction Attack

• Diffusion’s semantic regeneration can preserve coarse structure yet alter non-watermark details; DnD instead targets decoder- or detector-relevant evidence while keeping the carrier close to the original.Low-strength diffusion can test whether residual evidence reappears after latent drop and help refine the post-drop image.Source: §8.3 · p. 16

9 Conclusion

• Robustness to compression, resizing, blur, or color changes is insufficient when a learned remover can discard residual evidence while preserving the semantic image carrier.Source: §9 · p. 16
• Across seven watermark families, one shared DnD setting transfers, while stronger noise or diffusion mainly improves removal by damaging the image.Useful removal requires quality-aware latent dropping and residual repair rather than simply more aggressive perturbation.Source: §9 · p. 16
• Future work should test representation-level learned removers on broader datasets, unseen watermark keys, and stronger adaptive watermark designs.Source: §9 · p. 16