Source-linked AI summary
ProxyEraseAgent: Blind Watermark Removal in the Wild
Jun Yao, Chao Wang, Yupeng Qiu, Zehua Ma, Weiming Zhang, Bin Liu, Han Fang
TL;DR
Single-image blind watermark removal must preserve visible content while disrupting a watermark whose decoder is unavailable, leaving no direct feedback on removal progress. ProxyEraseAgent retrieves public decoders as proxies and uses their responses to guide sequential search; across 11 systems, it achieves 94.8% overall cASR.
Problem
In single-image blind removal, the attacker has only the disseminated image and cannot directly assess whether an operation weakened the hidden watermark while preserving visible content.
Source: §1 · p. 1Method
ProxyEraseAgent ranks public decoders by calibrated responses to the query image and uses top proxies to guide sequential search over heterogeneous removal operations under perceptual-quality constraints.
Source: §1 · p. 2, §3.3 · p. 5, §3.4.2 · p. 5Results
Across 11 watermarking systems, ProxyEraseAgent achieves 94.8% overall cASR, outperforming the strongest fixed attack by 15.3%.
Source: §5 · p. 10Takeaways & Limitations
The results support using feedback from external watermarking models to guide adaptive, image-specific planning when the underlying mechanism is unknown and inaccessible.
Source: §5 · p. 10
Abstract
from arXiv · showhide
Invisible image watermark removal has received growing attention. Despite substantial progress, existing attacks face a tension between practicality and specificity. Attacks exploiting detector outputs, decoder responses, or paired images can be tailored to the watermark decision boundary, but require information rarely available in realistic scenarios. Conversely, attacks based on compression, geometric distortion, or reconstruction are easily deployed from a single watermarked image, but remain largely open-loop: they apply generic transformations without knowing if the image is moving toward watermark failure. Thus, the key challenge in single-image blind watermark removal is not merely how to transform the image, but how to obtain a useful removal direction without accessing the hidden decoder. To bridge this gap, we propose ProxyEraseAgent, an agent-driven framework recovering attack specificity through proxy decoder responses. Publicly available watermarking schemes provide a natural knowledge base of candidate decoders, where some are informative for a given unknown image. Our insight is that a decoder producing a strong calibrated response to the query image likely shares a nearby decoding boundary with the hidden target mechanism. ProxyEraseAgent ranks these decoders by calibrated response strength and uses the top ones as proxy boundary estimators. Their responses then guide a progressive search over heterogeneous removal operations (e.g., geometric distortion, JPEG compression, image reconstruction, and gradient perturbation) under perceptual-quality constraints. Experiments across 11 watermarking systems show ProxyEraseAgent achieves a 94.8% attack success rate, demonstrating the effectiveness of response-guided proxy retrieval and feedback-driven sequential planning for blind watermark removal.
1 INTRODUCTION
Single-image blind watermark removal must disrupt an inaccessible watermark while preserving visible content, but the image alone gives no direct signal of removal progress. ProxyEraseAgent retrieves public decoders as proxies and uses their responses to guide removal, achieving a 94.8% attack success rate across 11 systems.
1 INTRODUCTION
- The attacker sees only a disseminated watermarked image and must make its hidden watermark unrecoverable while preserving visible content.Source: §1 · p. 1
- Without access to the hidden watermark, appearance alone cannot show whether an operation weakened decoding, while stronger edits may unnecessarily reduce image quality.Source: §1 · p. 2
- Existing attacks either use target-specific feedback that is rarely available or apply prior-free transformations without estimating watermark disruption.Source: §1 · p. 2
- Public decoder responses to the query image can indicate which candidates may approximate the hidden decoding boundary, without identifying the exact watermarking algorithm.Source: §1 · p. 2
- ProxyEraseAgent ranks calibrated responses from public decoders, then uses top proxies to guide sequential exploration of heterogeneous removal operations under perceptual-quality constraints.Source: §1 · p. 2
- 94.8% attack success rate across 11 image watermarking systems demonstrates ProxyEraseAgent's effectiveness without querying the hidden target decoder.Source: §1 · p. 3
2 RELATED WORK
Watermark systems vary in mechanisms and robustness objectives, so removal attacks can transfer unevenly across them. Existing approaches include fixed transformations and surrogate-guided attacks, while ProxyEraseAgent retrieves public decoders as proxies for adaptive search.
2 RELATED WORK
- Watermarking systems span different mechanisms, payload capacities, and robustness objectives, making attack transfer across systems uneven.Source: §2 · p. 3
- Predefined transformations include geometric changes, JPEG compression, denoising, adversarial perturbations, and generative reconstruction (An et al., 2024; Saberi et al., 2024; Jiang et al., 2023; Zhao et al., 2024).Source: §2 · p. 3
- Surrogate-guided attacks transfer perturbations from trained models, whereas ProxyEraseAgent retrieves public decoders by their calibrated responses to the query image.Source: §2 · p. 3
3 METHOD
ProxyEraseAgent pairs public-decoder retrieval with adaptive attack planning, using proxy responses rather than the hidden target decoder during search. It evaluates attack states for proxy disruption and perceptual distortion, then retains trade-offs through bounded Pareto search.
3.1 SYSTEM OVERVIEW
- The retrieval and planning agents use proxy-decoder feedback to construct an adaptive removal path; the target decoder is accessed only for final evaluation.Source: Figure 2 · p. 4, §3.1 · p. 4
3.4 PROXY-GUIDED ATTACK PLANNING AGENT
- The planning agent searches attack sequences using proxy-decoder feedback while evaluating both watermark disruption and perceptual quality.Source: §3.1 · p. 4, §3.4.2 · p. 5
3.2 KNOWLEDGE BASE
- The heterogeneous knowledge base combines public watermarking systems’ encoders, decoders, and inference metadata, using decoders for feedback and encoders for self-watermarked references.Source: §3.2 · p. 4
3.3 RESPONSE-GUIDED PROXY RETRIEVAL
- Proxy retrieval calibrates decoder response strength against each decoder’s own-encoder reference, ranks candidates, and uses top-K scores as attack-search importance weights.Response strength is one minus mean binary entropy; the calibration makes heterogeneous decoder responses more comparable.Source: §3.3 · pp. 4–5
3.4.1 ATTACK ACTION SPACE
- The attack space combines geometric distortion, compression distortion, regeneration, and adversarial perturbation, with multiple strengths for each family.Source: §3.4.1 · p. 5
3.4.2 PROXY-GUIDED SEQUENTIAL ATTACK SEARCH
- Proxy bit error rate measures changes from each proxy’s original decoded message, while LPIPS measures perceptual distortion from the original image.Source: §3.4.2 · p. 5
3.4.3 PARETO BEAM SEARCH
- Pareto beam search removes dominated candidates, keeps a width-three beam for at most T steps, and selects the highest-disruption state within the quality constraint.In the experiments, τ_select = 0.05; ties favor lower distortion and then shorter sequences.Source: §3.4.3 · p. 6
4 EXPERIMENTS
Experiments evaluate ProxyEraseAgent across 11 watermarking systems using constrained attack success rate and compare it with fixed attacks and ablations. Results examine proxy selection, sequential search depth, robustness to BER thresholds, and self-calibrated retrieval scores.
4.1 EXPERIMENTAL SETUP
- The evaluation uses 100 MS-COCO 2017 images, independently watermarked by 11 systems, and measures cASR using BER ≥0.1 with AlexNet LPIPS ≤0.05.The attack space includes TPS warping, JPEG compression, VAE reconstruction, and lightweight adversarial optimization; comparisons include generic, reconstruction-based, and dedicated watermark-removal attacks.Source: §4.1 · p. 6
4.2 MAIN COMPARISON WITH WATERMARK-REMOVAL BASELINES
- 94.8% overall cASR versus 79.5% for the strongest fixed attack, Regen-VAE, gives ProxyEraseAgent a 15.3% improvement under the same perceptual-quality constraint.It achieves the best performance on 9 of 11 watermarking systems.Source: Table 1 · p. 7, §4.2 · p. 7
4.3 EFFECT OF CANDIDATE-DECODER SELECTION
- Under blind retrieval, calibrated-entropy selection reaches 94.82% cASR with K = 3, compared with 89.00% for randomly selecting three proxies.In the blind setting, K = 1 achieves 90.36%, K = 5 reaches 95.36%, and assuming target-decoder coverage yields 99.73% with K = 3.Source: §4.3 · pp. 7–8, Table 2 · p. 8
4.4 EFFECT OF SEQUENTIAL SEARCH DEPTH
- Increasing maximum search depth from one to four raises cASR from 83.91% to 94.82%, while average LPIPS rises from 0.0360 to 0.0421 and remains within the quality constraint.Average selected sequence length grows from 1.00 to 2.96, indicating that the planner uses multi-step paths when beneficial.Source: §4.4 · p. 8
4.5 ROBUSTNESS ACROSS BER THRESHOLDS
- ProxyEraseAgent maintains a clear advantage over baselines as BER success thresholds become stricter, with the perceptual constraint fixed at LPIPS ≤0.05.The results extend beyond the default BER ≥0.1 success threshold.Source: §4.5 · p. 8, Figure 4 · p. 9
4.6 ANALYSIS OF SELF-CALIBRATED RETRIEVAL SCORES
- Self-calibration normalizes decoder responses against each decoder’s self-watermarked references, making scores more comparable for proxy retrieval across heterogeneous systems.PIMoG’s narrow raw entropy distribution is less discriminable for ranking than its calibrated response distribution.Source: §4.6 · p. 9, Figure 4 · p. 9
5 CONCLUSION
ProxyEraseAgent retrieves public proxy decoders using calibrated responses and uses their feedback to adaptively construct attack paths. Across 11 watermarking systems, it achieves an overall cASR of 94.8%, outperforming the strongest fixed attack by 15.3 %.
5 CONCLUSION
- ProxyEraseAgent retrieves informative proxy decoders from public models using calibrated responses and uses their feedback to adaptively construct attack paths.The framework targets single-image blind watermark removal without access to the target decoder.Source: §5 · pp. 9–10
- 94.8% overall cASR, outperforming the strongest fixed attack by 15.3 %.The experiments span 11 watermarking systems.Source: §5 · p. 10
- The results support recovering feedback from external watermarking models to guide adaptive, image-specific attack planning when the target mechanism is unknown and inaccessible.The paper presents this as a perspective for evaluating watermark robustness in realistic settings.Source: §5 · p. 10
6 APPENDIX
The appendix specifies watermark-generation settings, attack operations, proxy-decoder sources, comparison baselines, and search constraints. Its configurations cover 11 watermarking systems and combine heterogeneous operations under a perceptual-quality limit.
6.1 WATERMARK GENERATION SETTINGS.
- Watermarked outputs from 11 systems used varied payloads and model settings, with higher-resolution outputs resized to 128 × 128 for downstream comparison.Inputs were converted to RGB and resized to 128 × 128 before embedding.Source: §6.1 · p. 11
6.2 ATTACK-SPACE CONFIGURATION
- The attack space includes TPS warping, JPEG compression, VAE reconstruction, and low-frequency adversarial perturbation with appearance regularization.Configurations specify four TPS scales, seven JPEG quality factors, six VAE quality levels, and 50, 100, or 200 optimization iterations.Source: §6.2 · pp. 11–12
6.3 KNOWLEDGE-BASE CONFIGURATION.
- The proxy-decoder knowledge base draws on 11 publicly released watermarking systems spanning diverse embedding mechanisms, architectures, payload configurations, and robustness objectives.Models are used with their corresponding inference configurations.Source: §6.3 · p. 12
6.4 ATTACK BASELINES ONFIGURATIONS
- Baselines include conventional transformations, VAE and diffusion regeneration, black-box adversarial embedding attacks, and UnMarker.UnMarker uses its HL configuration because CHL introduces substantially larger distortion and often exceeds the evaluation quality constraint.Source: §6.4 · p. 12
6.5 ATTACK-SEARCH CONFIGURATION.
- ProxyEraseAgent retrieves the top three proxies and runs Pareto beam search with width three and depth four, selecting results under LPIPS ≤0.05.The target decoder remains inaccessible during search and is used only for final evaluation.Source: §6.5 · p. 12
- Figure 5 presents examples of JPEG compression, VAE reconstruction, AdvB-RN18, Unmarker, and ProxyEraseAgent attacks.The figure juxtaposes examples from these attack approaches.Source: Figure 5 · p. 13