PaperlayerView exact revision on arXiv

Source-linked AI summary

AngelFingerprint: A Traceable, Explainable, and White-Box Stealthy Watermark for Text-Guided Image Editing

Bo-Han Kung, Futa Waseda, Ching-Chun Chang, Isao Echizen, Shang-Tse Chen

arXiv:2609.04709v1cs.CV

TL;DR

Text-guided diffusion editing makes convincing misinformation easier, while fixed or separable watermarks provide limited semantic provenance and remain vulnerable to white-box inspection. AngelFingerprint embeds each editing prompt’s CLIP embedding into LoRA-modified diffusion-model weights and recovers it from image pixels. Its frequency filtering and velocity-alignment designs support a hidden, robust watermark for accountable open-weight editing.

  • Problem

    Text-guided editing raises provenance concerns because fixed-ID watermarks do not explain edits and separable watermark modules can be located and removed by white-box adversaries.

  • Method

    AngelFingerprint embeds the editing prompt’s CLIP text embedding into LoRA-modified generator weights and uses an extractor to recover it from image pixels alone.

  • Results

    The framework provides prompt-specific watermarking while preserving the original architecture and computation, with robust prompt recovery reported on MagicBrush.

  • Takeaways & Limitations

    Embedding a semantic edit record into model weights unifies traceability, explainability, and white-box stealthiness for text-guided diffusion editing.

  • Takeaways & Limitations

    The reported detector evidence uses an SD3-medium norm scale, limited negative sets, and 200 images per group, so thresholds and extreme-tail false-positive rates are not broadly established.

Abstract

from arXiv · show

Text-guided diffusion editing raises disinformation concerns, making reliable image provenance essential. While watermarks are commonly used for this purpose, most methods carry a fixed ID that cannot explain what was changed and which prompt produced it. Furthermore, under open-source white-box access, attackers can easily locate and remove watermarks added as separate modules. Targeting this setting, we propose AngelFingerprint, a novel watermarking framework ensuring edit traceability, explainability, and white-box stealthiness. It integrates a LoRA into the diffusion model to embed the editing prompt's CLIP text embedding directly into the model's weights. An extractor then recovers this embedding from the image pixels alone. This semantic payload explains the edit, while the weight-integrated design makes it hard to detect and isolate even under full white-box access. Two techniques make this possible: a velocity-alignment anchor that preserves edit quality, and a specially designed frequency filter that keeps the watermark imperceptible yet recoverable and robust. On the MagicBrush dataset, our extractor achieves $86\%$ top-1 accuracy in a 200-way prompt retrieval, versus $20\%$ for prompt inversion.

1 Introduction

Text-guided diffusion editing enables convincing misinformation, but existing watermarks often cannot explain edits and can be removed by white-box inspection. AngelFingerprint embeds each editing prompt’s CLIP embedding into model weights, using two designs to preserve quality and produce a hidden, recoverable watermark.

  • Text-guided editing can preserve much of an original photograph, making fabricated evidence, disinformation, and non-consensual imagery convincing.
  • Existing watermarks often carry fixed payloads that identify a model or user but do not reveal what changed or the edit’s intent.
  • White-box adversaries can inspect open-source architectures and remove watermarking modules that are structurally separate from the diffusion pipeline.
  • The framework jointly targets traceability, explainability, and white-box stealthiness for text-guided diffusion editing.
  • AngelFingerprint embeds the editing instruction’s CLIP text embedding into LoRA-modified generator weights and extracts it from image pixels alone.The architecture, computation, and inference code remain unchanged, leaving no additional module or forward pass to isolate.
  • The frequency filter stabilizes training, prevents extractor shortcuts, and keeps the watermark invisible and robust, while the velocity-alignment anchor preserves editing quality.The authors report that removing these designs causes training to collapse and prevents a usable watermark from emerging.

2 Related Work

Prior diffusion-model watermarks commonly encode fixed bit strings, while semantic-payload methods generally use separable modules. AngelFingerprint is positioned as combining prompt recovery with white-box stealthiness.

  • Most surveyed diffusion watermarks carry non-semantic, predefined bit-string payloads rather than image-specific editing prompts.Examples include Stable Signature, Tree-Ring, Gaussian Shading, Spherical, Robust-Wide, and AquaLoRA.
  • AngelFingerprint is presented as the only method in the comparison that is both white-box stealthy and prompt-recovering.White-box stealthiness means retaining the original architecture and computation, making the watermark difficult to detect or isolate.
  • Semantic-payload methods embed captions, text vectors, or prompts, but their separable injection modules remain easy for white-box adversaries to locate and remove.
  • Prompt-inversion methods infer prompts from image content without watermarks, which the paper characterizes as unreliable for text-guided editing.

3 Method

AngelFingerprint embeds each editing prompt’s CLIP text embedding into diffusion-model weights and recovers it from filtered image signals. Its training objective combines embedding recovery, fidelity alignment, and frequency filtering to support readable, imperceptible, robust watermarks.

  • System overview: AngelFingerprint inserts a LoRA into DiT attention projections, embedding the editing prompt’s CLIP representation directly into generator weights without an external module.The extractor recovers the embedding from image pixels, while the architecture and computation remain unchanged.
  • Embedding recovery: The embedding-recovery objective aligns each recovered embedding with its prompt target and uses symmetric in-batch negatives to optimize retrieval margins.The contrastive terms address the weakness of diagonal-only cosine alignment, which can yield high similarity without discriminative retrieval.
  • Fidelity anchor: The velocity-alignment anchor matches LoRA-augmented and vanilla model predictions across conditional and unconditional branches to preserve editing fidelity across guidance choices.The unconditional branch receives greater weight because it forms the base for classifier-free guidance and its drift is more damaging to fidelity.
  • Frequency filtering: The fixed frequency filter passes only a mid-frequency band to the extractor, stabilizing joint training and forcing the watermark away from content-dominant frequencies.Filtering also blocks embedding-recovery gradients from the low- and high-frequency bands.
  • Frequency filtering: The frequency design supports imperceptibility and robustness by leaving appearance-dominant low frequencies unchanged and avoiding fragile high frequencies lost during compression or resampling.The resulting watermark is constrained to a more stable, less visible spectral region.

4 Experiments

Experiments evaluate prompt recovery, imperceptibility, visual fidelity, white-box stealthiness, and robustness on MagicBrush using SD3-medium and UltraEdit. AngelFingerprint recovers editing prompts from watermark signals while preserving image quality and resisting several attacks.

  • Experiments measure watermark performance, white-box stealthiness, visual fidelity, and robustness on MagicBrush with SD3-medium and UltraEdit backbones.
  • Watermark Performance: Top-1 reaches 0.860 and MRR 0.917 on SD3 medium, while UltraEdit reaches Top-1 0.655 for prompt retrieval.The recovered embedding has cosp = 0.802 on SD3 medium.
  • Watermark Performance: Δcos=0.438 and ΔT1=+0.860 show that the extractor relies on the watermark rather than image content.Unwatermarked-image retrieval drops to chance.
  • Visual Fidelity: The watermark remains close to LoRA-off outputs across visual metrics and appears as faint, grid-like texture requiring close inspection.Pixel metrics are only loosely meaningful because no ground-truth clean watermarked counterpart exists.
  • Visual Fidelity: FID and KID remain close to the vanilla SD3 realism floor when using the JPEG mid-band filter.
  • Robustness: JPEG compression and cropping preserve the watermark, whereas resizing threatens recovery by disrupting mid-frequency components.
  • White-box Stealthiness: Quantization preserves Top-1 at 0.86 through int8, while int4 lowers Top-1 to 0.505 and raises KID from 14.3 to 164.4.After fine-tuning, Top-1 remains 0.840 at 50 steps and 0.575 at 300 steps.
  • Open-set Verification: Open-set verification recovers embeddings from edited images and ranks candidate sentences by cosine similarity, enabling verification without a fixed prompt pool.

5 Conclusion

AngelFingerprint embeds prompt embeddings into LoRA weights to unify traceability, explainability, and white-box stealthiness. Experiments show watermark-driven prompt recovery, while the framework points toward accountable open-weight releases and text decoding as a next step.

  • AngelFingerprint unifies traceability, explainability, and white-box stealthiness by embedding prompt CLIP embeddings into LoRA weights.
  • The framework supports open-weight releases that embed a self-explaining record into every generated edit.The authors identify direct text decoding from payload embeddings as a natural next step.
  • Near-zero retrieval on unwatermarked images indicates that the extractor reads the embedded watermark rather than using image content as a shortcut.

A.2 Robustness

Robustness evaluation covers image post-processing and post-hoc fine-tuning, with implementation details for testing attacks on released weights. The mid-band design preserves recovery under band-preserving operations but is vulnerable to resizing-related spectral disruption.

  • Robustness is evaluated against image post-processing and post-hoc fine-tuning.
  • Post-Processing: Cropping and JPEG compression preserve the watermark, while resizing can disrupt recovery by desynchronizing 8×8 DCT blocks.
  • Frequency Filtering: The band-pass filter supplies the extractor with mid-band signals while removing semantic content that could enable shortcut learning.
  • Post-Hoc Fine-Tuning: The fine-tuning attack trains an additional LoRA while keeping the base transformer, VAE, text encoders, and extractor frozen.
  • Post-Hoc Fine-Tuning: The attack uses 20 held-out MagicBrush images, checkpoints through 300 steps, and evaluates Top-k retrieval, MRR, and cosine similarity.

B.1 Objective Function

The objective combines losses for visual fidelity, embedding recovery, and velocity alignment. Ablations show that InfoNCE, separate velocity anchoring, and a balanced mid-frequency filter are important for retrieval and edit quality.

  • Loss ablations: Removing InfoNCE lowers Top-1 retrieval from 0.860 to 0.700, although visual quality slightly improves.The cosine term aligns embeddings with targets but does not separate them from one another.
  • Loss ablations: Removing the velocity anchor raises Top-1 to 0.895 but degrades PSNR from 22.9 to 19.2 and KID from 0.0029 to 0.0366.The unconstrained LoRA writes a visible watermark that is easier to read and see.
  • Loss ablations: Aligning only the guidance difference leaves fidelity poor, with DreamSim at 0.295, so both velocity branches require separate anchoring.
  • Anchor-weight ablation: The velocity-anchor weights (α, β)=(1, 2) give the best retrieval, with Top-1 0.860, while β > α protects fidelity and preserves readability.
  • Filter ablation: The 26/64 mid-frequency mask provides the best trade-off because wider bands destabilize training and narrower bands reduce the available watermark signal.It passes enough mid-frequency information for extraction while regularizing joint LoRA-extractor optimization.

C Deployment

AngelFingerprint deploys as a merged LoRA whose unchanged architecture and computation conceal the watermark in model weights. Verification first detects provenance, then retrieves the editing instruction from a recovered CLIP embedding.

  • Deployment: Every edit carries an invisible record of its instruction, while a verifier can detect the source model and recover that instruction from a suspect image.
  • Deployment: A merged watermarking LoRA leaves architecture, tensor shapes, forward computation, inference code, sampler, and network structure unchanged.The LoRA is folded into existing attention-projection weights without an extra branch or network.
  • Verification: The extractor-output norm provides provenance detection, reaching AUC near 1 with 100% true positive rate at 1% false positive rate.The recovered embedding is a 768-dimensional vector in CLIP-L text space.
  • Instruction recovery: Candidate instructions can combine hand-written hypotheses, vision-language-model proposals, and background instruction corpora before CLIP-L matching.This design avoids requiring the verifier to know the edit wording in advance.
  • Evaluation: On a 200-sentence pool, Top-1 recovery reaches 0.860 on SD3-medium and 0.655 on UltraEdit-SD3.Content baselines recover nothing under the same test, with Top-1 ≤0.225 and ∆T1 < 0 against the LoRA-off image.

D.2 Dataset: MagicBrush

MagicBrush supplies independent real-image editing turns for training and evaluation. The watermark is robust to cropping and JPEG compression but fragile to resizing, while the 26/64 filter offers the best reported fidelity-retrieval trade-off.

  • Dataset: MagicBrush consists of human-annotated real-image edit turns, each containing a source image, instruction, mask, and target image.Turns are grouped into sessions but treated independently.
  • Robustness: Cropping and JPEG compression preserve the watermark, whereas resizing is fragile because it desynchronizes the DCT band.
  • Filter configuration: The 26/64 mid-band mask gives the best overall trade-off between retrieval accuracy and visual quality among the tested masks.

D.3 Implementation Details

The implementation uses a multi-scale CNN extractor with a fixed mid-band DCT mask and jointly trains it with a LoRA using fidelity, embedding, and velocity-anchor objectives. Provenance detection relies on extractor-output magnitude evaluated against watermarked, unwatermarked, and natural-image groups.

  • Extractor: The extractor uses five stride-2 convolutional blocks, multi-scale pooling, and an MLP head projecting to a 768-dimensional CLIP-L target.
  • Extractor: A fixed block-DCT mask retains the mid-band diamond 4≤i+j≤7, or 26 of 64 coefficients per block, while removing low-frequency content.
  • Training objective: The loss combines flow matching, embedding recovery, and a four-point velocity anchor, with embedding recovery including cosine alignment and InfoNCE.
  • LoRA configuration: The LoRA uses rank r=96 and adapts attention projections throughout the DiT blocks, with additional text-stream projections for UltraEdit-SD3.
  • Provenance detection: Provenance detection measures extractor-output norm across watermarked images, LoRA-off negatives, and natural photographs.The evaluation uses three groups of 200 images.

E.2 The Output Norm Is a Perfect Detector

The extractor-output norm cleanly separates watermarked images from both LoRA-off images and natural photographs, enabling threshold-based detection. This separation is empirically strong but depends on model-specific calibration and limited negative sets.

  • AUC 1.000 separates watermarked images from both LoRA-off images and natural photographs in every comparison.Figure A4 shows clearly separated extractor-output norm distributions for the three groups.
  • Watermarked images have mean norm 40.3, while both negative groups have means near 7.The watermarked 5th percentile is 27.8, compared with 10.1 for the LoRA-off group's 95th percentile.
  • A threshold of ∥ˆe∥2 > 12 achieves 100% true positive rate at 1% false positive rate on this test.The detector requires no candidate prompt and is available to a verifier holding the extractor.
  • The norm scale requires recalibration across backbones, and the evaluation excludes stronger adversarial negatives and very low false-positive targets.The study uses same-generator LoRA-off images and natural photographs, with 200 images per group.
Loading 2609.04709v1…