PaperlayerView exact revision on arXiv

Source-linked AI summary

EditGuard: Versatile Image Watermarking for Tamper Localization and Copyright Protection

Xuanyu Zhang, Runyi Li, Jiwen Yu, Youmin Xu, Weiqi Li, Jian Zhang

arXiv:2312.08883v1cs.CV

TL;DR

EditGuard addresses the gap between copyright-protecting watermarking and tamper localization for realistic AIGC-edited images. It combines dual watermarking with united image-bit steganography to decouple training from tampering types, and experiments report accurate localization, copyright recovery, and generalization. The framework remains limited by an empirically selected localization watermark and several future-work boundaries.

  • Problem

    Existing watermarking protects copyright and traceability but falls short of localizing realistic tampering, while prior methods do not jointly balance precise localization and broad generalizability.

  • Method

    EditGuard embeds localization and copyright watermarks using a united image-bit steganography design whose training is decoupled from tampering types.

  • Results

    EditGuard reports notable localization precision, generalization, and copyright accuracy across constructed and classical benchmarks without tampering-specific labels or additional training.

  • Takeaways & Limitations

    The framework offers a proactive watermarking mechanism for jointly decoding copyright information and tampered areas in AIGC-edited images.

  • Takeaways & Limitations

    The localization watermark is selected empirically, and the authors plan to optimize it end-to-end and replace its RGB representation with grayscale.

Abstract

from arXiv · show

In the era where AI-generated content (AIGC) models can produce stunning and lifelike images, the lingering shadow of unauthorized reproductions and malicious tampering poses imminent threats to copyright integrity and information security. Current image watermarking methods, while widely accepted for safeguarding visual content, can only protect copyright and ensure traceability. They fall short in localizing increasingly realistic image tampering, potentially leading to trust crises, privacy violations, and legal disputes. To solve this challenge, we propose an innovative proactive forensics framework EditGuard, to unify copyright protection and tamper-agnostic localization, especially for AIGC-based editing methods. It can offer a meticulous embedding of imperceptible watermarks and precise decoding of tampered areas and copyright information. Leveraging our observed fragility and locality of image-into-image steganography, the realization of EditGuard can be converted into a united image-bit steganography issue, thus completely decoupling the training process from the tampering types. Extensive experiments demonstrate that our EditGuard balances the tamper localization accuracy, copyright recovery precision, and generalizability to various AIGC-based tampering methods, especially for image forgery that is difficult for the naked eye to detect. The project page is available at https://xuanyuzhang21.github.io/project/editguard/.

1. Introduction

AIGC makes realistic image creation and manipulation easier, intensifying copyright and information-security concerns. Existing watermarking protects authenticity or copyright but does not jointly provide precise, tamper-agnostic localization and copyright recovery.

  • AIGC-enabled manipulation makes artistic works vulnerable to malicious tampering and unauthorized recreations, creating security, ethical, and legal concerns.
  • Existing forensic watermarking methods primarily detect authenticity or protect copyrights, but fall short of localizing tampered areas.
  • Dual forensics requires recovering an image’s copyright despite tampering and degradation while pinpointing manipulated regions without restricting tampering types.
  • EditGuard converts the task into united image-bit steganography, decoupling training from tampering types and enabling zero-shot localization.
  • EditGuard embeds dual invisible watermarks and decodes tampered areas and copyright information for universal tamper localization and copyright protection.

2. Related works

Prior work includes passive forensic localization and forensic watermarking, but these approaches separately address manipulation detection, copyright, or traceability. EditGuard is motivated by I2I steganography’s fragility and locality as a route toward universal localization.

  • Passive forensic methods often localize specific manipulation types, while universal methods commonly rely on artifacts or anomalies in tampered images.
  • Forensic watermarking verifies authenticity and traceability, typically hiding and recovering bitstreams with encoder-decoder or flow-based models.

3. Overall Framework of EditGuard

EditGuard combines sequential dual-watermark encoding with parallel decoding to recover copyright information and a tamper mask. Its design uses I2I steganography for localization and bit-into-image steganography for robust copyright recovery.

  • 3.1. Motivation: I2I steganography provides a localization mechanism because tampering damages the revealed secret image near altered pixels.
  • 3.2. Framework Design and Forensics Process: The framework embeds a spatial localization watermark and a global copyright watermark, creating compatibility conflicts between local/global and semi-fragile/robust requirements.
  • 3.2. Framework Design and Forensics Process: Sequential encoding hides the localization and copyright watermarks in order, while parallel decoding separately predicts the tamper mask and copyright watermark.
  • 3.2. Framework Design and Forensics Process: The decoded outputs support three cases: unregistered or severely tampered images, registered images with detected tampering, and registered images without detected tampering.
  • 3.3. United Image-Bit Steganography Network: EditGuard realizes localization through image hiding and revealing and copyright recovery through bit encryption and recovery within a united image-bit steganography network.

4. United Image-bit Steganography Network

The united Image-bit Steganography Network (IBSN) sequentially embeds localization and copyright watermarks, then recovers the original image, tamper-localization watermark, and copyright bits. Its modules combine invertible image hiding, bit encryption and recovery, degradation-prompt posterior estimation, and mask extraction.

  • Network architecture: IBSN sequentially hides a localization watermark in the original image, modulates copyright bits into the intermediate result, and recovers both watermark types alongside the original image.The network comprises image hiding, bit encryption, bit recovery, and image revealing modules.
  • Image hiding and revealing: Discrete wavelet features and enhanced additive affine coupling layers project the original image and localization-watermark branches within stacked invertible blocks.These blocks construct the image hiding and revealing modules for multimedia recovery.
  • Prompt-based posterior estimation: Prompt-based posterior estimation predicts vanished localization-watermark information from container-image features, using residual and channel-wise transformer blocks plus three learnable degradation prompts.Dynamic prompt weighting fuses degradation representations with extracted local and non-local features to obtain the posterior estimate.
  • Bit encryption and recovery: The copyright watermark is expanded into message feature maps and fused with multi-level image features for encoding, then recovered through a U-shaped sub-network and MLP.The watermark is represented as wcop∈{0, 1}L and decoded from the received container image.
  • Optimization and construction: Bi-level optimization first trains bit encryption and recovery, then jointly trains image hiding and revealing while freezing the bit modules.Training introduces degradations only to the container image and exposes IBSN to no tampering samples.
  • Optimization and construction: EditGuard compares a predefined localization watermark with its decoded version and thresholds their per-pixel absolute differences at τ=0.2 to produce a binary tamper mask.The localization watermark may be any natural image or a solid-color image, and the resulting mask is extracted by the mask extractor.

5. Experiments

Experiments show that EditGuard simultaneously localizes tampering, recovers copyright information, preserves visual quality, and generalizes to unseen AIGC-based edits and degradations. Its proactive design achieves strong results without labeled tampered data for training.

  • Localization comparisons: 0.102, 0.116, 0.441, and 0.065 F1-score margins over SOTA across four benchmarks were achieved without labeled data or tampered samples.EditGuard also pinpoints pixel-level tampered areas, whereas other methods often produce rough outlines or work only in some cases.
  • Watermarking comparisons: The container image had higher fidelity than SepMark, PIMoG, and MBRS, slightly lower fidelity than CIN, and the best NIQE performance.Under Stable Diffusion inpainting, dual watermarks remained imperceptible while bit accuracy was close to PIMoG and CIN despite carrying substantially more information.
  • AIGC-based editing: F1-score and AUC above 95%, around 90% IoU, and over 99.8% bit accuracy were maintained across AIGC tampering types in AGE-Set-C.The comparison methods did not provide copyright protection capabilities.
  • AIGC-based editing: EditGuard clearly annotated meticulously tampered areas whose tamper types were absent from training, while other localization methods almost entirely failed.The method relies on the fragility and locality of I2I steganography rather than image artifacts or instance-wise semantic information.
  • Robustness analysis: F1-score remained above 0.9 with only slight bit-accuracy decreases under Gaussian noise, JPEG compression, and Poisson noise.MVSS-Net† showed noticeable degradation under the same conditions.
  • Ablation study: Removing bi-level optimization produced near-random bit accuracy, while removing LFIM, transformer blocks, or prompt-based fusion reduced localization or degradation robustness.Without LFIM and transformer blocks, IoU declined by 0.032 and 0.009; without prompt-based fusion, F1/AUC/IoU fell by 0.035/0.031/0.046 under random degradations.

6. Conclusion

The conclusion presents EditGuard as a versatile watermarking mechanism that embeds imperceptible localization and copyright watermarks and decodes both outputs accurately. Future work targets stronger robustness, semantic localization, and extension to additional modalities.

  • Conclusion: EditGuard embeds imperceptible localization and copyright watermarks while decoding copyright information and tampered areas for artistic and legal-forensic use.The paper characterizes this as its first versatile watermarking mechanism.
  • Future work: Future work will improve robustness, add semantic-wise outcomes beyond pixel-wise localization, and extend the framework to video, audio, and 3D scenes.These directions are stated as planned extensions of EditGuard.

7. More Implementation Details

The implementation constructs AGE-Set from multiple AIGC editing methods and uses invertible, multi-branch networks to embed and recover localization and copyright watermarks. Training and comparison settings specify the datasets, model components, and recovery pipeline.

  • Dataset preparation: AGE-Set-C uses over 30K 512×512 COCO and CelebA images edited by Stable Diffusion Inpaint, ControlNet, SDXL, RePaint, LaMa, and FaceSwap.Segment Anything extracts tampered regions, and AGE-Set-C separates training and testing images across editing methods.
  • Comparison settings: The comparison covers state-of-the-art tamper-localization networks and pure bit-hiding watermarking methods to evaluate localization and copyright protection.The comparison settings distinguish tamper localization, tamper detection, and copyright protection.
  • Invertible image hiding: Invertible blocks apply DWT-based frequency decoupling and enhanced affine coupling layers to the original image and localization-watermark branches.Dense convolution blocks provide coarse semantic features, while LFIM adds nonlinear transformation and low-cost long-range dependency modeling before inverse DWT reconstruction.
  • Copyright embedding: The copyright bitstream is mapped from {0, 1}^L to {-0.5, 0.5}^L, processed by stacked MLPs, reshaped into multi-scale message features, and fused with image features.The implementation uses downsampling and upsampling paths to combine message information at multiple resolutions.
  • Watermark recovery: A U-shaped decoder extracts the received copyright watermark through an MLP, thresholds it at zero, and converts it back to binary values.The BEM combines the mediated image and copyright streams at multiple levels to produce the dual-encoded container image.
  • Implementation settings: The implementation uses six invertible blocks, eight residual convolution blocks, eight transformer blocks, three U-shaped enhancement layers, and a learnable degradation prompt.A JPEG simulator is also introduced during training.

8. Addtional Experiments

Additional experiments evaluate EditGuard’s efficiency, robustness, watermark flexibility, steganographic security, and comparison with competing image-into-image methods. The results support accurate localization under degradation, adaptable watermarking, and secure dual-watermark embedding.

  • Computational efficiency: EditGuard uses the lowest number of parameters and the second-lowest inference time and FLOPs among compared tamper localization methods.The evaluation uses a 3×512×512 tensor on an NVIDIA 3090Ti.
  • Robustness to degradation: EditGuard maintains accurate localization under Gaussian noise, JPEG compression, Poisson noise, and WeChat degradation, with only slight precision loss as degradation increases.The visualizations remain close to the ground truth across simulated and real-world degradation settings.
  • Localization watermark analysis: EditGuard works with different localization watermark images, demonstrating generalizability across watermark choices.The method performs better with simple localization watermarks.
  • Comparison with image-into-image steganography: EditGuard simultaneously maintains high container-image fidelity, 1D and 2D watermark hiding, accurate secret recovery, and decent robustness, outperforming HiNet and RIIS in the evaluated setting.HiNet degrades under slight degradation, while RIIS sacrifices reconstruction fidelity for robustness.
  • Security analysis: 50% detection probability corresponds to random guessing, and EditGuard is less detectable than HiNet, RIIS, SepMark, and MBRS while remaining slightly below CIN.The comparison uses StegExpose on methods embedding different watermark types.

9. Discussion and Analysis

The discussion explains how EditGuard’s proactive design differs from passive localization and analyzes architectural choices for jointly embedding localization and copyright information. Experiments motivate sequential encoding, parallel decoding, and an asymmetric bit-recovery network.

  • Proactive versus passive localization: Passive localization analyzes posthoc image artifacts, whereas proactive localization detects pre-embedded signals to reveal alterations.EditGuard belongs to the proactive category because it integrates watermark signals before foreseeable manipulation.
  • Comparison with related methods: Unlike prior approaches, EditGuard does not require tampered samples during training and performs zero-shot localization.The discussion contrasts this property with MaLP, which requires tampered samples and a fixed learned template.
  • Sequential encoding and parallel decoding: Parallel encoding of a 2D localization watermark and a reshaped 64-bit watermark achieves strong clean-condition performance but approaches 50% bit accuracy under JPEG compression with Q = 70.Treating bits as an image creates local and fragile hiding that limits robustness.
  • Sequential encoding and parallel decoding: Sequential encoding with parallel decoding lets the image and bit branches be trained flexibly and assigned different degradation layers and robustness levels.The design reduces interference between branches with different hiding capacities.
  • Asymmetric bit network design: The asymmetric encoder-decoder is preferred over a flow model because the flow-based variant has unsatisfactory bit reconstruction robustness and degrades under some conditions.The flow model’s bit reconstruction accuracy and robustness are reported as just above 95%.

10. Limitations and Future Works

The authors identify empirical localization-watermark selection as a current limitation and propose reducing IBSN capacity, extending the framework to other modalities, and adding self-recovery functionality.

  • Current limitation: Localization-watermark selection is currently empirical, motivating future end-to-end learning of an optimal watermark.The authors also plan to replace the three-channel RGB watermark with a grayscale image.
  • Future watermark design: A grayscale localization watermark is intended to reduce IBSN information capacity and further enhance robustness.This is proposed as an alternative to the current three-channel RGB watermark.
  • Future extensions: The framework could be extended to video, hyperspectral data, and 3D scenes, with future work also considering cross-modal verification and self-recovery of original images.The proposed extensions include temporal and audio information for localization and recovery functionality.

11. More Visual Results

Additional visual evaluations show that EditGuard consistently localizes tampering across batch-processed and carefully edited AIGC images. Competing methods generally do not achieve comparable localization accuracy or generalization.

  • Visual localization results: EditGuard consistently produces precise localization on both batch-processed AGE-Set-C images and carefully edited AGE-Set-F images.The evaluations cover the visual results shown in Figures 13–15.
  • AGE-Set-C comparison: EditGuard can pinpoint pixel-wise tampered areas, while other methods often struggle with AIGC-based tampering.This comparison is presented on the constructed AGE-Set-C benchmark.
  • Hard-to-detect AIGC edits: For AIGC edits designed to be difficult for the naked eye to detect, almost all compared methods fail while EditGuard still produces precise tampered masks.The examples use customized models and carefully designed prompts.
Loading 2312.08883v1…