PaperlayerView exact revision on arXiv

Source-linked AI summary

SepMark: Deep Separable Watermarking for Unified Source Tracing and Deepfake Detection

Xiaoshuai Wu, Xin Liao, Bo Ou

arXiv:2305.06321v2cs.CVcs.MMeess.IV

TL;DR

Passive Deepfake forensics has not addressed preventive measures for pristine faces, while existing approaches face robustness limits and separate source-tracing and detection needs. SepMark embeds one watermark for two decoders with different robustness levels. Experiments report that this supports trusted-source tracing and alteration detection across typical Deepfakes.

  • Problem

    Passive forensics has not considered preventive measures for pristine faces, while robust and fragile watermarking alone cannot jointly provide reliable source tracing and Deepfake detection.

    Source: §1 · p. 2

  • Method

    SepMark embeds one watermark with a single encoder and uses a robust Tracer and selectively sensitive semi-robust Detector to extract it at different robustness levels.

    Source: §1 · p. 3

  • Results

    Experiments report that the robust decoder resists numerous distortions while the semi-robust decoder is selectively fragile to malicious Deepfake distortions, supporting source tracing and alteration detection.

    Source: §1 · p. 3, §5 · p. 8

  • Takeaways & Limitations

    The paired decoders provide a proactive-forensics approach for tracing the trusted source of a marked face and detecting whether it was altered.

    Source: §1 · p. 3

    Further work is needed to model Deepfake distortions more generally and generalize watermarking to unseen Deepfakes.

    Source: §5 · p. 8

Abstract

from arXiv · show

Malicious Deepfakes have led to a sharp conflict over distinguishing between genuine and forged faces. Although many countermeasures have been developed to detect Deepfakes ex-post, undoubtedly, passive forensics has not considered any preventive measures for the pristine face before foreseeable manipulations. To complete this forensics ecosystem, we thus put forward the proactive solution dubbed SepMark, which provides a unified framework for source tracing and Deepfake detection. SepMark originates from encoder-decoder-based deep watermarking but with two separable decoders. For the first time the deep separable watermarking, SepMark brings a new paradigm to the established study of deep watermarking, where a single encoder embeds one watermark elegantly, while two decoders can extract the watermark separately at different levels of robustness. The robust decoder termed Tracer that resists various distortions may have an overly high level of robustness, allowing the watermark to survive both before and after Deepfake. The semi-robust one termed Detector is selectively sensitive to malicious distortions, making the watermark disappear after Deepfake. Only SepMark comprising of Tracer and Detector can reliably trace the trusted source of the marked face and detect whether it has been altered since being marked; neither of the two alone can achieve this. Extensive experiments demonstrate the effectiveness of the proposed SepMark on typical Deepfakes, including face swapping, expression reenactment, and attribute editing.

1 INTRODUCTION

Deepfake forensics faces limits in robustness and in addressing both source tracing and alteration detection: robust watermarks can survive Deepfakes, while fragile ones lose source information. SepMark instead embeds one watermark for extraction by two decoders with different robustness, and experiments report that together they support tracing and detection.

1 INTRODUCTION

  • Existing passive forensics may fail to generalize to unseen forgery methods, and robustness under post-processing or adversarial perturbations remains unclear.The paper also contrasts proactive defenses that disrupt Deepfake creation with its source-tracing focus.Source: §1 · p. 2
  • A robust watermark can survive Deepfake manipulation, preserving source identification but making genuine and forged images hard to distinguish.The paper says robust watermarking alone cannot detect Deepfakes while tracing trusted sources.Source: §1 · p. 2
  • Fragile watermarking can indicate malicious alteration, but its disappearance prevents source tracing and can leave forged and pristine images indistinguishable without prior knowledge.The paper concludes fragile watermarking alone cannot always trace the trusted source or detect Deepfakes.Source: §1 · p. 2
  • SepMark uses one encoder to embed one watermark that two decoders extract at different robustness levels, unifying source tracing and Deepfake detection.The paper presents this as a new deep-separable-watermarking paradigm.Source: §1 · p. 3
  • Experiments on face images report that the robust decoder withstands numerous distortions while the semi-robust decoder is selectively fragile to malicious Deepfake distortions.Together, the decoders are reported to trace the trusted source and detect whether a marked face has been altered.Source: §1 · p. 3

2 RELATED WORK

Related work covers passive Deepfake forensics, proactive defenses, and deep watermarking. These approaches provide context for SepMark's combination of distortion-sensitive detection and watermark-based source tracing.

2.1 Deepfake Passive Forensics

  • Self-supervised passive-forensics methods synthesize Deepfakes using prior cues, including warping, blending boundaries, source-feature inconsistencies, and identity consistency.Their reliance on prior knowledge in synthetic-data production motivates more general modeling toward unseen forgeries.Source: §2.1 · p. 3

2.2 Deepfake Proactive Defense

  • Proactive defenses add imperceptible signals before manipulation and use adversarial perturbations either to disrupt or nullify Deepfake generation.The paper notes that visible artifacts may escape passive detectors, while perturbation-removal methods can permit reconstruction.Source: §2.2 · p. 3

2.3 Deep Watermarking

  • Deep blind watermarking uses neural encoders and decoders with distortion modeling to improve robustness, including differential and black-box distortions.The surveyed approaches include physical cross-media distortions; the paper characterizes deep fragile watermarking as largely unexplored.Source: §2.3 · p. 3

3 PROACTIVE FORENSICS: SEPMARK

SepMark marks pristine faces in advance and uses two separately decoded versions of one watermark to trace trusted sources and detect later manipulation. Its architecture combines an encoder, distortion pool, separable decoders, and discriminator.

3.1 Problem Statement

  • Only the Tracer and Detector together support source tracing and detection: their outputs distinguish genuine marked faces from those manipulated after marking.The Tracer retains the source message under common or malicious distortions, while the Detector returns the correct message under common distortions but an incorrect one under malicious distortions.Source: §3.1 · p. 4

3.2 Model Architecture

  • SepMark uses one encoder, a random forward noise pool, two separable decoders, and an adversary discriminator to train end to end.The encoder produces a marked image; the noise pool generates distorted images; the decoders extract messages at different robustness levels; and the discriminator classifies pristine versus encoded image patches.Source: §3.2 · pp. 4–5

3.2.1 Single Encoder.

  • The single encoder combines the cover image with a redundant representation of the message, then uses a skip connection and 1 × 1 convolution to produce the encoded image.Its UNet-like structure processes the cover image through downsampling and upsampling stages, with message features injected at multiple resolutions.Source: §3.2.1 · p. 5

3.2.2 Random Forward Noise Pool.

  • RFNP randomly samples distortions for each mini-batch and uses detached real-distortion differences to construct differentiable noised images.The pool forms batches using common and malicious distortions together, common distortions only, or malicious distortions only.Source: §3.2.2 · p. 5

3.2.3 Separable Decoders.

  • The Tracer and Detector share a structural design but not parameters, and their training objectives give them different robustness to distortions.Tracer is optimized to recover the embedded message under arbitrary distortions; Detector is optimized to recover it under common distortions and differ from it under malicious distortions.Source: §3.2.3 · p. 5

3.2.4 Adversary Discriminator.

  • Adversarial training supervises encoded-image visual quality, while the relativistic average discriminator estimates whether the cover image is more pristine than the encoded image.The discriminator is a fully convolutional PatchGAN that classifies image patches as pristine or encoded.Source: §3.2.4 · p. 5, §3.3 · p. 5

3.3 Loss Functions

  • The loss functions preserve visual similarity, train the Tracer to recover the embedded message under arbitrary distortions, and train the Detector to recover or reject it selectively.Detector’s objectives target the embedded message under common distortions and zero under malicious distortions; the paper reports that a simple L2 constraint suffices for end-to-end training.Source: §3.3 · p. 5

4 EXPERIMENTS

SepMark is evaluated for visual quality, robustness to common and malicious distortions, and the effects of design choices. The results support using its two decoders together: the Tracer remains robust while the Detector is selectively fragile to malicious distortions.

4.1 Implementation Details

  • Experiments use CelebA-HQ faces and three Deepfake types, with robust and fragile watermarking methods as baselines.The study also evaluates generalizability on CelebA and COCO; images are tested at 128 × 128 and 256 × 256 resolutions.Source: §4.1 · p. 6

4.2 Visual Quality

  • SepMark slightly outperforms PIMoG and substantially outperforms MBRS and FaceSigns on PSNR, SSIM, and LPIPS, with no clear visible artifacts.CIN has impressive objective visual quality, but its architecture is not flexible enough for SepMark and its robustness is non-ideal under quantization noise.Source: Table 1 · p. 6, §4.2 · p. 7

4.3 Robustness Test

  • Under common and malicious distortions, Tracer remains robust while Detector approaches 50% BER for malicious distortions, supporting their complementary roles.Under common distortions, both decoders extract nearly the same message; under malicious distortions, their extracted messages mismatch.Source: Table 3 · p. 7, Table 2 · p. 7, §4.3 · pp. 7–8
  • Cross-dataset testing shows minor BER and PSNR fluctuations on CelebA, while a face-trained model also works on COCO natural images.The model trained at 128 × 128 can also work on arbitrary-size inputs, which the authors attribute to interpolation when fusing the watermark into image features.Source: §4.3 · p. 8

4.4 Ablation Study

  • A smaller embedded-message range α improves visual quality but reduces robustness; a larger α strengthens robustness while lowering visual quality.Two-stage decoder training is less robust than SepMark’s end-to-end training, and sequential robust-then-fragile watermarking can impair robust decoding and visual quality.Source: §4.4 · p. 8

5 CONCLUSION

SepMark uses one watermark with two separable decoders to support source tracing and Deepfake detection. The authors report encouraging results on typical Deepfakes and identify generalizing to unseen Deepfakes as future work.

5 CONCLUSION

  • SepMark is an end-to-end deep separable watermarking architecture for proactive forensics that combines source tracing with Deepfake detection.Source: §5 · p. 8
  • Its robust decoder resists numerous distortions, while its semi-robust decoder is selectively sensitive to malicious distortions.Source: §5 · p. 8
  • Experiments show an encouraging start on typical Deepfakes; further work is needed to model Deepfake distortions more generally and generalize to unseen Deepfakes.Source: §5 · p. 8

A.1 Additional Visual Examples

Figure 6 presents a cover image, its encoded version, a noised version, and two residual signals for comparison.

A.1 Additional Visual Examples

  • Figure 6 includes the cover image and its encoded version as the first two visual inputs.Source: Figure 6 · p. 11
  • The third image is the noised version of the encoded image sequence.Source: Figure 6 · p. 11
  • The final two images show residual signals of N for the cover-versus-encoded and noised-versus-encoded comparisons.Source: Figure 6 · p. 11

A.2 Additional Experimental Results

The appendix adds tests of pristine images, arbitrary input sizes, and robustness across distortions and datasets, alongside further visual examples.

A.2 Additional Experimental Results

Related papers

Loading 2305.06321v2…