PaperlayerView exact revision on arXiv

Source-linked AI summary

VideoShield: Regulating Diffusion-based Video Generation Models via Watermarking

Runyi Hu, Jie Zhang, Yiming Li, Jiwei Li, Qing Guo, Han Qiu, Tianwei Zhang

arXiv:2501.14195v2cs.CV

TL;DR

Video watermarking has received limited attention, while frame-by-frame post-processing can degrade quality and image-based localization misses temporal changes. VideoShield embeds watermarks during diffusion-based video generation and uses inversion and template bits for extraction and tamper localization. Experiments report effective extraction and localization across T2V and I2V models without compromising video quality, with applicability to generated images.

  • Problem

    Existing methods were not specifically designed for AI-generated videos; framewise watermarking can degrade quality, and image-based localization misses temporal tampering.

    Source: §1 · pp. 1–2

  • Method

    VideoShield embeds watermarks during video generation and uses DDIM Inversion and template bits to support watermark extraction and spatial and temporal tamper localization.

    Source: §1 · p. 2

  • Results

    Across T2V and I2V models, VideoShield effectively extracts watermarks and localizes tampering without compromising video quality, and also localizes tampering in generated images.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The reported approach is adaptable across video and image generation models.

    Source: §4 · p. 10

Abstract

from arXiv · show

Artificial Intelligence Generated Content (AIGC) has advanced significantly, particularly with the development of video generation models such as text-to-video (T2V) models and image-to-video (I2V) models. However, like other AIGC types, video generation requires robust content control. A common approach is to embed watermarks, but most research has focused on images, with limited attention given to videos. Traditional methods, which embed watermarks frame-by-frame in a post-processing manner, often degrade video quality. In this paper, we propose VideoShield, a novel watermarking framework specifically designed for popular diffusion-based video generation models. Unlike post-processing methods, VideoShield embeds watermarks directly during video generation, eliminating the need for additional training. To ensure video integrity, we introduce a tamper localization feature that can detect changes both temporally (across frames) and spatially (within individual frames). Our method maps watermark bits to template bits, which are then used to generate watermarked noise during the denoising process. Using DDIM Inversion, we can reverse the video to its original watermarked noise, enabling straightforward watermark extraction. Additionally, template bits allow precise detection for potential temporal and spatial modification. Extensive experiments across various video models (both T2V and I2V models) demonstrate that our method effectively extracts watermarks and detects tamper without compromising video quality. Furthermore, we show that this approach is applicable to image generation models, enabling tamper detection in generated images as well. Codes and models are available at https://github.com/hurunyi/VideoShield.

1 INTRODUCTION

Video generation’s growing capability and value heighten concerns about content control, while image-oriented approaches do not address video’s temporal and quality needs. VideoShield embeds watermarks during generation and supports extraction and tamper localization; experiments report effectiveness across video models without compromising quality.

1 INTRODUCTION

  • No existing method was specifically designed for AI-generated videos, and framewise watermarking can degrade quality while image-based localization misses temporal changes.The cited discussion also identifies passive localization’s reliance on extensive training and limited transferability.Source: §1 · p. 2
  • The training-free framework embeds watermarks during video generation and uses DDIM Inversion for extraction and template bits for tamper localization.Watermark bits map to Gaussian noise, while template bits support localization across temporal and spatial dimensions.Source: §1 · p. 2
  • VideoShield effectively extracts watermarks and localizes tampering across T2V and I2V models without compromising video quality.The framework is also reported to localize tampering in images generated by T2I models.Source: §1 · p. 2

2 METHODOLOGY

VideoShield links watermark bits, template bits, Gaussian noise, and generated frames through reversible conversions to support embedding, extraction, and tamper localization. Its design uses template bits tied to pixels and frame positions to support spatial and temporal localization.

2.1 DESIGN PRINCIPLES

  • Template bits provide a one-to-one relation with generated pixels and differ across frames, supporting spatial and temporal tamper localization.The design avoids a separate template image requiring extensive training.Source: §2.1 · pp. 2–3

2.4 TAMPER LOCALIZATION

  • The localization pipeline compares inverted bits with template bits, using temporal matching before spatial analysis and hierarchical refinement.The overall framework describes both stages as training-free.Source: Figure 1 · p. 3, §2.4 · p. 4

2.2 OVERVIEW

  • VideoShield maps watermark bits through template bits into Gaussian noise, then denoises it into video frames; DDIM Inversion reverses this chain for extraction.The framework describes the conversions as m ⇔ TP ⇔ ZT ⇔ FR.Source: §2.2 · p. 3, Figure 1 · p. 3

2.3 WATERMARK EMBEDDING AND EXTRACTION

  • Watermark bits are mapped to Gaussian noise via template bits, and video frames are produced by iterative denoising; DDIM Inversion recovers noise for extraction.The noise is generated by truncated sampling conditioned on encrypted template bits, and the paper states it follows a standard Gaussian distribution.Source: §2.3 · pp. 3–4

2.4.1 TEMPORAL TAMPER LOCALIZATION

  • For temporal localization, each inverted frame is compared with template bits at every original frame position, and the highest-scoring match restores its position.Frames without an original match, such as inserted frames, are marked as -1.Source: §2.4.1 · p. 5

2.4.2 SPATIAL TAMPER LOCALIZATION

  • Spatial localization averages comparison bits, thresholds regional scores, and repeats each regional prediction across its area; HSTR trades finer granularity for accuracy by gathering more adjacent bits.Thresholding distinguishes regions using the non-overlapping parts of watermarked and original accuracy distributions; HSTR averages masks across levels.Source: §2.4.2 · pp. 5–6

3 EXPERIMENTS

VideoShield is evaluated for watermark extraction, video quality, and spatial and temporal tamper localization across diffusion video models. The experiments also test its design choices, generality across models and image-generation settings, and robustness to distortions.

3.1 EXPERIMENTAL SETTING

  • Experiments use ModelScope and Stable-Video-Diffusion, with VBench prompts and generated or real-image-conditioned videos, and compare against image watermarking and localization baselines.The evaluations cover watermark bit accuracy and video quality, temporal localization accuracy, and spatial localization metrics.Source: §3.1 · p. 7

3.2 MAIN RESULTS

  • VideoShield embeds more bits with comparable Bit Accuracy and produces noticeably higher-quality videos than baseline watermarking methods.The authors attribute the quality advantage to avoiding post-processing.Source: §3.2 · p. 8
  • VideoShield localizes temporal tampering without a baseline comparison and handles three spatial tamper types where image-oriented baselines struggle on generated videos.Table 2 reports temporal localization accuracy; Table 3 compares spatial localization methods across tamper types.Source: Table 2 · p. 8, Table 3 · p. 8, §3.2 · p. 8

3.3 ABLATION STUDY

  • PTB improves spatial localization over omitting the module; the best tested quantile is 99 for ModelScope and 98 for Stable-Video-Diffusion, while HSTR performs best on average at L=3.The L analysis reports increasing Precision with larger L, but Recall begins to decline beyond 3 for ModelScope and 2 for Stable-Video-Diffusion.Source: §3.3 · p. 9, Table 5 · p. 9, Figure 6 · p. 9

3.5 ROBUSTNESS

  • Under distortions, watermark extraction is more robust than all baselines across almost all types, averaging 0.983 on ModelScope and 0.955 on Stable-Video-Diffusion.For spatial localization, distortion-adjusted thresholds improve robustness to distortions but reduce performance on clean tampered videos; setting L=4 for Stable-Video-Diffusion also enhances robustness.Source: Figure 7 · p. 9, §3.5 · p. 10, Table 8 · p. 10

3.4 GENERALITY

  • VideoShield extracts watermarks and localizes tampering on ZeroScope and I2VGen-XL, and achieves high image tamper-localization performance on Stable Diffusion 1.4, 1.5, and 2.1.The authors report a localization performance gap for Stable-Video-Diffusion relative to the other tested video models, attributing it to lower generated-video quality.Source: §3.4 · pp. 9–10, Table 6 · p. 10, Table 7 · p. 10

4 CONCLUSION

VideoShield uses watermarked noise and template bits with DDIM Inversion for watermark extraction and tamper localization. The surrounding sections describe diffusion video models, watermarking approaches, and tamper-localization methods.

4 CONCLUSION

  • VideoShield maps watermark bits to watermarked Gaussian noise and uses template bits recovered through DDIM Inversion for extraction and localization across video and image-generation models.The conclusion reports effectiveness in both tasks and adaptability across video and image generation models.Source: §4 · p. 10

A.1 DIFFUSION-BASED VIDEO GENERATION MODELS

  • Compared with 2D-VAE/U-Net models’ limited motion and temporal consistency, 3D-VAE/DiT models capture spacetime relationships; DDIMs permit nearly reversible noising and denoising.The paper identifies DDIM Inversion as a key feature for image reconstruction.Source: §A.1 · p. 14

A.2 WATERMARKING

  • AIGC watermarking includes post-processing and in-generation approaches, but no existing in-generation watermarking method was designed for video generation.The paper also describes training-based proactive tamper-localization methods whose post-processing can consume resources and degrade quality; their models were unavailable as baselines.Source: §A.2 · pp. 14–15, §A.3 · p. 15

A.3 TAMPER LOCALIZATION

  • Passive localization learns tampered-region boundaries but may generalize poorly across dataset distributions; prior proactive methods embed information but require training and post-processing.The authors report that EditGuard and V2A-Mark could not be used as comparison baselines because their models were not openly available.Source: §A.3 · p. 15

B RELATIONSHIP WITH GAUSSIAN SHADING

VideoShield distinguishes itself from Gaussian Shading through template bits and video-specific localization contributions. The paper also identifies PRC as an alternative way to implement template bits.

B RELATIONSHIP WITH GAUSSIAN SHADING

  • Template bits, absent from Gaussian Shading, connect watermark embedding, extraction, and tamper localization without training and can extend across images, videos, and other modalities.The paper presents them as a flexible basis for multifunctional watermarking.Source: B RELATIONSHIP WITH GAUSSIAN SHADING · p. 15
  • Gaussian Shading is one template-bit implementation, while PRC can map watermark bits to one-to-one pseudo-random content bits and combine their signs with absolute Gaussian-noise values.The paper describes PRC as an alternative implementation method.Source: B RELATIONSHIP WITH GAUSSIAN SHADING · p. 15
  • VideoShield introduces temporal tamper localization and proposes HSTR and PTB to enhance spatial-localization robustness.The paper frames these as innovations specific to the video modality.Source: B RELATIONSHIP WITH GAUSSIAN SHADING · p. 15

C DATASETS

The datasets appendix lists prompts used for text-to-video generation and real images used for image-to-video models.

C.1 PROMPTS USED FOR T2V MODELS

  • The text-to-video prompt set is organized into five categories.The appendix introduces the five categories as prompts used for video generation.Source: §C.1 · p. 15
  • Listed prompts include animal, activity, and scene descriptions, including a red panda eating leaves, a dog playing soccer, and a riverfront city.Source: §C.1 · pp. 15–16

C.2 REAL IMAGES USED FOR I2V MODELS

D.1 VIDEO QUALITY METRICS CALCULATION DETAILS

The appendix specifies video-quality metrics, temporal and spatial tampering operations, and configurations for generality tests across video and image-generation models.

D.1 VIDEO QUALITY METRICS CALCULATION DETAILS

  • Subject and background consistency use cross-frame DINO and CLIP feature similarities, respectively.The feature vectors are normalized to unit length; the DINO passage identifies dot product as the cosine-similarity operation.Source: §D.1 · p. 18
  • Motion smoothness uses frame-interpolation error, dynamic degree measures the proportion of non-static videos, and imaging quality averages normalized MUSIQ frame scores.The appendix lists these alongside subject and background consistency as the five VBench metrics.Source: §D.1 · p. 18

D.2 TEMPORAL TAMPER

  • Temporal tampering tests include frame drop, frame insert, and frame swap, which alter frame counts and/or order.Drop and insert use a randomly chosen position; the described swap procedure exchanges adjacent frames at regular intervals.Source: §D.2 · pp. 18–19

D.3 SPATIAL TAMPER

  • Spatial tampering tests use Crop&Drop at a 0.5 ratio and inpainting, including object removal with STTN or ProPainter.Crop&Drop simulates outpainting or inpainting; the authors also manually mask main objects in some videos before inpainting.Source: §D.3 · p. 19

D.4 GENERALITY TEST

  • Generality tests cover 200 ZeroScope videos at resolution 256, 200 I2VGen-XL videos at 512, and 500 images from three Stable Diffusion versions at 512.The I2VGen-XL videos use images generated by Stable Diffusion 2.1; image tamper tests use Crop&Drop.Source: §D.4 · p. 19

D.5 DISTORTIONS FOR TESTING ROBUSTNESS

The evaluation tests extraction and localization under specified video and image distortions, then assesses video quality with objective metrics and a user study. VIDEOSHIELD receives the most user-study votes, while post-processing causes greater degradation on Dynamic Degree and Imaging Quality.

D.5 DISTORTIONS FOR TESTING ROBUSTNESS

  • The robustness tests include MPEG-4, frame averaging, frame swapping, and eight image distortions applied to each video frame.The extraction tests include frame swapping; the spatial-localization tests instead include MPEG-4 and frame averaging alongside image distortions.Source: §D.5 · p. 19

E.1 VIDEO QUALITY

  • Post-processing causes greater degradation in Dynamic Degree and Imaging Quality, although it sometimes slightly improves Background Consistency and Motion Smoothness.The authors attribute the continuity-metric increases to blurring from watermark embedding.Source: §E.1 · p. 20, Table 9 · p. 20
  • VIDEOSHIELD receives the most user-study votes for both MS and SVD, followed by RivaGAN, while CIN, PIMoG, and SepMark receive very few.The study randomly selected 10 videos from each of 200 videos generated by MS and SVD and asked 24 users to compare quality.Source: §E.1 · p. 20, Table 10 · p. 20

E.2 WATERMARK ROBUSTNESS

The section compares extraction accuracy under distortions, reports results for different inference and inversion configurations, and tests extraction after spatial tampering. Extraction accuracy remains close to 100% after the tested spatial tampering.

E.2 WATERMARK ROBUSTNESS

  • Table 11 provides extraction-accuracy comparisons across watermarking methods and different distortions.The supplied passage describes the table’s scope but does not state its cell values or identify a best method.Source: §E.2 · p. 20, Table 11 · p. 20

E.3 IMPACT OF DIFFERENT INFERENCE AND INVERSION CONFIGURATIONS

  • Table 12 reports results across inference and inversion configurations; Stable-Video-Diffusion is omitted for non-default schedulers because it cannot generate high-quality video with them.Gray cells mark the default setting.Source: Table 12 · p. 21, §E.3 · p. 21

E.4 WATERMARK EXTRACTION FROM SPATIAL TAMPERED VIDEOS

  • Watermark extraction accuracy remains close to 100% after various types of spatial tampering.Table 13 reports extraction accuracy on spatially tampered videos.Source: §E.4 · p. 21, Table 13 · p. 21

E.5 ROBUSTNESS AGAINST MORE DISTORTIONS

Tests find that severe compression can degrade performance, while color adjustments leave it high; VIDEOSHIELD outperforms EditGuard on five models, and fewer inversion steps can reduce runtime. Distillation experiments on images suggest a possible efficiency path for video models.

E.5 ROBUSTNESS AGAINST MORE DISTORTIONS

  • Extraction and localization performance for MS and SVD degrade significantly only under extreme compression above CRF 28, while color-adjustment performance remains high.Adjusting the threshold substantially improves spatial localization under compression, when video quality is already severely degraded.Source: §E.5 · p. 21, Table 14 · p. 22

E.6 COMPARISON WITH EDITGUARD

  • VIDEOSHIELD outperforms EditGuard on five tested models, except SVD and I2VGen.The comparison uses Crop&Drop tampering on videos and images; the authors attribute lower SVD performance to generated-content quality.Source: §E.6 · p. 21, Table 15 · p. 22

E.7 COMPUTATION OVERHEAD

  • Reducing inversion steps can substantially lower runtime with only a slight performance sacrifice; inversion is the main concentration of GPU memory use and runtime overhead.The overhead measurements use one NVIDIA RTX A6000 GPU in FP16 mode, with batch size 1.Source: §E.7 · p. 22, Table 16 · p. 22
  • Distillation reduces required inversion steps while maintaining comparable image-localization performance, suggesting a possible efficiency approach for video diffusion models.The experiments compare SD 1.5 with its distilled version, SD 1.5 Turbo.Source: §E.7 · p. 22

F MORE ANALYSIS

The analysis examines how PTB distribution choices, model inversion accuracy, video resolution, and distortion-adjusted thresholds relate to localization performance. It reports that higher comparison accuracy and resolution correspond to better spatial localization, while threshold adjustment can improve robustness under distortion.

F MORE ANALYSIS

  • Table 17 compares inversion-runtime reduction for SD 1.5 Turbo while maintaining VIDEOSHIELD performance.SD 1.5 Turbo is identified as a distilled version of SD 1.5 that enables generation in significantly fewer steps.Source: Table 17 · p. 23
  • With ModelScope at μ = 2, k = 100 produces greater overlap between watermarked and original accuracy ranges than k = 99, potentially reducing localization performance.The reported ranges are [0.28, 1.00] and [0.06, 0.84] for k = 100, versus [0.56, 0.93] and [0.28, 0.68] for k = 99.Source: F MORE ANALYSIS · pp. 22–23, Table 18 · p. 23
  • Higher comparison-accuracy distributions correspond to better spatial tamper localization, and distributions are similar among models of the same type.The analysis covers seven models and states that these patterns agree with Tables 6 and 7.Source: F MORE ANALYSIS · p. 23, Figure 9 · p. 23
  • VIDEOSHIELD performance is positively correlated with the video model’s average inversion accuracy.Figure 10 relates inversion accuracy to performance in three dimensions; its inversion-accuracy measure averages points in the comparison-bits matrix.Source: F MORE ANALYSIS · p. 23, Figure 10 · p. 24
  • For ZeroScope videos, spatial localization improves as resolution increases; the authors attribute this to more detail improving template integration and inversion accuracy.Table 19 reports performance at different resolutions.Source: F MORE ANALYSIS · p. 24, Table 19 · p. 24
  • Under distortion, adjusting PTB thresholds improves spatial-localization robustness but slightly reduces performance on clean tampered videos.For Stable-Video-Diffusion, the authors report better distorted-video results with L = 4, attributing this to its inherently low inversion accuracy.Source: F MORE ANALYSIS · p. 24

G.1 MAIN RESULTS

The appendix reports spatial tamper-localization evaluations across hierarchical levels and distortion settings, and illustrates generality across video and image models.

G.1 MAIN RESULTS

  • Spatial localization is evaluated as hierarchical level L changes on clean and distorted videos, with examples at different levels.The accompanying figures also plot performance against distortion using clean and distortion thresholds.Source: §G.1 · p. 24, Figure 12 · p. 25, Figure 14 · p. 25
  • Accuracy distributions compare local values µ across original, watermarked, and distorted watermarked videos.Additional examples show localization for different spatial tamper types on ModelScope and Stable-Video-Diffusion videos.Source: §G.1 · p. 24, Figure 11 · p. 25

G.2 GENERALITY

  • Spatial localization results are shown for other video models and for images generated by different text-to-image models.The video examples cover ZeroScope and I2VGen-XL; the image examples use different Stable Diffusion versions.Source: §G.2 · p. 25

G.3 FAILURE CASES OF STABLE-VIDEO-DIFFUSION

Sampling-configuration adjustments improve localization in Stable-Video-Diffusion examples, while failures arise when generated video content resembles the conditional image.

G.3 FAILURE CASES OF STABLE-VIDEO-DIFFUSION

  • When generated regions replicate conditional-image content, template noise is not effectively denoised, so inversion mismatches template bits and marks regions as tampered.The paper attributes this limitation to the model's difficulty generating video with seamless overall content motion.Source: §G.3 · pp. 25–26
  • Sampling-configuration adjustments generate more original video content rather than merely copying conditional images, improve integration of template-bit noise with relevant video content, and improve inversion reconstruction, yielding better localization than the original template bits.Figure 24 shows default settings, Inference Step = 50, and Image Guidance = 2 across frames 1, 6, 11, and 16.Source: §G.3 · p. 26, Figure 24 · p. 31
  • Figures 22 and 23 show poor spatial-localization cases on Stable-Video-Diffusion videos.Other visual examples cover STTN and ProPainter tampering on ModelScope and Stable-Video-Diffusion, plus Crop&Drop tampering on other video and image models.Source: Figure 15 · p. 26, Figure 16 · p. 26, Figure 17 · p. 27
Loading 2501.14195v2…