Source-linked AI summary
LETHE: Defending against Unauthorized Image-to-Video Generation via Value Diversity Collapse
Boheng Li, Yueming Xing, Yiming Li, Jie Zhang, Tianwei Zhang
TL;DR
Modern I2V models can turn personal images into realistic videos without consent, while the factor governing reference reading remains insufficiently understood. LETHE uses convex-geometric analysis to target reference-value diversity with value collapse and attention attraction. It outperforms evaluated baselines across the reported settings and transfers to models not used in optimization.
Problem
Modern I2V models can produce realistic, subject-consistent videos from personal images, while existing protections struggle against DiT-based models at commonly used perturbation budgets.
Source: §1 · p. 1Method
LETHE combines value collapse, which homogenizes reference values, with attention attraction, which draws subsequent frames toward those collapsed tokens.
Source: §1 · p. 2Results
LETHE achieves the best performance across all 16 settings spanning two models, two datasets, and four metrics, and transfers to models not involved in optimization.
Source: §1 · p. 3, §5.2 · p. 7Takeaways & Limitations
Reference-value diversity is a useful and efficient target for protective perturbations, and LETHE remains effective against transformations and various attacks.
Source: §1 · pp. 2–3The formulation bounds the released-image perturbation by an ℓ∞ budget, while allowing attacker-side transformations, arbitrary prompts, and random noise.
Source: §4 · p. 5
Abstract
from arXiv · showhide
Image-to-video (I2V) diffusion models can animate a single reference image into high-quality, subject-consistent videos, posing threats of misuse as anyone can turn personal photos into fabricated videos without consent. A promising solution is to proactively add an adversarial perturbation to the image before release, such that videos generated from it would be unfaithful or visually corrupted. However, due to an insufficient understanding of what governs how subsequent frames read the reference image in modern I2V models, existing protections struggle to offer strong protection under common perturbation budgets. In this paper, we approach the problem by exploring this underlying reading mechanism. Through a convex geometric analysis, we theoretically identify the spread of the reference value vectors as a key factor governing this reading process, and empirically find that even a mild contraction of this spread can severely degrade the generated video, making it an efficient lever for protection. Guided by this insight, we propose LETHE, a novel protection method that directly disrupts this reading process. LETHE pairs two complementary objectives, where a value collapse objective drives the reference value vectors toward uniformity, while an attention attraction objective draws subsequent frames toward these collapsed tokens to amplify corruption. We further construct two human-centric I2V protection benchmarks and conduct extensive experiments with multiple modern I2V models, which demonstrate the state-of-the-art protection performance and versatility of LETHE.
1 INTRODUCTION
I2V models can turn shared personal images into realistic videos without consent, while existing protections struggle to disrupt modern models at common perturbation budgets. LETHE targets how subsequent frames read reference information and is evaluated on two human-centric protection benchmarks.
1 INTRODUCTION
- I2V systems can animate a personal photo into a subject-consistent video depicting prompted actions, creating a risk of realistic nonconsensual fabrications.Protective perturbations add carefully crafted noise before image sharing to mislead generative models and block forgeries at the source.Source: §1 · p. 1
- At 16/255 on Wan2.1-I2V-14B, existing methods leave identity recognizable despite local artifacts or mild visual changes.The comparison is shown in Figure 1.Source: Figure 1 · p. 2, §1 · p. 2
- LETHE contracts the spread of reference value vectors and attracts subsequent-frame attention to collapsed tokens to disrupt reference reading.The paper identifies value-vector spread as an upper bound on reference-readout diversity; even mild contraction can severely degrade generated video.Source: §1 · p. 2
- The authors construct two human-centric benchmarks by re-annotating CelebV-Text and Kinetics-700 with curated reference frames and human-verified motion prompts.They evaluate LETHE on multiple I2V models.Source: §1 · p. 2
- LETHE achieves the best performance across all 16 settings spanning two models, two datasets, and four metrics.It also transfers to models not involved in optimization and remains effective against transformations and various attacks.Source: §1 · p. 3
2 RELATED WORK
I2V models have shifted from UNet to DiT architectures, whose spatiotemporal attention supports improved fidelity and consistency. Prior image and video protections offer precedents, but the internal reference-reading factor in modern I2V remains unclear.
2 RELATED WORK
- Recent DiT-based I2V models use full spatiotemporal attention and interact with reference-image tokens to acquire visual information for subsequent frames.Earlier UNet models have more limited generation fidelity and temporal consistency.Source: §2 · p. 3
- Image-protection methods such as PhotoGuard and AdvDM perturb VAE latents or maximize denoising loss to impede faithful image editing and customization.Later work targets efficiency or particular protection scenarios.Source: §2 · p. 3
- Prior I2V protections target encoders, intermediate features, or denoising, but the factor governing reference reproduction through deep spatiotemporal attention remains unclear.This uncertainty can leave protection objectives focused on proxies only indirectly related to what subsequent frames read.Source: §2 · p. 3
3 MOTIVATING ANALYSIS
The analysis traces reference-image reading to attention-weighted value vectors and shows that their geometric spread constrains readout diversity. Contracting that spread markedly degrades generated videos, motivating it as a protection target.
Theoretical Analysis
- Attention readouts are convex combinations of reference values, so subsequent-frame tokens are confined to their convex hull.The nonnegative attention weights sum to one, mapping the probability simplex into the values’ convex hull.Source: §3 · p. 4
- The minimum-enclosing-ball radius R(V) of reference values bounds readout diversity; shrinking it lowers the ceiling regardless of attention weights.If all reference values coincide, normalized readouts in that head are identical.Source: §3 · p. 4
Empirical Validation
- The intervention contracts each value toward the minimum-enclosing-ball center while leaving other components unchanged; equal-magnitude random perturbations degrade videos less.The intervention is applied at every layer in the described setup.Source: §3 · pp. 4–5
- A 20% contraction (γ = 0.8) introduces visible facial artifacts, while 30% (γ = 0.7) collapses videos into unrecognizable color blocks.The figure compares generated frames and ArcFace identity similarity across contraction ratios.Source: Figure 2 · p. 5, §3 · p. 5
4 METHODOLOGY
LETHE optimizes a bounded image perturbation to collapse reference-token values and attract subsequent-frame attention to those tokens. The paired objectives aim to leave later frames with little distinguishable reference information and limited capacity to compensate through other tokens.
4 METHODOLOGY
- The value collapse loss penalizes reference-value energy not shared across tokens, pushing their values toward a single vector.Unlike the minimum-enclosing-ball radius objective, its gradient involves all reference values, and its normalized terms support aggregation across layers and heads.Source: §4 · p. 5
- Because subsequent-frame tokens divide attention across the sequence, LETHE raises their average logits to reference keys to increase attention allocated to reference tokens.The queries are treated as constant, so the perturbation acts through the keys; QK normalization bounds the average logit.Source: §4 · pp. 5–6
- LETHE jointly optimizes value collapse and attention attraction under an ℓ∞ perturbation budget, aiming to degrade generated videos persistently.The value objective homogenizes reference values, while the attention objective steers subsequent-frame readouts toward those collapsed tokens.Source: §4 · pp. 5–6
- The perturbation is optimized with MI-FGSM for 100 iterations, with α balancing the two objectives under the ε budget.The objectives are imposed over selected layers and heads of a surrogate model.Source: §4 · pp. 5–6
5 EXPERIMENTS
Across two human-centric benchmarks and multiple I2V models, LETHE is evaluated against existing protection methods. It leads the white-box results and remains effective in transfer and robustness tests.
5.1 EXPERIMENTAL SETUP
- Experiments use Wan2.1 and CogVideoX, four additional transfer models, five baselines, and two curated human-centric datasets, assessing identity preservation and video quality under a default 16/255 perturbation budget.The datasets use CelebV-Text and Kinetics-700 videos, with a clearly visible face frame and annotated appearance-and-motion prompts selected for each video.Source: §5.1 · pp. 6–7, Table 1 · p. 7
5.2 OVERALL PERFORMANCE
- LETHE led all 16 model–dataset–metric settings, outperforming evaluated baselines on Wan2.1 and CogVideoX.On CelebV-Text with Wan2.1, LETHE reduced ISM-A from 0.817 on clean images to 0.428 and QAF from 0.912 to 0.397, versus second-best baseline values of 0.524 and 0.589.Source: Table 1 · p. 7, §5.2 · p. 7
- Qualitative comparisons show LETHE erasing the subject on Wan2.1 and smearing it into unrecognizable textures on CogVideoX.The comparisons also show more complete degradation of identity and visual structure than the baselines.Source: Figure 3 · p. 7, §5.2 · p. 7
- Under MLLM evaluation, LETHE scored highest on subject preservation, structural consistency, dynamic consistency, and artifact suppression across both datasets.The evaluation uses a failure-oriented protocol and inverts scores so higher values indicate stronger protection.Source: Figure 4 · p. 8, §5.2 · p. 8
5.3 TRANSFERABILITY & ROBUSTNESS
- Wan2.1-crafted perturbations lower all metrics by ∼40% on LightX2V and AccVideo; on SVD and LTX-Video they yield ∼30% ISM reduction and outperform Anti-Prompt on all four metrics.The latter models differ substantially from Wan2.1 in architecture.Source: Figure 5 · p. 8, §5.3 · p. 8
- Under input transformations and purifiers, LETHE remains effective: ISM-A rises from 0.43 to ∼0.56 after purifiers, versus ∼0.80 on clean images; metrics vary by no more than ∼0.03 across prompts and seeds.The tested purifiers include IMPRESS and GrIDPure, which are designed to remove protective perturbations.Source: Figure 5 · p. 8, §5.3 · p. 9
5.4 ABLATION STUDIES
- Adding attention attraction to value collapse further lowers all four metrics across both datasets; value collapse alone reduces ISM-A from 0.817 to 0.590 and QAF from 0.912 to 0.522 on CelebV-Text.The table reports means across three random seeds.Source: Table 2 · p. 9, §5.4 · p. 9
- All four metrics decrease monotonically as ε grows; at ε = 8/255, LETHE reaches ISM-A 0.58 and QAV 0.69, comparable to several strong baselines at ε = 16/255.The figure’s shaded bands denote one standard deviation over three random seeds.Source: Figure 6 · p. 9, §5.4 · p. 9
6 CONCLUSION
Conclusion links attention-based reference reading to value diversity: contracting reference-value spread disrupts generation, motivating LETHE’s paired losses. The ethics statement limits data release to annotations and identifiers while reporting licensed, non-commercial source use.
6 CONCLUSION
- Across multiple I2V models and two re-annotated human-centric datasets, LETHE shows strong performance in various settings.Source: §6 · p. 9
- Attention primarily carries reference-frame visual content to later frames, and reference-value spread bounds readout diversity; contracting that spread disrupts generation.Source: §6 · p. 9
- LETHE combines value-collapse loss, which suppresses reference-value diversity, with attention-attraction loss, which draws later-frame attention to collapsed tokens.Source: §6 · p. 9
ETHICS STATEMENT
- Evaluation uses CelebV-Text and Kinetics-700 non-commercially; re-annotation adds text only, and release is limited to annotations, video identifiers, and frame indices.Qualitative examples are limited to those needed to illustrate the protection effect.Source: ETHICS STATEMENT · p. 10