PaperlayerView exact revision on arXiv

Source-linked AI summary

FADE: Frame-Aware Diffusion-Transformer-based Multi-Concept Erasure for Video Unlearning

Yuchen Li, Kaiyuan Deng, Chaoran Feng, Zhenyu Tang, Li Yuan

arXiv:2610.03980v2cs.CV

TL;DR

Existing T2V erasure methods can leave erased concepts in isolated frames and are usually evaluated one target at a time. FADE combines a joint closed-form edit with frame-aware per-concept adapters and prompt-similarity routing. It reports 4.9% residual object accuracy versus 15.5% for the strongest baseline, with its advantage extending across judges, prompts, identities, and backbones.

  • Problem

    Existing T2V erasure methods apply frame-agnostic suppression that can leave concepts resurfacing in some frames, and are generally evaluated on one target concept or category at a time.

    Source: §1 · p. 2

  • Method

    FADE combines a joint closed-form key/value edit with per-concept adapters gated by frame and timestep, trained with other targets as negatives and composed by a prompt-similarity router.

    Source: §1 · p. 2

  • Results

    4.9% residual object accuracy versus 15.5% for the strongest of eight baselines, with the VBench average within 0.9% of the unedited model and the lead carrying across judges, compositional prompts, 30 identities, and four backbones.

    Source: §1 · p. 3, §6 · p. 10

  • Takeaways & Limitations

    FADE’s reported lead extends beyond object erasure on one backbone to compositional prompts, 30 identities, and four backbones.

    Source: §6 · p. 10

    The evaluation covers up to 30 simultaneously erased concepts and clips of up to 81 frames; larger concept sets, incremental additions, longer clips, and autoregressive generators remain future work.

    Source: §5 · p. 10

Abstract

from arXiv · show

Text-to-video (T2V) diffusion models can reproduce copyrighted, violent, or explicit content, which motivates concept erasure: removing designated concepts from a pretrained model while preserving its behavior on everything else. Existing T2V erasure methods leave two problems open. Their frame-agnostic suppression can leave isolated frames in which an erased concept resurfaces, a frame-reactivation gap that clip-level averages obscure; and they are usually evaluated with one target concept or category at a time. We propose Frame-Aware Diffusion Erasure (FADE), a multi-concept video unlearning framework. FADE first applies a joint closed-form key/value edit that suppresses all target concepts, then trains per-concept frame-aware low-rank adapters whose strength is gated by the frame index and the denoising timestep to remove residual per-frame leakage. Each adapter is trained with the other targets' prompts as hard negatives, which keeps the concept-specific components of different adapters well separated, and a similarity-based soft router combines the adapters according to the prompt. With 16 concepts (objects, artistic styles, and nudity) erased from a single Wan2.1-T2V-1.3B backbone, FADE reduces the residual accuracy on the object benchmark to 4.9%, against 15.5% for the strongest of eight baselines, while keeping the VBench average within 0.9% of the unedited model. The ranking is unchanged under a VLM judge and a blinded human study, and the advantage over the strongest baseline carries over to prompts that combine several erased concepts, to 30 simultaneously erased celebrity identities, and to Wan2.1-T2V-14B, CogVideoX-2B, and HunyuanVideo-1.5.

1 Introduction

T2V’s ability to generate copyrighted or harmful content motivates removing designated concepts while preserving other generation. Existing methods leave frame-level resurgence and simultaneous multi-concept erasure unresolved; FADE combines frame-aware suppression with multi-concept composition.

1 Introduction

  • T2V models’ ability to generate visually compelling clips also enables copyrighted, violent, disinformation, and sexually explicit content, motivating concept erasure that preserves other generation.The paper frames machine unlearning as removing designated concepts while preserving generative capacity elsewhere.Source: §1 · pp. 1–2
  • Existing T2V erasure methods use frame-agnostic suppression that can leave a concept resurfacing in a few frames, while evaluations generally target one concept or category at a time.Clip-level metrics can average away mid-clip reactivation; training one shared adapter on all targets severely degrades generation.Source: §1 · p. 2
  • FADE combines a joint closed-form edit with frame- and timestep-gated per-concept adapters and similarity-based composition at inference.The adapters target residual per-frame leakage; each is trained to act as the identity on prompts for the other erased concepts.Source: §1 · p. 2
  • The frame-reactivation gap measures the difference between peak-frame and clip-mean concept presence, exposing failures that clip-level accuracy can miss.Removing FADE’s frame conditioning leaves clip-mean accuracy almost unchanged but triples the gap.Source: §1 · p. 2
  • 4.9% residual object accuracy versus 15.5% for the strongest of eight baselines, with VBench average within 0.9% of the unedited model and leads across judges, prompts, identities, and backbones.The reported evidence includes compositional prompts, 30 identities, and four backbones.Source: §1 · p. 3

2 Related Work

Prior concept-erasure work spans several image-generation approaches and video-generation methods, but T2V methods generally suppress concepts uniformly across frames and evaluate one target at a time. FADE combines frame-aware, multi-concept erasure with separately trained experts and prompt-based composition.

Concept erasure for image generation

  • T2I erasure includes fine-tuning, closed-form editing, adapters, inference-time steering, and neuron pruning; MACE combines closed-form refinement with per-concept LoRAs fused into one model.FADE instead keeps per-concept experts separate, gates them by frame and timestep, and composes them at inference.Source: §2 · p. 3

Concept erasure for video generation

  • T2V methods use inference-time intervention, fine-tuning, or training-free weight updates, but apply the same suppression to every frame and evaluate one target concept or category at a time.FADE adds frame- and timestep-dependent suppression and multi-concept composition.Source: §2 · p. 3

Adapter composition

  • Unlike learned-gate or gradient-free LoRA composition, C-MoE uses a fixed prompt-similarity rule, with experts trained against one another’s concepts as negatives to keep their sum selective.Source: §2 · p. 3

3 Method

FADE combines a joint closed-form edit with frame-aware per-concept adapters and prompt-based expert routing to erase multiple concepts from one video diffusion model. It defines a peak-versus-mean diagnostic for frame reactivation and uses temporal gating and training objectives to address residual leakage.

3.1 Problem setup and diagnostic

  • The Reactivation Gap is the difference between peak-frame and clip-mean concept presence, exposing resurfacing that clip-level averages can hide.Multi-concept erasure also requires forgetting co-occurring targets while preserving neutral and designated preserve-set generations.Source: §3.1 · p. 4

3.2 Concept Redirect

  • Concept Redirect jointly edits text-conditioning key/value projections, averaging motion-paraphrased concept embeddings and tapering edit strength during texture refinement.The edit is frame-agnostic and, alone, leaves target concepts largely visible; it moves the backbone toward the erasure target for subsequent FAE training.Source: §3.2 · pp. 4–5

3.3 Frame-Aware Erasure

  • FAE adds per-concept low-rank adapters on the FFN path, with FrameGate using frame index and timestep to gate a shared frame-aware branch.Other erased concepts’ unsafe prompts serve as hard negatives, while a flicker penalty damps high-frequency gate oscillation on benign prompts.Source: §3.3 · pp. 5–6
  • FAE trains against negative guidance, worst-frame leakage, neutral-prompt anchoring, and frame-to-frame gate variation.The peak-frame leakage term penalizes only the worst-leaking frames.Source: §3.3 · pp. 5–6

3.4 C-MoE routing

  • C-MoE uses fixed, prompt-similarity soft gates to combine multiple per-concept experts for compositional prompts, adding no routing parameters.Per-expert calibration skips low-gate experts, so most clean prompts use the CR-edited backbone alone.Source: §3.4 · p. 6

4 Experiments

On one Wan2.1-T2V-1.3B model, FADE erases 16 concepts across objects, styles, and nudity while preserving video fidelity. Its advantage extends to compositional prompts, other backbones, and adversarial evaluations.

4.1 Experimental setting

  • The evaluation uses one Wan2.1-T2V-1.3B model to erase 16 concepts, with FAE trained on five latent frames and queried at 21.Baselines share the backbone, prompts, seeds, sampler, and judges; evaluation reports per-frame mean accuracy and neutral-pool FVD and VBench metrics.Source: §4.1 · pp. 6–7

4.2 Object erasure

  • 4.9% average mean-frame-acc versus 15.5 for VideoEraser, with the strongest erasure on all nine evaluated classes and the smallest FVD increase.FADE’s average mean-frame-acc falls from 77.8, a 93.7% relative reduction; its FVD rises by 7.8%, compared with 25.5–46.5% for baselines.Source: §4.2 · p. 7, Table 1 · p. 7

4.3 Artistic style erasure

  • FADE lowers the style score from 90.6 to 35.4, versus 74.6 for VideoEraser and 81.2 for Refusal, with the smallest FVD increase.The per-frame score is CLIP image similarity to held-out works by five painters.Source: §4.3 · p. 8, Table 3 · p. 8

4.4 Unsafe content and identity erasure

  • FADE flags 19% of frames for nudity, versus 54% for Refusal and 42% for VideoEraser, and flags 16–17% for violence and hate.It recognizes 6.4% of 30 erased celebrity identities while retaining 86.1% recognition on 10 identities; the unedited model scores 90.8%.Source: §4.4 · p. 8

4.5 Multi-concept composition and scaling

  • On prompts combining two erased classes, FADE leaves 4.0 average residual presence versus 12.9 for VideoEraser and 47–57 for other baselines.For nested sets of 1–10 classes, residual accuracy rises from 3.1 to 4.9 and FVD from 151.2 to 158.8, without an abrupt change.Source: §4.5 · pp. 8–9, Table 4 · p. 9

4.6 Other backbones

  • Across Wan2.1-T2V-14B, CogVideoX-2B, and HunyuanVideo-1.5, FADE removes 87–94% of concept presence on five shared Imagenette classes.FVD increases range from 7.8–8.4% on two small models to 13.6% on Wan2.1-14B and 17.2% on HunyuanVideo-1.5.Source: §4.6 · p. 9, Table 5 · p. 10

4.7 Robustness to adversarial prompts

  • FADE has the lowest attack success rate under all four nudity jailbreak suites; VideoEraser’s rate is 1.3–1.7× higher.Adaptive attacks against FADE’s router or FrameGate were not evaluated.Source: §4.7 · p. 10, Table 6 · p. 10

4.8 Ablations and cost

  • Removing FrameGate raises Gapreact from 2.7 to 8.5, while one union-trained LoRA lowers mean-frame-acc to 1.8 but raises FVD to 729.1.Training FAE without CR raises mean-frame-acc to 17.1 and FVD to 225.0; alternative gates erase less or cost more fidelity than the 290-parameter joint gate.Source: §4.8 · p. 10, Table 7 · p. 10

5 Limitations

FADE’s robustness evaluation uses established jailbreak suites and covers up to 30 erased concepts and clips of up to 81 frames; broader attacks and scaling remain future work.

5 Limitations

  • Robustness is evaluated with established jailbreak suites; router- or FrameGate-specific attacks remain a natural next step.The evaluated scope reaches 30 simultaneously erased concepts and clips of up to 81 frames; larger sets, incremental additions, longer clips, and autoregressive generators remain future work.Source: §5 · p. 10

6 Conclusion

FADE combines a joint closed-form edit, frame- and timestep-gated per-concept adapters, and prompt-based routing to erase multiple concepts from a T2V model. It outperforms eight baselines on object erasure and FVD, with its lead also reported across judges, compositional prompts, 30 identities, and four backbones.

6 Conclusion

  • FADE combines a joint closed-form edit, frame- and timestep-gated per-concept adapters, and a prompt-based router for multi-concept erasure.The conclusion reports advantages over eight baselines on object erasure and FVD, with the lead carrying across judges, compositional prompts, 30 identities, and four backbones.Source: §6 · p. 10

A Implementation details

The appendix specifies FADE’s shared backbone settings, stage-specific hyperparameters, and the joint closed-form solution for Concept Redirect. It also describes routing calibration, including a fallback for concepts whose unsafe and neutral prompt scores do not separate.

B Hyperparameters

  • All three stages use Wan2.1-T2V-1.3B, with Tlat=5 for training and 21 for inference, and flow-matching UniPC at 50 steps, shift 5, CFG 5, and 832×480.The appendix also lists stage-specific settings, including Concept Redirect’s λ=0.5, se=1.0, and sp=0.1, and Frame-Aware Erasure’s ranks and training schedule.Source: B Hyperparameters · p. 16
  • Routing thresholds are calibrated on held-out unsafe and neutral prompts with cross-concept hard negatives; experts are disabled when the score distributions do not separate.The fallback τc=1.0 applies to the collapsed-gap regime; tench is the only benchmark concept in that regime, and its expert is disabled because Wan2.1 renders a duck rather than a fish for that token.Source: B Hyperparameters · p. 16, Table 8 · p. 17

C Closed-form derivation of the Concept Redirect edit

  • Concept Redirect jointly solves for edited K/V projections across all erase concepts using one matrix inversion per block and projection, applied once.Motion-Paraphrase Embeddings replace bare-token erase anchors when available; the edit uses the UCE solver with a preserve set.Source: C Closed-form derivation of the… · pp. 16–17

D FADE training algorithms

FADE trains a joint Concept Redirect edit followed by per-concept frame-aware adapters, then combines the applicable experts during inference. The appendix explains the architectural choice of K/V editing and the design rationale for its self-supervised and temporal losses.

D FADE training algorithms

  • Concept Redirect produces edited cross-attention K/V projections, and Frame-Aware Erasure trains a per-concept FA-LoRA on that edited backbone.The frozen CR model supplies unconditional, unsafe-conditioned, and neutral-conditioned predictions; the student matches the negative-guidance target on unsafe prompts and the base model on neutral prompts.Source: D FADE training algorithms · p. 17
  • Other concepts’ unsafe prompts join the neutral pool as hard negatives, separating experts’ concept-specific components without extra data.The training procedure also uses a four-term objective, including peak-frame leakage, motion, and temporal-smoothness terms.Source: D FADE training algorithms · pp. 17–18

E FADE inference algorithm

  • At inference, routing gates are computed once, K/V projections are reblended at each denoising step, and active experts are added frame-by-frame in the FFN path.When no expert is active, as is usually the case for neutral prompts, inference reduces to Concept Redirect alone.Source: E FADE inference algorithm · p. 19

F Architectural rationale: why CR edits cross-attention K/V, not the FFN

  • FADE edits cross-attention K/V because they directly carry text conditioning into Wan2.1’s video stream, whereas its single FFN mixes video and cross-attention residuals.Pilot experiments found that editing the FFN degraded neutral-prompt motion; FAE instead handles residual frame reactivation on the FFN path.Source: F Architectural rationale: why CR… · pp. 19–20

G Loss-design discussion

  • The ESD-style negative-guidance target is synthesized from the frozen CR teacher’s predictions for each sampled noisy input, timestep, and unsafe prompt.This self-supervised construction avoids requiring concept-free ground-truth videos and supervises the student in the backbone’s prediction space.Source: G Loss-design discussion · p. 20
  • Noise-space motion supervision preserves suppression capacity, while intermediate-feature matching over-regularized the LoRAs in pilot runs.The noise-space loss allows earlier blocks to reroute so long as the final neutral-prompt prediction matches the frozen CR model.Source: G Loss-design discussion · p. 20
  • The peak-frame leakage loss penalizes the worst three frames: mean leakage missed spikes, while a single-worst-frame choice made optimization unstable in pilot runs.The authors therefore use k=3 to capture leakage spikes while maintaining stable training.Source: G Loss-design discussion · p. 20
  • The shared FrameGate’s smoothness penalty is evaluated once per example; applying it per block over-smoothed the gate toward a constant in pilot runs.The gate receives its frame-dependent signal through the peak-frame leakage term.Source: G Loss-design discussion · p. 20

H Routing design rationale

Pooled-cosine routing poorly separates motion-in-scene concepts, while token-level max-similarity recovers stronger margins. Multi-token anchors are retained to preserve the intended concept scope and avoid collateral effects on related concepts.

H Routing design rationale

  • Pooled-cosine margins stayed within |∆|<0.025 across four variants, below the sigmoid gate’s resolution at T=0.05.Centering helped object-free concepts but hurt spaniel and church prompts whose scene statistics overlapped with neutral prompts.Source: H Routing design rationale · p. 21
  • Token-level max-similarity raised every test concept’s median margin above +0.04 by selecting the most concept-aligned prompt-token and anchor-token pair.Centering was harmful with this rule because it rotated concept tokens off-axis without increasing separation.Source: H Routing design rationale · p. 21
  • Shortened anchors risk matching sibling breeds outside the training scope, so multi-token anchors preserve named-concept routing despite some modifier-token false positives.The routing scope is intended to match the user’s named concept, such as “english springer spaniel,” rather than all spaniels.Source: H Routing design rationale · p. 21

I Why FAE experts compose with little interference

The appendix explains why additive FAE experts can compose with limited interference, while stating the assumptions and untested regimes behind that argument. It also reports unsafe-content and celebrity results, backbone adaptations, evaluation protocols, and expert geometry.

I Why FAE experts compose with little interference

  • Expert composition relies on cross-concept hard negatives that keep each adapter near identity on other concepts, small updates under a linearization assumption, and gate pruning that limits active experts.Typically one or two experts fire on a concept prompt and none on a clean prompt, though shared modifier tokens can cause false activations.Source: I Why FAE experts compose with little… · p. 21
  • The leading-order cross-talk bound links training residuals and off-target gate magnitudes to composition interference, but it is not a guarantee and excludes prompts containing both concepts.The bound relies on the small-update linearization assumption; such compositional prompts are evaluated empirically instead.Source: I Why FAE experts compose with little… · pp. 21–22

J.2 Per-class results on other backbones

  • The five Imagenette classes are reported per class across backbones, with averages over those classes; corresponding nudity results are shown across three backbones.The cited tables cover CogVideoX-2B and Wan2.1-T2V-14B object results.Source: §J.2 · p. 22, Table 10 · p. 23, Table 11 · p. 24

J.1 Unsafe categories and celebrity identities

  • 19% of nudity frames and 16–17% of violence and hate frames were flagged with FADE, less than half of VideoEraser’s rate in each category.The evaluation used 4,703 I2P prompts with five seeds each; NudeNet judged nudity and Q16 judged violence and hate.Source: §J.1 · p. 22, Table 9 · p. 23
  • FADE’s erased-celebrity top-1 recognition rate was 6.4 while retained-identity accuracy was 86.1, versus 90.8 on the unedited model.VideoEraser’s erased-identity rate was 19.8 and its retained-identity accuracy was 16.3 points lower.Source: §J.1 · p. 22, Table 9 · p. 23

J.3 Porting FADE to other backbones

  • Backbone-specific interfaces determine where CR edits text-conditioned key/value projections and where FA-LoRA attaches; FAE hyperparameters stay fixed while router thresholds are recalibrated automatically.CogVideoX shares QKV and FFN weights between text and video tokens, unlike the separate streams described for HunyuanVideo-1.5.Source: §J.3 · p. 23

J.4 VLM judge and human study

  • The VLM evaluation judges every decoded frame, while the human study hides method identities and averages annotators’ five-point concept-presence ratings.Human ratings are rescaled to 0–100 for comparison with mean-frame accuracy.Source: §J.4 · p. 23

J.5 Geometry of the per-concept experts

  • After removing the shared update component, concept-specific expert components had a 70.2° median smallest principal angle, versus 88.8° for rank-matched random subspaces.All experts shared a suppression template with coherence 0.83, carrying about 78% of total update energy; the concept-specific comparison does not bound interference through that shared component.Source: §J.5 · pp. 23–24

J.6 FrameGate architecture

The FrameGate ablation favors joint frame-and-timestep conditioning over single-axis variants. The appendix also presents computational cost by stage and its dependence on concept count.

J.6 FrameGate architecture

  • Frame-only and timestep-only gates were clearly worse than the full gate; the temporal convolution improved erasure slightly to 4.5 versus 4.9 but raised FVD from 158.8 to 254.0.The larger MLP erased less and cost more fidelity; the authors suggest it may overfit the five-latent-frame training grid.Source: §J.6 · p. 24, Table 12 · p. 24

J.7 Computational cost

  • Table 13 reports FADE’s cost by stage and how it depends on the number of concepts N.The stated setup is one NVIDIA L20 with Wan2.1-T2V-1.3B, bf16, 832×480 output, 50-step UniPC, and CFG 5.Source: Table 13 · p. 25, §J.7 · p. 25

K Additional qualitative results

The appendix adds qualitative samples, including failure cases, and describes broader impacts and deployment caveats. FADE’s erasure can be misused, is not a guarantee, and is presented as complementary to output filtering.

K Additional qualitative results

L Broader impacts

  • Concept erasure can suppress legitimate content, requires concept names and per-concept training, and remains imperfect because attack success is non-zero and adaptive attacks are untested.The authors say it should complement rather than replace output filtering; the NSFW evaluation reports aggregate metrics and does not release generated explicit content.Source: L Broader impacts · p. 25
Loading 2610.03980v2…