Source-linked AI summary
EraseSAE: Surgical Concept Erasure in Text-to-Video Diffusion Models via Sparse Autoencoders
Xinghao Wang, Dong Li, Wei Yu, Yingwei Pan, Tao Gong, Qi Chu, Nenghai Yu, Ting Yao
TL;DR
T2V diffusion models pose safety and copyright risks, while existing concept-erasure methods operate too coarsely for fine-grained distributed representations. EraseSAE uses sparse autoencoders, contrastive attribution, and dynamic masks to erase target concepts in active regions. Across models and tasks, it reports precise removal with limited quality degradation and stronger performance than state-of-the-art methods.
Problem
T2V models trained on loosely curated data create safety and copyright risks, while existing concept erasure can be too coarse to remove targets without degrading broader generation.
Method
EraseSAE uses PConvSAE to decompose spatiotemporal activations into monosemantic features, contrastive attribution to identify target kernels, and dynamic masks with spatially-modulated guidance for localized erasure.
Results
EraseSAE achieves state-of-the-art erasure effectiveness while preserving generation quality, outperforming the strongest baseline by 34.5% in erasure accuracy and 8.3% in SSIM.
Takeaways & Limitations
The framework shifts concept erasure from entangled neuron-level intervention to localized suppression in interpretable feature space while preserving essential scene content.
Abstract
from arXiv · showhide
Recent advances in text-to-video (T2V) diffusion models have demonstrated remarkable generative capabilities, yet their reliance on loosely curated training data raises pressing safety and copyright concerns. Concept erasure offers a principled remedy by removing unwanted semantics from pretrained models while preserving remaining concepts. However, existing approaches typically operate at a coarse granularity misaligned with the fine-grained, distributed nature of concept representations, leading to incomplete removal or degraded generation quality. We argue that surgical erasure fundamentally requires intervention at the level of monosemantic features, where each unit encodes a single interpretable concept. To this end, we propose EraseSAE, a novel framework that leverages sparse autoencoders to achieve surgical concept erasure in DiT-based T2V diffusion models via a principled decompose-attribute-erase pipeline. We first introduce the Partitioned Convolutional Sparse Autoencoder, which decomposes dense spatiotemporal activations into disentangled, interpretable sparse features while preserving spatiotemporal coherence. A contrastive attribution mechanism then contrasts activations from paired prompts to isolate concept-specific feature kernels. At inference, timestep-resolved spatiotemporal masks derived from the identified kernels confine erasure to regions where the target concept is active, leaving unrelated content intact. Extensive experiments across diverse diffusion models and concept erasure tasks demonstrate that EraseSAE achieves precise and robust concept removal with minimal quality degradation, substantially outperforming state-of-the-art methods. The code is available at https://github.com/HiDream-ai/EraseSAE.
1 Introduction
EraseSAE addresses the challenge of surgically removing target concepts from T2V diffusion models without degrading unrelated generation. It intervenes on monosemantic sparse features through a decompose–attribute–erase pipeline and reports stronger erasure and quality preservation than prior methods.
- Motivation: T2V models create safety and copyright risks, while retraining is impractical and existing erasure methods struggle to remove targets without broader quality degradation.Concept erasure is presented as an alternative to filtered-data retraining because it selectively removes target semantics from pretrained models.
- Motivation: Surgical erasure requires monosemantic features that encode single interpretable concepts rather than entangled polysemantic units.SAEs provide a sparse representation for isolating specific features while preserving unrelated ones and covering distributed representations.
- Method: EraseSAE introduces PConvSAE, a partitioned convolutional sparse autoencoder that decomposes dense visual representations into interpretable sparse features while preserving spatiotemporal coherence.Its partitioned architecture separates concept-specific semantics from concept-agnostic scene structure.
- Method: A contrastive attribution mechanism isolates concept-correlated features and derives timestep-resolved spatiotemporal masks that restrict erasure to active target regions.The masks are designed to preserve non-target content during inference.
- Results: 34.5% higher erasure accuracy and 8.3% higher SSIM than the strongest baseline are reported across multiple T2V models and diverse erasure tasks.The authors describe these results as state-of-the-art erasure effectiveness with preserved generation quality.
2 Related Works
Related work spans training-free and training-based concept erasure, while SAE-based approaches provide feature-level interpretability but had not addressed T2V spatiotemporal representations. EraseSAE extends this direction with a video-specific sparse autoencoder framework.
- Concept Erasure: Training-free methods redirect inference without changing model weights, but their surface-level suppression leaves internal concept representations vulnerable to adversarial attacks.Examples include negative prompting, text embedding manipulation, and conditional guidance.
- Sparse Autoencoders: SAEs decompose dense activations into sparse combinations of monosemantic features and support interpretable feature-level control in language and image generation models.Prior work established SAEs as tools for mechanistic interpretability and concept erasure outside T2V models.
- Gap: EraseSAE addresses this gap with a decompose–attribute–erase pipeline for DiT-based T2V diffusion models.Its framework overview combines sparse feature decomposition, concept attribution, and masked inference-time suppression.
- Gap: Prior SAE concept-erasure work did not investigate T2V diffusion models, whose video representations retain essential spatiotemporal locality.Conventional MLP-based SAEs flatten high-dimensional hidden states, compromising that locality.
3 Methodology
EraseSAE decomposes entangled video representations into complementary context and concept features, attributes target-specific kernels, and applies timestep-resolved spatial erasure to suppress target semantics while preserving unrelated content.
- Decompose: PConvSAE transforms dense spatiotemporal activations into disentangled monosemantic features while preserving spatial structure and temporal coherence.It operates at an identified intervention layer using convolutional processing rather than flattening video activations.
- Decompose: PConvSAE preserves native video topology by folding time into the batch axis and applying spatial convolutions directly to activation tensors.This avoids the information loss associated with flattening high-dimensional hidden states.
- Decompose: Its Context Branch encodes concept-agnostic scene structure, while its Concept Branch isolates target-bound semantics for selective removal.The branches are structurally decoupled and jointly form a complete, non-redundant decomposition.
- Decompose: Spatial-Aware Local Activation selects the K channels with the strongest peak spatial responses and applies ReLU only to those channels.The mechanism produces spatially coherent sparse codes while suppressing unselected channels across their spatial extent.
- Decompose: The joint reconstruction objective assigns background reconstruction to the Context Branch and target-region reconstruction to the Concept Branch.Their additive combination recovers the original hidden state, enabling later removal of the Concept Branch contribution.
- Erase: Dynamic masks intersect foreground and concept-activation constraints at every denoising timestep, and spatially modulated guidance suppresses target semantics only where those masks are active.Outside masked regions, the original positive-conditioned prediction is preserved to maintain background content.
4 Experiments
Experiments evaluate EraseSAE on nudity and celebrity erasure across DiT-based video models, with additional Flux.1 testing and ablations. Results show strong removal, adversarial robustness, preservation of unrelated content and identities, and benefits from the proposed architecture and inference strategy.
- Nudity Erasure: EraseSAE achieves the lowest Gen-set nudity exposure rates on CogVideoX-5B and HunyuanVideo, reaching 2.62 and 7.13, respectively.The comparison is against state-of-the-art methods, including T2VUnlearning.
- Nudity Erasure: Across Ring-A-Bell difficulty levels K16, K38, and K77, EraseSAE maintains consistently low detection rates under adversarial prompt manipulation.The authors attribute this robustness to contrastive attribution and suppression of intrinsic concept features rather than prompt patterns.
- Nudity Erasure: EraseSAE produces SSIM scores of 74.99 and 65.69 on CogVideoX-5B and HunyuanVideo, surpassing the next best method by 27.97 and 5.02 points.VBench object-class and subject-consistency scores remain competitive with unmodified models, while CogVideoX-5B latency is 3.66 s/frame versus 3.61 for the original.
- Nudity Erasure: Qualitatively, EraseSAE removes nudity while preserving unrelated visual content, unlike training-free methods and global weight editing that leave sensitive content or introduce artifacts.The comparison is shown on HunyuanVideo.
- Celebrity Erasure: Celebrity erasure reduces average target detection accuracy to 8.00 on CogVideoX-5B and 19.20 on HunyuanVideo while preserving non-target identities at 58.60 and 61.50.The method is evaluated on five public figures, and its preservation scores closely match or exceed those of baselines.
- Ablation Studies: Adding concept-separation, leakage, and temporal losses reduces nudity rate from 24.5 to 8.09 and raises SSIM to 65.69, while SM-CFG avoids the trade-off between insufficient suppression and scene corruption.SAE-Sub has a 51.30 nudity rate, whereas SAE-Mask reaches 6.56 but degrades SSIM to 53.51.
- Ablation Studies: The dual-branch PConvSAE performs best across architectural variants, separating concept-specific and context-agnostic representations for erasure and preservation.The linear SAE yields a nudity rate of 12.76 and SSIM of 56.97, whereas the dual-branch design is best on all metrics.
5 Conclusion
EraseSAE enables surgical concept erasure in T2V diffusion models by shifting intervention from polysemantic neurons to monosemantic features, preserving generation quality.
- EraseSAE uses PConvSAE and contrastive attribution to isolate and suppress target concepts within their active spatiotemporal regions.The framework follows a decompose–attribute–erase pipeline while preserving structural coherence.