Source-linked AI summary
Your Unlearning Gives You Away: Identifying Erased Concepts in Diffusion Models
Kaiyuan Deng, Yuchen Li, Yang Xiao, Bo Hui, Geng Yuan, Xiaolong Ma
TL;DR
Existing attacks assume erased concepts are known, while providers may not disclose which concepts they removed. TRACER uses weight-footprint analysis and candidate representations to identify erased concepts and estimate their number. Across image and video models, it reports substantial speedups over MIA and brute-force search.
Problem
Existing attacks focus on recovering known erased concepts, while providers may not disclose targets and base-model access alone does not reveal them.
Source: §1 · pp. 1–2Method
TRACER aggregates spectral weight footprints and compares them with template-averaged candidate representations to identify erased concepts.
Source: §3.2 · p. 5, §3.3 · p. 5Results
Takeaways & Limitations
TRACER provides a computationally efficient framework for assessing what publicly released model weights reveal about erasure targets despite suppressed generation.
Source: §5 · p. 9The adversary’s candidate vocabulary is assumed to contain the erased concepts, and the setting requires access to both original and edited weights.
Source: §3.1 · p. 3
Abstract
from arXiv · showhide
Existing attacks on unlearned diffusion models assume that the erased concepts are known in advance and focus on recovering them. In practice, however, model providers may not disclose which concepts have been removed, and even with access to the original base model, an adversary may still lack a clear target to attack. In this paper, we aim to answer the following critical but overlooked questions: which concepts have been erased from the model, and how many have been erased in total? To this end, we present Tracer, a framework that rapidly and accurately identifies erased concepts and estimates their number. Tracer efficiently identifies erased concepts without generating and classifying images. By combining lightweight spectral analysis of weight footprints, it enables efficient search over large candidate vocabularies. To distinguish multiple erased concepts, we introduce a footprint coverage objective that guides sequential discovery. Tracer estimates the number of erased concepts by detecting a sharp decline in candidate confidence as the selected concepts account for the erasure footprint, without requiring labeled examples for calibration. The framework requires only lightweight linear algebra and limited forward probes, with no prior knowledge of the unlearning algorithm. Experiments across text-to-image and text-to-video backbones and diverse unlearning methods demonstrate that Tracer identifies erased concepts and estimates their number in seconds, achieving 150 to 137,000 times and 133 to 20,000 times speedups over MIA and brute-force search on image and video models, respectively, with substantially higher identification accuracy.
1 INTRODUCTION
The paper asks how an adversary can identify erased concepts and estimate their number when providers do not disclose targets. TRACER searches candidate vocabularies using weight footprints, and experiments report faster, more accurate identification than existing alternatives.
1 INTRODUCTION
- Unknown-target identification matters because base-model access alone does not reveal which concepts a provider selected for erasure.The paper uses privacy-related removal of individuals’ identities as an example.Source: §1 · p. 1
- MIA and generation-based search face non-target effects, unknown stopping points, and high costs across vocabularies of tens of thousands of words.Changes can affect target and non-target concepts, making them difficult to distinguish without target labels for calibration.Source: §1 · p. 2
- TRACER builds erasure footprints from weight differences, then uses subspace coverage to select concepts explaining the remaining footprint and a confidence drop to estimate the erased-set size.The confidence cutoff estimates the count without labeled calibration data.Source: §1 · p. 2
- TRACER identifies multiple erased concepts without generating or classifying images and estimates their number without labeled examples or knowledge of the unlearning algorithm.The approach is designed for broad searches over large candidate vocabularies.Source: §1 · p. 2
- Across image and video models, TRACER achieves 150–137,000× and 133–20,000× speedups over MIA and brute-force search, respectively.Experiments cover text-to-image and text-to-video backbones and diverse unlearning methods.Source: §1 · p. 3
2 RELATED WORK
Prior work spans methods for erasing concepts and attacks that recover erased content. TRACER instead targets identifying unknown erasure targets and estimating their number.
2.1 CONCEPT UNLEARNING IN DIFFUSION MODELS.
- Fine-tuning methods steer target-concept noise toward anchors, while closed-form methods edit attention projections and inference-time methods use guidance or activation intervention.Related work also describes extensions to flow-matching transformers and text-to-video models.Source: §2.1 · p. 3
2.2 ADVERSARIAL VULNERABILITIES OF UNLEARNED MODELS.
- Attacks recover erased concepts through white-box signals, black-box prompt search, non-prompt inversion, and weight-space methods.Classical privacy attacks separately infer training-set membership or reconstruct training images.Source: §2.2 · p. 3
2.3 POSITIONING OF OUR STUDY.
- TRACER shifts the task from recovering content tied to known concepts to identifying unknown targets and estimating their number without image generation or knowledge of the erasure algorithm.The paper positions target identification as distinct from attacks on known erased concepts.Source: §2.3 · p. 3
3 METHOD
TRACER infers unknown erased concepts from weight-change footprints by encoding candidates in compatible input spaces and pursuing footprint coverage. It calibrates the search without labels and uses confidence declines to determine how many selected concepts to retain.
3.1 UNLEARN THREAT MODEL
- Given base and edited weights plus a vocabulary assumed to contain the targets, the adversary knows neither erased-set size nor algorithm and probes changed-matrix inputs without sampling or observing outputs.Source: §3.1 · pp. 3–4
3.2 ERASURE FOOTPRINT ASSEMBLY
- TRACER groups changed-weight imprints by probed input representation, then fuses spectrally reshaped Gram matrices with inverse-edit weighting into one footprint per group.Source: §3.2 · pp. 4–5
3.3 GEOMETRY-AWARE CANDIDATE ENCODING
- Candidates are template-averaged in each group’s input space, then covariance-whitened to attenuate shared high-energy directions and unit-normalized before footprint comparison.Source: §3.3 · p. 5
3.4 UNLEARNED CONCEPT DISCLOSURE
- A greedy pursuit selects candidates with greatest additional footprint coverage, standardizing group gains and weighting groups by initial candidate separation; lexical overlaps are suppressed using token-set inclusion.Source: §3.4 · pp. 5–6
- The retained set ends at a confidence-profile drop: use the largest drop by default, but after an initial rise use the first drop reaching α of the maximum.The estimate determines how many candidates from the sequence are retained.Source: §3.4 · pp. 6–7
4 EXPERIMENT
Across image and video models, TRACER identifies erased concepts and estimates their number with high accuracy and much lower runtime than traditional adversarial methods. Experiments also show that confidence-drop size detection and the framework’s component choices contribute to its results.
4.1 EXPERIMENTAL SETTINGS
- Experiments cover three text-to-image and two text-to-video backbones, with object, explicit-content, artistic-style, and generation-quality evaluations.For erased-set-size tests, concept sets are sampled from Imagenette-50; evaluated settings stay within each method’s stable erasure range.Source: §4.1 · p. 7, §4.2 · p. 7
4.2 DISCLOSING ERASED CONCEPT IN UNLEARNED MODELS
- Video experiments show perfect recovery on CogVideoX-5B for both methods and on Wan2.1-T2V-1.3B for Refusal; VideoEraser remains perfect for individual targets but declines modestly under joint erasure.The results indicate that weight footprints remain informative across video architectures with different temporal modeling.Source: §4.2 · pp. 7–8, Table 4 · p. 8
4.3 COMPARISON WITH TRADITIONAL ADVERSARIAL METHODS
- TRACER maintains high Recall@K across methods and set sizes, while baseline recall generally declines as K grows; allowing ten candidates still leaves targets unidentified.Table 3 reports Recall@K with the true K provided; recovering three of five targets scores 60%.Source: Table 3 · p. 8, §4.3 · pp. 8–9
4.2 DISCLOSING ERASED CONCEPT IN UNLEARNED MODELS
- TRACER achieves perfect erased-set-size estimation in most image and video settings, with occasional joint-erasure errors for ConceptPrune and SPEED.For UCE and Refusal, normalized confidence drops sharply when the selected sequence reaches the true erased-set size.Source: §4.2 · p. 8, Table 2 · p. 8, Figure 3 · p. 8
4.3 COMPARISON WITH TRADITIONAL ADVERSARIAL METHODS
- Across evaluated vocabulary sizes, TRACER is 150–137,000× faster than MIA and 133–20,000× faster than brute-force probing on image and video models, with modest runtime growth.Weight analysis with limited forward probes avoids repeated generation and sample-level membership evaluation.Source: Figure 4 · p. 9, §4.3 · p. 9
4.4 ABLATION STUDIES
- 100% Recall@5 across four methods falls by 5.2–16.0 points with a single imprint, 7.6–23.2 without background suppression, and up to 43.6 with static ranking on ESD.The ablations support integrating evidence across weight changes and updating candidate scores as coverage gains change.Source: Table 5 · p. 9, §4.4 · p. 9
- 100% exact-set recovery with the full method compares with 78%–92% using fixed calibration and 64%–80% when always selecting the largest confidence drop.The size-detection ablation accounts for confidence profiles that may rise before declining.Source: Table 6 · p. 9, §4.4 · p. 9
5 CONCLUSION
TRACER identifies erasure targets and estimates their number from publicly released model weights, without image generation. Experiments span image and video backbones and diverse unlearning methods.
5 CONCLUSION
- TRACER identifies erasure targets and estimates their number from publicly released model weights in seconds, without image generation.It supports label-free searches over large candidate vocabularies and achieves substantial speedups over membership inference and brute-force generation-based probing.Source: §5 · p. 9
- Experiments demonstrate TRACER’s effectiveness across image and video backbones and diverse unlearning methods.Source: §5 · p. 9
- TRACER assesses what model weights reveal about erasure targets despite suppressed generation.Source: §5 · p. 9
AI USE STATEMENT
Generative AI tools were used for grammatical correction and English-language clarity, not for scientific work. The authors state that scientific content and contributions were developed independently.
AI USE STATEMENT
- Generative AI tools were used solely to correct grammatical errors and improve the clarity of the English writing.Source: AI USE STATEMENT · p. 10
- The tools were not used to generate research ideas, develop the methodology, conduct experiments, or write or analyze code.Source: AI USE STATEMENT · p. 10
- The tools were also not used to interpret results, formulate claims, or create figures and other research artifacts; the authors say scientific content and contributions were developed independently.Source: AI USE STATEMENT · p. 10
ETHICS STATEMENT
The authors report that the study uses publicly available models and datasets for academic research and involves no human subjects, personal data, or sensitive user information. They state that they are unaware of ethical concerns or compliance issues.
ETHICS STATEMENT
- The work involves no human subjects, personally identifiable information, private data, or sensitive user information.Source: ETHICS STATEMENT · p. 10
- All models and datasets used are publicly available and employed solely for academic research.Source: ETHICS STATEMENT · p. 10
- The authors report no known ethical concerns, conflicts of interest, or issues involving discrimination, privacy, security, or legal compliance.Source: ETHICS STATEMENT · p. 10
REPRODUCIBILITY STATEMENT
The authors describe the methodological pipeline and approach components in the main paper, with additional implementation details in the appendix. They intend these materials to support independent reproduction and state that source code and configurations will be released upon acceptance.
REPRODUCIBILITY STATEMENT
- The main paper describes the complete methodological pipeline and all components of the proposed approach.Source: REPRODUCIBILITY STATEMENT · p. 10
- Additional implementation details are provided in the appendix, and the descriptions are intended to enable independent reproduction of the method and results.Source: REPRODUCIBILITY STATEMENT · p. 10
- The authors state that they will publicly release the complete source code and accompanying configurations upon acceptance.Source: REPRODUCIBILITY STATEMENT · p. 10
A.1 HYPERPARAMETER SETTINGS AND IMPLEMENTATION DETAILS
The appendix specifies TRACER’s calibration and evaluation settings, then details an extension that searches for multiword concepts by expanding and scoring candidate phrases. The extension’s cross-length stopping rule is heuristic and may return a shorter phrase than the complete target.
A.1 HYPERPARAMETER SETTINGS AND IMPLEMENTATION DETAILS
- Calibration searches 24 settings, fixes the selected setting throughout the search, and uses κ = 0.25, J = 96, B = 8, and α = 0.30 for confidence and erased-set-size detection.The size detector requires increases in at least seven of the first eight transitions and a decline reaching 30% of the largest log-confidence drop.Source: §A.1 · p. 16, Table 7 · p. 16, Table 8 · p. 17
- For text-aware groups, token offset o selects a prompt’s representation position; larger offsets select earlier positions, and pooled representations are unaffected.For CLIP, o = 1 selects the end-of-text token and o = 2 the last content token.Source: §A.1 · p. 16
- Evaluation uses Imagenette-20 and artist sets, while object identification searches either 1,000 ImageNet classes or an expanded 73,406-entry vocabulary.The 70,000-entry vocabulary used in runtime comparisons is a subset of the expanded pool.Source: §A.1 · pp. 16–17
- Probe matching and candidate encoding average representations across five prompt templates, including a template with the vocabulary entry alone.Probe matching distinguishes text-aware from text-agnostic imprints; artistic-style experiments also assess sensitivity to template choice.Source: §A.1 · p. 17, Table 9 · p. 17
A.2 EXTENSION TO MULTIWORD CONCEPT IDENTIFICATION
- The multiword extension builds phrases from single-token vocabulary entries, using a beam of top-scoring candidates and expanding each by inserting vocabulary words at every position.Duplicate sequences are removed, while distinct word orders remain candidates; the extension selects phrase length without knowing the target length in advance.Source: §A.2 · pp. 17–18
- Each phrase is encoded with its final content-token representation and scored by its normalized footprint energy in the compatible text-aware group.Candidate scores are standardized within each expansion round and penalized for added words.Source: §A.2 · p. 18
- Cross-length stopping compares each round’s best penalized score with earlier rounds, but this heuristic can return a shorter phrase even when the complete target was generated.The phrase-length stopping rule is separate from erased-set-size detection.Source: §A.2 · p. 18
A.3 BASELINE IMPLEMENTATIONS
The baselines rank candidate concepts using changes between the original and unlearned models: one probes generated images, and the other adapts a membership-inference statistic. Both can reflect effects on non-target concepts.
A.3 BASELINE IMPLEMENTATIONS
- Both baselines rank candidates in descending order of scores computed from changes between the public base model and unlearned model.The table caption identifies identification accuracy as the comparison measure but does not provide values in the supplied passage.Source: §A.3 · p. 18, Table 10 · p. 20
- Brute-force probing scores each candidate by the decrease in average concept evidence in images generated by the original versus unlearned model.It requires image generation for every candidate, and related non-target concepts may also receive high scores.Source: §A.3 · p. 19
- The adapted PIA attack averages sample-level error-statistic changes over each candidate’s evaluation samples to produce a concept score.It ranks candidates by statistic changes rather than making binary membership decisions, and scores may reflect direct erasure or non-target effects.Source: §A.3 · p. 19
B ADDITIONAL EXPERIMENTAL RESULTS
Across three text-to-image backbones, TRACER identifies erased artistic styles with near-perfect or perfect accuracy in almost all evaluated settings. Facial-identity unlearning is excluded because tested methods also degraded non-target generation.
B ADDITIONAL EXPERIMENTAL RESULTS
- ESD averages 98.0% on Stable Diffusion v1.5 (92% with ten jointly erased styles); all other evaluated configurations reach 100%, including twenty-style UCE, ScaPre, and FIA.The results extend weight-footprint identification beyond object concepts to artistic styles across backbones and unlearning methods.Source: B ADDITIONAL EXPERIMENTAL RESULTS · p. 20
- Facial-identity unlearning is excluded because tested methods suppressed target identities but also degraded non-target generation, limiting evaluation of selective identification.The resulting weight changes reflect broader losses in generative capability as well as target suppression.Source: B ADDITIONAL EXPERIMENTAL RESULTS · p. 20
C CODE SOURCES FOR UNLEARNING AND ATTACK METHODS
The appendix lists public code repositories for the unlearning methods and the membership-inference baseline used in the experiments.
C CODE SOURCES FOR UNLEARNING AND ATTACK METHODS
- Public repositories are listed for numerous unlearning methods and the PIA membership-inference baseline.The listed methods include ESD, UCE, MACE, ConceptPrune, VideoEraser, and others.Source: C CODE SOURCES FOR UNLEARNING AND… · p. 21