Source-linked AI summary
Stress Testing Unlearning Algorithms
Noam Diamant, Ethan Fetaya, Neta Glazer
TL;DR
LLM unlearning benchmarks can miss both collateral damage to safe, near-distribution knowledge and recovery of supposedly erased information under adversarial prompting. WMDP++ addresses these gaps with Boundary questions and adversarial extraction tests, establishing a more stringent benchmark for LLM unlearning.
Problem
Existing unlearning benchmarks provide limited evidence about preserving safe knowledge near the forget set and resisting deliberate extraction attempts.
Method
WMDP++ extends WMDP with a Boundary dataset for safe near-distribution questions and adversarial evaluations using in-context probes and jailbreak attacks.
Results
WMDP++ establishes a more realistic evaluation standard by exposing collateral damage and distinguishing robust erasure from surface-level suppression.
Takeaways & Limitations
The benchmark highlights the need for unlearning methods to guarantee robust erasure while preserving boundary performance.
Takeaways & Limitations
Boundary-question validity relied on human-expert evaluation in cybersecurity, while biology validation was more demanding because it required substantial specialized-expert time.
Abstract
from arXiv · showhide
Recently, machine unlearning, the removal of specific training data influence from a model, has gained increasing attention. In large language models (LLMs), unlearning is particularly challenging due to the ambiguity of inputs and outputs. Con- sequently, rigorous evaluation is critical for assessing both safety and utility, and for driving progress in unlearning meth- ods. We identify two key shortcomings in existing unlearning benchmarks: (1) they do not actively test whether unlearned information can still be forcibly extracted, and (2) they fail to evaluate performance preservation on boundary questions, be- nign queries that are semantically close to the unlearned con- tent. Here we introduce WMDP++, an extension of WMDP that addresses these gaps by incorporating targeted extrac- tion of unlearned information and systematic evaluation on boundary questions. WMDP++ provides a more stringent and informative benchmark for evaluating unlearning in LLMs.
Introduction
LLM unlearning seeks to remove targeted training-data influence without retraining, but ambiguous elicitation and conventional benchmarks make genuine erasure difficult to establish. WMDP++ addresses these evaluation gaps with boundary preservation and adversarial extraction tests.
- Web-scale training data can absorb private, copyrighted, and hazardous content, while retraining large models after discovery is prohibitively expensive.Machine unlearning instead aims to remove targeted data influence without the cost of full retraining.
- LLM unlearning is difficult to verify because the same knowledge can be elicited through diverse phrasings, contextual cues, and multi-turn interactions.Open-ended inputs and outputs make definitive success assessments harder than in classification settings.
- Existing benchmarks can miss collateral damage because their retain sets are often semantically distant from the unlearned content.WMDP, for example, uses broad MMLU retention questions, only a small fraction of which relate closely to the forget set.
- Direct-question benchmarks may overestimate unlearning efficacy when surface suppression leaves latent knowledge vulnerable to jailbreak extraction.Adversarial prompts can reveal whether supposedly forgotten information remains recoverable.
- WMDP++ extends WMDP with boundary questions and adversarial robustness evaluations, including in-context learning probes and jailbreak attacks.These additions test safe near-distribution knowledge preservation and deliberate recovery of unlearned information.
WMDP++ Benchmark
WMDP++ evaluates unlearning beyond direct forget-set performance by testing safe, conceptually adjacent knowledge and deliberate extraction attempts. It combines a Boundary dataset with adversarial evaluations to provide a more stringent benchmark.
- Boundary Concept Preservation: Standard retain sets can conceal degradation on conceptually adjacent knowledge, or concept bleeding, despite strong general-retain scores.Near-distribution questions are a harder test of preserving safe knowledge surrounding the forget set.
- Boundary Concept Preservation: WMDP’s broad MMLU retain set contains few questions on closely related biology and cybersecurity subtopics, so substantial local degradation may barely affect its overall score.Topic-level MMLU evaluation remains too general to capture boundary performance and is not ubiquitous.
- Boundary Concept Preservation: The Boundary queries dataset uses expert-level, safe biology and cybersecurity questions conceptually adjacent to hazardous forget knowledge.It targets legitimate scientific reasoning and technical proficiency in the near-distribution rather than broad general knowledge.
- Boundary Concept Preservation: Cybersecurity Boundary questions received human-expert evaluation, whereas biology-question validation was substantially more demanding for specialized experts.The authors report that cybersecurity questions were validated as relevant, correct, and safe.
- Robust Unlearning Evaluation: Traditional direct-question metrics overlook adversarial settings that may recover supposedly forgotten information, distinguishing suppression from genuine forgetting.The benchmark therefore treats recovery under deliberate extraction as a central robustness concern.
- Robust Unlearning Evaluation: WMDP++ adds in-context learning probes using forget-set, retain-set, or Wikipedia contexts, alongside jailbreak evaluations such as enhanced GCG, Crescendo, and DIA.These tests assess robustness under multiple adversarial settings rather than standard interactions alone.
- Robust Unlearning Evaluation: Figure 1 organizes Biology-domain Target, MMLUS, and Boundary QA results by model, with Llama-3-8B in the first two rows and Zephyr-7B in the final two.The caption identifies the QA-set contexts and row grouping used for comparison.
Experiments
The experiments evaluate five unlearning methods on two open-source LLMs using target, jailbreak, general-retention, and boundary metrics. Results show that conventional target and general benchmarks can mask extraction vulnerabilities and severe degradation on benign near-distribution knowledge.
- Experimental Setup: Experiments use Llama-3-8B and Zephyr-7B to evaluate RMU, ELM, NPO, simNPO, and OrthoGrad.The evaluation includes target accuracy, MMLU, Max-JB, and Boundary accuracy, with Boundary questions designed to resemble the unlearned knowledge while remaining benign.
- Main Results: Target accuracy fails to distinguish robust erasure from superficial suppression: Llama-3-8B Biology RMU and simNPO reach 24.98% and 26.08%, but their Max-JB scores are 28.12% and 60.09%.RMU therefore appears substantially more robust under adversarial extraction despite similar target-set accuracy.
- Main Results: Boundary accuracy exposes collateral damage that general benchmarks miss, including a 51% drop for Zephyr-7B with RMU in Biology and a 59% drop for Llama-3-8B with ELM in Cybersecurity.The corresponding MMLU and MMLUS drops were only roughly 1% and 3% for Zephyr-7B RMU, and 6% and 4% for Llama-3-8B ELM.
- Detailed Results: Few-shot QA context usually fails to recover unlearned knowledge, with RMU, NPO, simNPO, and OrthoGrad remaining near 24%–28% accuracy, while Biology ELM is a notable exception.ELM generally improves as the number of prepended QA exemplars increases before a minor decline at higher exemplar counts.
- Detailed Results: White-box GCG usually extracts more suppressed knowledge than DIA or Crescendo, but exceptions include Biology ELM on Zephyr-7B, where Crescendo reaches 54.05% versus GCG’s 49.25%.In Cybersecurity, Zephyr-7B with RMU also shows slightly higher extraction under DIA than GCG: 31.81% versus 31.00%.
Conclusion
WMDP++ exposes blind spots in existing LLM unlearning benchmarks by testing collateral damage to adjacent knowledge and whether adversarial prompts can recover supposedly erased information.
- WMDP++ incorporates the Boundary dataset to detect collateral damage to safe, conceptually adjacent near-distribution knowledge.
- Its robustness evaluations use adversarial jailbreaks and in-context steering to test whether unlearned information remains extractable.
- The framework finds that many state-of-the-art methods suppress surface-level outputs rather than genuinely erasing latent knowledge.
- WMDP++ sets a more realistic standard requiring robust erasure and precise preservation of boundary knowledge.