RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models

1 Introduction

• A comprehensive real-world benchmark must test adversarial disclosure and effects on neighboring knowledge, general and reasoning abilities, truthfulness, and factuality.Existing unlearning evaluations are described as non-adversarial, often using multiple-choice or question-answer formats.Source: §1 · p. 2
• Unlearning leaves models more vulnerable to fill-in-the-blank and adversarial probes than question-answer probes, appears ineffective against MIAs, can affect locality and utility, and batch-target unlearning is harder and may cause collapse.Gradient ascent, negative preference optimization, and simple in-context unlearning perform relatively well among the baselines.Source: §1 · p. 3
• RWKU covers 200 real-world targets with 13,131 forget probes and 11,379 neighbor probes in a setting with neither forget nor retain corpora available.The setting is motivated by the practical difficulty of supplying potentially sensitive or copyrighted data and locating all training points associated with parameterized knowledge.Source: §1 · pp. 2–3
• The forget set combines four MIA methods and nine kinds of adversarial probes, while the retain set assesses neighbor perturbation and multiple utility capabilities.The utility dimensions include general and reasoning ability, truthfulness, factuality, and fluency.Source: §1 · p. 3
• The benchmark could also support work on knowledge probing, knowledge localization, and model jailbreak, and its datasets and code are publicly released.The release site is http://rwku-bench.github.io.Source: §1 · p. 3

2.1 Knowledge Unlearning for Large Language Models

• LLM unlearning methods include gradient ascent, preference optimization, representation controlling, rejection tuning, and task arithmetic, while supposedly forgotten knowledge can remain extractable.Earlier work also studies removal of specific classification tasks, memorized sequences, copyrighted books, and toxic capacities.Source: §2.1 · p. 3

2.2 Unlearning Benchmarks for Large Language Models

• WHP uses Harry Potter prompts, WMDP targets hazardous knowledge with expert-written questions, and TOFU uses fictitious authors whose knowledge is added through fine-tuning.Their retain evaluations draw on general-ability and fluency datasets or other QA subsets and real-author and world-fact tasks.Source: §2.2 · pp. 3–4

3.1 Task Definition and Setting

• RWKU supplies only the target and original model, then uses model-generated text as a synthetic forget corpus for existing unlearning methods.The setting omits both forget and retain corpora.Source: §3.1 · p. 4

3.2 Data Collection and Construction

• The benchmark selects the 200 most popular people from a ranked list and finds their knowledge is better memorized than other sources across Phi-2, LLaMA2, and LLaMA3.The source comparison includes general knowledge, unseen knowledge, and C4 sequences.Source: §3.2 · pp. 4–5, Figure 1 · p. 5
• Forget probes are generated from target Wikipedia passages, retained only when LLaMA3 recalls the answers, and manually checked; neighbor probes use related entities filtered by popularity and GPT-4 analysis.Neighboring knowledge is related to, but not entirely contained within, the target scope.Source: §3.2 · p. 5

3.3.1 Forget Assessment.

• RWKU evaluates target knowledge through memorization and manipulation, probing whether models recall training-seen fragments or use acquired knowledge, including under adversarial attacks.Its framework applies to real-world famous-person targets.Source: §3.3.1 · p. 5, §3.3 · p. 5, Figure 2 · p. 6
• Four membership-inference methods audit retained target knowledge by comparing forget-member and retain-member scores; the benchmark primarily reports LOSS, where higher scores indicate lower retention likelihood.The methods are LOSS, Zlib Entropy, Min-K% Prob, and Min-K%++ Prob.Source: §3.3.1 · p. 6
• Nine adversarial attack types complement question-answer probes, while fill-in-the-blank probes test recall of target-related text from Wikipedia.The attack types include prefix injection, role playing, synonym manipulation, and cross-lingual queries.Source: §3.3.1 · pp. 5–6

3.3.2 Retain Assessment.

• Model utility covers general ability, reasoning, truthfulness, factuality, and fluency, alongside neighboring-knowledge performance.The listed measures use MMLU, BBH, TruthfulQA, TriviaQA, and AlpacaEval-based fluency scoring.Source: §3.3.2 · p. 7
• Retain assessment tests locality by checking that unlearning does not perturb related knowledge outside the target scope.Forgetting Stephen King should not erase who played Jack Torrance in The Shining.Source: §3.3.2 · p. 7

4.1 Model and Data Preparation

• Experiments use LLaMA3-Instruct (8B) and Phi-3 Mini-4K-Instruct (3.8B), with model-generated target descriptions as a synthetic forget corpus and Wikipedia descriptions as pseudo ground truth.LLaMA2-Chat and Mistral-Instruct are also reported for reference.Source: §4.1 · p. 7

4.2 Baseline Methods

• Baselines include in-context unlearning, representation engineering, and gradient ascent, which respectively use instructions, activation control, and an objective that maximizes forget-corpus negative log-likelihood.Representation engineering derives a control vector from expert and novice prompts.Source: §4.2 · p. 7
• DPO trains on counterfactual positive and synthetic-forget negative examples; NPO drops the positive examples, while rejection tuning fine-tunes on target-related refusals.Rejection data is made by replacing model answers with “I do not know the answer.”Source: §4.2 · pp. 7–8
• The main experiments average single-target unlearning results over 100 targets and compare full fine-tuning, partial-layer fine-tuning, and LoRA.Implementation details and hyperparameters are provided in the appendix.Source: §4.2 · p. 8

Overall Results

• Unlearned models are more susceptible to fill-in-the-blank and adversarial probes than question-answer probes, and almost all methods trained on C_s^f fail under MIAs.The findings indicate that probes can detect memorized knowledge even when models seem to have forgotten how to use it.Source: §5 · pp. 8–9
• Fine-tuning on the synthetic forget corpus C_s^f achieves greater unlearning efficiency than using C_f*, which includes broader target knowledge but also irrelevant material.C_s^f, generated by the original model, is likely more aligned with its internal memory.Source: §5 · pp. 8–9
• ICU works best on LLaMA3 but has almost no effect on Phi-3; GA and NPO perform relatively well among parameter-changing methods, while LoRA forgets less than full fine-tuning.LoRA also forgets less on the retain set than full fine-tuning.Source: §5 · p. 9

Batch-target Unlearning

• In batch unlearning, DPO and NPO do not complete unlearning while maintaining original performance, GA risks model collapse at 30 targets, and RT is more stable.RT does not significantly affect neighbor knowledge; experiments used target sizes of 10, 20, 30, 40, and 50.Source: Figure 5 · p. 10, §5 · p. 10

Partial-layer Unlearning

• Fine-tuning early LLaMA3 layers yields better unlearning without affecting neighbor knowledge; the authors suggest early layers may store more factual knowledge.They also suggest early-layer unlearning may twist the meanings of target-related keywords.Source: §5 · p. 10

Case Study

• ICU and RT usually lead the model to refuse answers, whereas GA, DPO, and NPO tend toward erroneous alternatives.The finding comes from the paper’s case study of forgetting effects.Source: §5 · p. 10

6 Conclusion and Future Work

• RWKU benchmarks unlearning of 200 real-world famous people in a more practical and challenging setting.The conclusion describes the benchmark’s three design factors as task setting, knowledge source, and evaluation framework.Source: §6 · pp. 10–11
• Its evaluation combines membership inference and adversarial probes for efficacy with measures of locality and utility, including reasoning, truthfulness, factuality, and fluency.Locality and utility are also assessed through neighbor perturbation and general ability.Source: §6 · p. 11
• Future work will diversify knowledge sources, add attack methods such as gradient-based attacks, and adopt more comprehensive metrics, including efficacy-locality balance.Source: §6 · p. 11

A.1 Motivation

• RWKU provides the target and original model without forget or retain corpora, and evaluates forgetting and retention with memorization, manipulation, adversarial, neighbor, and utility probes.The utility dimensions include general ability, reasoning, truthfulness, factuality, and fluency.Source: §A.1 · p. 19

A.2 Composition

• The benchmark samples utility questions from MMLU, BBH, TruthfulQA, TriviaQA, and AlpacaEval, and its dataset contains testing sets.For each target, the utility sample sizes are 171, 81, 50, 100, and 50 instances, respectively.Source: §A.2 · p. 20

A.3 Collection Process

• GPT-4 generates excess forget and neighbor query-answer probes; open-source models filter for existing knowledge, and students manually check format and type.The probes are generated with temperature = 1, and sampling is random.Source: §A.3 · p. 20

A.4 Preprocessing/cleaning/labeling

• The appendix reports no preprocessing, cleaning, or labeling of the dataset.The authors state the dataset was collected in April 2024.Source: §A.4 · p. 21, §A.3 · p. 21

A.5 Uses

• Beyond unlearning, the dataset is identified as usable for knowledge probing and knowledge localization.The appendix says it is distributed through Hugging Face and GitHub under CC-BY-4.0.Source: §A.5 · p. 21, §A.6 · pp. 21–22

A.7 Maintenance

• The author team supports the dataset, retains older versions, and accepts proposed additions through Hugging Face subject to validation and verification.The authors report collecting 100 historical-event targets for future release and aim to diversify target knowledge sources.Source: §A.7 · p. 22

C.1 Limitation

• RWKU’s major limitation is its single knowledge source: real-world famous people are the only unlearning targets.The authors say they are working to diversify sources and plan to explore historical events, conceptual knowledge, and rule-based knowledge.Source: §C.1 · p. 23

C.2 Potential Impact and Ethics Statement

• The 200 targets’ information is publicly available on Wikipedia, which the authors say raises no privacy concerns.The paper also says it considers unlearning side effects and uses adversarial probes to assess robustness.Source: §C.2 · p. 23
• The authors frame RWKU’s purpose as supporting evaluation of unlearning methods and addressing ethical concerns surrounding LLMs.They state that unlearning specific knowledge can effectively eradicate sensitive information, copyrighted data, and potentially harmful capabilities within LLMs.Source: §C.2 · p. 23

D Benchmark Comparison

• Table 4 is identified as a comparison between existing unlearning benchmarks and RWKU, but the supplied passage does not report its entries.The caption gives the table’s scope without specifying comparison dimensions or results.Source: D Benchmark Comparison · p. 24, Table 4 · p. 24

E.1 Prompt Templates

• Forget memorization templates generate level-1 cloze probes and level-2 questions from target-related text, with short, unique answers and unambiguous factual queries.The templates specify correct answers and a question-type tag.Source: §E.1 · pp. 25–26
• Forget manipulation templates use adversarial questions, including affirmative suffixes, prefix injection, synonym manipulation, context hints, reverse queries, role play, cross-lingual prompts, and in-context learning.The stated purpose is to induce the model to provide target-related information that may have been unlearned.Source: §E.1 · p. 27
• Neighbor templates test retention of knowledge related to, but distinct from, the target using cloze and simple-question formats.The instructions specify that neighbor probes must focus on the neighbor and not involve the target.Source: §E.1 · pp. 28–29

E.2 Data Examples

• Forget-set examples include cloze and direct-question probes, plus adversarial prompts such as prefix injection, synonym manipulation, role play, context hints, and reverse queries.Examples target facts about Stephen King, including a pseudonym and works.Source: §E.2 · pp. 30–32
• Neighbor examples ask about related works and people while keeping the target separate, such as questions about The Shawshank Redemption and its characters.The examples include both cloze and simple-question formats.Source: §E.2 · pp. 33–34

F.1 Evaluation Prompts

• Evaluation prompt examples cover knowledge memorization and manipulation probes as well as MMLU, BBH, TruthfulQA, TriviaQA, and AlpacaEval.The examples show formats including fill-in-the-blank, question answering, multiple choice, and instruction prompts.Source: §F.1 · pp. 35–36

• Table 5 reports RWKU dataset statistics, and an asterisk marks datasets constructed in this paper.Source: Table 5 · p. 37

G.1 Data Generation Prompt

• Generation prompts ask for a target’s factual biography, a made-up counterfactual biography, or a question based on known information.Source: §G.1 · p. 38

G.2 Original Wikipedia Descriptions

• The appendix includes original Wikipedia descriptions of Stephen King as source text for the target.Source: §G.2 · p. 38

G.3 Generated Data Examples

• Generated examples include biographical descriptions and questions paired with refusal-style answers about Stephen King.Source: §G.3 · pp. 39–40

H.1 Baseline Details

• RepE forms a control vector by subtracting expert-prompt representations from novice-prompt representations; GA maximizes the original causal-language-model log-likelihood objective.Source: §H.1 · p. 41

H.2 Hyper-parameter Settings

• Single-target experiments average results over 100 targets on LLaMA3-Instruct (8B) and Phi-3 Mini-4K-Instruct (3.8B); batch-target sizes range from 10 to 50.Synthetic-corpus methods train for three epochs in the main setting and two in batch-target experiments; GA trains for four epochs in the main setting.Source: §H.2 · p. 42

I Additional Experiments

• Figure 7 presents a partial-layer unlearning experiment on LLaMA3-Instruct (8B).Source: Figure 7 · p. 42

J Case Study

• ICU and RT usually lead the model to refuse, whereas GA, DPO, and NPO incline it to give an erroneous alternative answer.The case study covers LLaMA3-Instruct (8B).Source: J Case Study · p. 42