PaperlayerView exact revision on arXiv

Source-linked AI summary

OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics

Vineeth Dorna, Anmol Mekala, Wenlong Zhao, Andrew McCallum, Zachary C. Lipton, J. Zico Kolter, Pratyush Maini

arXiv:2506.12618v2cs.CL

TL;DR

LLM unlearning needs to remove sensitive training-data influence without sacrificing utility, but fragmented methods and unreliable evaluations hinder rigorous comparison. OpenUnlearning unifies benchmarks, algorithms, metrics, and stress tests, then meta-evaluates metric faithfulness and robustness. The framework identifies ES and EM as particularly reliable metrics and finds strong performance from SimNPO and RMU, while highlighting ranking and benchmark limitations.

  • Problem

    LLM unlearning research is fragmented, and reliably evaluating forgetting alongside retained utility, privacy, and safety remains difficult.

  • Method

    OpenUnlearning provides a standardized, extensible framework integrating unlearning methods, evaluation metrics, benchmarks, stress tests, and a ground-truth model testbed for metric meta-evaluation.

  • Results

    ES and EM are identified as particularly reliable metrics, Truth Ratio shows high faithfulness, and SimNPO and RMU are reported as strong unlearning methods.

  • Takeaways & Limitations

    The framework supports more systematic comparison of unlearning methods and the development of more faithful and trustworthy evaluation metrics.

  • Takeaways & Limitations

    The framework is limited by popular benchmarks that may not reflect realistic learning or unlearning dynamics, and its findings require extension to more setups and algorithms.

Abstract

from arXiv · show

Robust unlearning is crucial for safely deploying large language models (LLMs) in environments where data privacy, model safety, and regulatory compliance must be ensured. Yet the task is inherently challenging, partly due to difficulties in reliably measuring whether unlearning has truly occurred. Moreover, fragmentation in current methodologies and inconsistent evaluation metrics hinder comparative analysis and reproducibility. To unify and accelerate research efforts, we introduce OpenUnlearning, a standardized and extensible framework designed explicitly for benchmarking both LLM unlearning methods and metrics. OpenUnlearning integrates 13 unlearning algorithms and 16 diverse evaluations across 3 leading benchmarks (TOFU, MUSE, and WMDP) and also enables analyses of forgetting behaviors across 450+ checkpoints we publicly release. Leveraging OpenUnlearning, we propose a novel meta-evaluation benchmark focused specifically on assessing the faithfulness and robustness of evaluation metrics themselves. We also benchmark diverse unlearning methods and provide a comparative analysis against an extensive evaluation suite. Overall, we establish a clear, community-driven pathway toward rigorous development in LLM unlearning research.

1 Introduction

LLM unlearning seeks to remove the influence of specific training data while preserving general model behavior, but fragmented methods and inconsistent evaluations impede rigorous comparison. OpenUnlearning addresses this gap with a unified, extensible benchmark for methods and metrics.

  • Motivation: LLMs can memorize sensitive, copyrighted, or harmful training content, while retraining for deletion requests is often prohibitively costly.Unlearning instead uses efficient post-training interventions to remove undesirable data influence while maintaining overall performance.
  • Research gap: The rapid growth of LLM unlearning research has produced many methods and benchmarks but no consensus on the best evaluations.Assessing knowledge, privacy, and utility is described as potentially as difficult as achieving unlearning itself.
  • Contribution: OpenUnlearning introduces a unified and extensible benchmark intended to standardize, scale, and accelerate LLM unlearning research.The framework is presented as a shared environment where methods and evaluations can be iteratively improved.
  • Contribution: The framework meta-evaluates unlearning metrics using more than 450 open-sourced models with known ground-truth states.It compares 12 metrics for faithfulness and robustness, forming a benchmark for assessing evaluation methods.
  • Benchmarking: OpenUnlearning compares 8 unlearning methods with 10 metrics and reports SimNPO as the best-performing method under the applied ranking procedure.The authors also note limitations in that ranking methodology and release evaluated checkpoints for further research.
  • Adoption: Since its March 2025 release, the project has received 250+ GitHub stars and 20k+ model downloads across 450+ public checkpoints.Popular unlearning benchmarks also point to the repository as their official maintenance location.

2 Overview of LLM Unlearning

LLM unlearning aims to remove a forget set’s influence while retaining utility on unrelated tasks, using weight-based interventions and benchmark-specific evaluations. Existing benchmarks span fine-grained knowledge, privacy, memorization, and safety-oriented settings, but evaluation reliability remains contested.

  • Definition: LLM unlearning removes the influence of a forget set from a target model while retaining utility on unrelated downstream tasks.A disjoint retain set may support and assess preservation of model performance.
  • Evaluation: Unlearning efficacy is measured by metrics that quantify remaining forget-set influence, while utility metrics assess performance outside that data.The framework distinguishes forgetting measurements from preservation of general capabilities.
  • Methods: Weight-based methods commonly use tailored losses, optimization changes, localized parameter updates, or alternative data.Prompting-based approaches detect sensitive queries at inference time but are described as impractical to scale as forgetting accumulates.
  • Benchmarks: Benchmarks cover fine-grained knowledge, copyright, privacy, memorization, and other unlearning settings across datasets such as TOFU and MUSE.TOFU uses QA-style data about fictitious authors, while MUSE targets knowledge, memorization, and privacy influence in news and copyrighted material.
  • Evaluation: Unlearning evaluations range from probability judgments to membership-inference attacks, but later work has questioned metric reliability.The literature contains dozens of metrics across benchmarks, making evaluation difficult.

3 OpenUnlearning

OpenUnlearning consolidates fragmented benchmarks, methods, metrics, datasets, and stress tests into a modular pipeline. Its extensible design supports broad component coverage, cross-benchmark evaluation, and community additions through reusable handlers and configuration files.

  • Fragmentation: Existing LLM unlearning research lacks unified technical implementations and evaluation methodology, hindering reproducibility and comprehensive comparison.Benchmarks and components are difficult to extend, and methods or metrics are not consistently available across tasks.
  • Fragmentation: UNDIAL is absent from TOFU, MUSE, and WMDP, while NPO uses different formulations across TOFU and MUSE and RMU appears only for WMDP.Evaluation coverage is similarly fragmented: MUSE’s MIA is not implemented in TOFU.
  • Framework: OpenUnlearning consolidates benchmarks, methods, metrics, datasets, and stress tests in one extensible pipeline.The goal is to streamline implementations and benchmarking while accelerating research.
  • Design of modules: New module variants are added by registering a handler and defining a YAML configuration that names the handler and its parameters.This modular design lets researchers integrate datasets, metrics, methods, and complete benchmarks.
  • Features: The framework supports 13 algorithms, 8 model architectures, 5 datasets, 16 unlearning metrics, and 3 stress-testing approaches.It focuses on TOFU, MUSE, and WMDP and enables metrics developed for one benchmark to be used on another.
  • Extensibility: The living framework has expanded from TOFU and MUSE to WMDP, additional methods, new evaluations, and cross-benchmark integrations.Its current inventory includes 3 benchmarks, 5 datasets, 13 methods, 16 metrics, 8 architectures, and 3 stress tests.

4 Evaluating Unlearning Evaluations

OpenUnlearning evaluates whether unlearning metrics faithfully detect target knowledge and remain robust under benign and non-benign interventions. Its meta-evaluation uses ground-truth model pools and aggregates faithfulness with robustness to compare metrics systematically.

  • Desiderata: Reliable unlearning metrics must distinguish models with and without target knowledge while remaining stable under stress-testing interventions.Faithfulness concerns knowledge detection; robustness concerns benign changes and behavior relative to a retain model under non-benign changes.
  • Faithfulness: The faithfulness evaluation compares diverse positive and negative model pools, testing whether metrics detect knowledge rather than training-data artifacts.Each pool contains 30 models, with positive models exposed to target information in varied formats and negative models lacking it.
  • Results: For ROUGE, faithfulness has AUC 0.79, relearning robustness is 0.48, and quantization robustness is 0.93.The results indicate prediction overlap for faithfulness, faster knowledge recovery for many models under relearning, and no distinctive quantization-related metric spikes.
  • Robustness: Robustness testing probes metric behavior after relearning, information extraction, and quantization, while filtering models with large utility drops or insufficient unlearning.The analysis uses roughly 400 diverse models from multiple unlearning methods to reflect practical model distributions.
  • Results: Extraction Strength is most reliable overall, while Truth Ratio has stronger faithfulness but lower robustness and MIA metrics show high faithfulness with limited robustness.The study aggregates robustness with a harmonic mean and then combines faithfulness and robustness into an overall assessment.
  • Implications: The released 450+ model testbed supports continued development and expansion of more faithful and trustworthy unlearning metrics.The framework is designed to incorporate additional setups, architectures, methods, and adversarial model configurations.

5 Benchmarking Unlearning Methods

OpenUnlearning provides a standardized comparison of unlearning methods using multiple metrics across memorization, privacy, and utility. On TOFU, SimNPO ranks first under harmonic-mean aggregation, while ranking sensitivity remains a key limitation.

  • Benchmark setup: OpenUnlearning evaluates 8 unlearning methods using 10 metrics on the TOFU benchmark.The framework covers methods including SimNPO, RMU, AltPO, NPO, UNDIAL, IdkPO, IdkNLL, and GradDiff.
  • Evaluation design: The composite evaluation aggregates four memorization, four privacy, and two utility metrics into a single score.Models receive 27 hyperparameter tuning trials per method, while validation uses accessible memorization and utility metrics.
  • Results: SimNPO ranks first under harmonic-mean aggregation across memorization, privacy, and utility.It remains close to the retain model on memorization, fully preserves utility, and achieves competitive privacy results.
  • Results: RMU is the next best performer, with strong memorization and privacy but a significant utility drop.The comparison highlights a balance among distinct dimensions rather than optimizing memorization alone.
  • Results and discussion: Memorization and privacy objectives create a tradeoff: under-unlearning can lower privacy scores, whereas over-unlearning can reduce utility.The paper therefore emphasizes detecting when model behavior has returned to a default state during unlearning.
  • Limitations: Different ranking schemes can produce very different method rankings, and metric selection lacks standardization across model selection and final evaluation.The authors release all evaluated checkpoints to support future research on fair evaluation.

6 Conclusion

OpenUnlearning addresses fragmented LLM unlearning research with a standardized, extensible platform for evaluating methods and metrics. Its meta-evaluation and released checkpoints support more rigorous and reproducible development.

  • Conclusion: OpenUnlearning integrates 13 unlearning algorithms, 16 evaluation metrics, and 3 major benchmarks into one framework.It supports both meta-evaluation of metrics and large-scale benchmarking of unlearning methods.
  • Conclusion: The meta-evaluation identifies Extraction Strength and Exact Memorization as particularly reliable, while Truth Ratio shows high faithfulness.Benchmarking also identifies SimNPO and RMU as strong performers, alongside sensitivity in ranking procedures.
  • Conclusion: OpenUnlearning releases numerous model checkpoints to provide common infrastructure for more rigorous and reproducible unlearning research.The framework is presented as a pathway toward improved unlearning techniques and evaluation protocols.

A Limitations

The framework and analysis remain bounded by existing benchmarks, limited experimental coverage, and a narrow set of metric desiderata.

  • Benchmark scope: The supported benchmarks may be weak measures of unlearning progress and may not reflect realistic learning or unlearning dynamics.The authors call for realistic, controlled, fine-grained benchmarks beyond currently popular setups.
  • Evaluation coverage: The meta-evaluation and method comparison need extension to more unlearning setups and algorithms.
  • Metric scope: The analysis treats knowledge faithfulness and metric robustness as minimal desiderata rather than a comprehensive definition of good metrics.
  • Implications: The framework’s practical adoption is presented as lowering barriers for academic and industrial use and supporting governance workflows.The broader deployment implications are described as long-term prospects for safety-critical and regulated settings.

C.1 Unlearning benchmarks

OpenUnlearning consolidates diverse unlearning benchmarks, datasets, metric families, and extensible data-handling components for evaluating forgetting, privacy, memorization, and retained utility.

  • Benchmarks: TOFU evaluates fine-grained knowledge unlearning on 200 fictitious author profiles with QA pairs and uses Truth Ratio as its primary metric.
  • Benchmarks: MUSE targets removal of books and news articles, measuring memorization, knowledge, and privacy through reproduction, QA, and membership-inference evaluations.
  • Benchmarks: WMDP probes hazardous biosecurity, cybersecurity, and chemical-security knowledge while testing preservation of general performance.It contains 3,668 multiple-choice questions paired with unlearning corpora and off-the-shelf chat LLMs.
  • Benchmark extensions: OpenUnlearning mitigates benchmark limitations by adding paraphrased holdouts, privacy metrics, additional membership-inference attacks, new starter models, and expanded metrics.The framework adds LLAMA-3 models at 1B, 3B, and 8B sizes, plus Extraction Strength, Exact Memorization, and Forget Fluency.
  • Framework design: Its shared dataset handlers support pretraining, completion, and QA formats for training, memorization or leakage evaluation, and interactive knowledge probing.Most dataset-class implementation is reusable across configurations and benchmarks, while custom classes can be added through the Figure 5 procedure.
  • Evaluation metrics: Metrics span memorization, privacy, utility, probability, overlap, and prompt-manipulation evaluations, but their applicability can depend on dataset structure.For example, ROUGE recall may capture templates rather than semantic content when answer formats differ across benchmarks.

C.5 Unlearning Methods

OpenUnlearning standardizes implementation across unlearning methods and benchmarks, reducing adaptation overhead while supporting diverse objectives and scalable training configurations.

  • Motivation: Cross-benchmark comparisons are hindered because researchers often evaluate methods on one benchmark and must reimplement code, pipelines, and metric alignment.
  • Unified infrastructure: Researchers can implement a method once through a custom Trainer and evaluate it across multiple benchmarks using infrastructure that abstracts benchmark-specific details.
  • Implemented methods: Supported baselines include gradient ascent, retain-set gradient descent, “I don’t know” objectives, NPO variants, DPO-style objectives, and representation manipulation.
  • Implemented methods: SimNPO replaces the NPO reference model with δ while preserving the core forgetting behavior.
  • Implemented methods: AltPO aligns generation toward alternate plausible in-domain facts to introduce ambiguity and suppress target knowledge.
  • Implemented methods: RMU suppresses forget-set memorization signals in model representations while preserving retain-set knowledge, using randomly sampled u and scaling parameter c.
  • Implemented methods: UNDIAL uses self-distillation and adjusted logits, minimizing KL divergence to reduce target-token confidence without affecting overall model performance.

D Experimental setup

The experiments use LLAMA-3.2-1B and a deliberately varied model pool to test metric behavior under relearning, quantization, and probing interventions.

  • Experimental configuration: All meta-evaluation and benchmarking experiments use LLAMA-3.2-1B with BF16 precision, one NVIDIA A100, batch size 32, and paged AdamW.
  • Model pools: The model pools distinguish positive models trained on target knowledge from negative models trained without that knowledge, with multiple transformed data variants.
  • Model pools: 60 models result from combining two pools, three dataset variants, five learning rates, and two checkpoints.
  • Data generation: Training variants were generated independently using LLAMA 3.1 405B for paraphrases and perturbations and Gemini for Wikipedia-style biographies.
  • Stress testing: The unlearned model pool is compared with retain-model references before and after interventions designed to provoke recovery or induction of forgotten knowledge.
  • Stress testing: Interventions include one-epoch relearning, 4-bit quantization, and layer-11 probing with a retain90-trained language-model head.

E.3 Additional Results

OpenUnlearning evaluates metric faithfulness and robustness through gold-standard comparisons and relearning, quantization, and probing stress tests. The analyses show that some metrics fail to capture residual knowledge and that filtering for utility affects robustness findings.

  • Evaluation setting: Removing the filter requiring at least 80% retained utility reduces robustness to quantization further.The authors retain the filter to align with common unlearning reporting practices.
  • Probing stress test: Probing is excluded from the meta-evaluation because all metrics achieve P scores of 1 on TOFU, providing little information.OpenUnlearning nevertheless supports probing as an intervention.
  • Practical considerations: Stress-testing interventions are better suited to final-stage audits because they require model-weight access and, for relearning and quantization, additional computation.The analysis is intended to inform robust metrics that work without stress-testing.
  • Metric meta-evaluation: Metrics are evaluated for faithfulness, relearning robustness, and quantization robustness against a gold-standard retain model.The framework benchmarks metric behavior under focused interventions rather than assuming linear metric changes.
  • Metric meta-evaluation: Faithfulness measures how effectively metrics distinguish models trained on target knowledge from models that are not.Figure 8 reports this distinction using AUC.
  • Relearning stress test: A significant fraction of unlearned models regain knowledge faster than the retained model after relearning, indicating that some metrics initially missed residual knowledge.These models fall into the unreliable red-shaded region.
  • Quantization stress test: For several metrics, some unlearned models show increased values after quantization, suggesting those metrics are not robust to the intervention.The reported subsets use models unlearned with low learning rates; utility filtering differs between Figures 10 and 11.

F Further discussion on benchmarking unlearning methods

The benchmarking suite aggregates memorization, privacy, and utility metrics while using development-available objectives for model selection. Its discussion cautions that memorization-focused rankings can reward over-unlearning and that existing benchmarking leaves important evaluation questions unresolved.

  • Metric aggregation: The suite evaluates three dimensions: memorization, privacy, and utility.Multiple metrics are considered within each dimension and aggregated using harmonic means.
  • Metric aggregation: The Memorization Score is the harmonic mean of inverted ES, EM, Paraphrased Probability, and Truth Ratio scores.Inversion makes higher scores represent more effective unlearning.
  • Metric aggregation: The Privacy Score aggregates MIA-based scores for LOSS, ZLib, Min-k, and Mink++, each measuring similarity to a gold-standard retain model.A higher individual sMIA score indicates greater similarity to the retain model.
  • Metric aggregation: The Utility Score combines TOFU performance across retain, real-world-author, and wrong-fact splits with response fluency using a harmonic mean.The component metrics are scaled against the initial finetuned model into the [0, 1] range.
  • Metric aggregation: Harmonic means ensure that a high aggregate score requires strong performance across all constituent metrics.This aggregation principle is used throughout the metric suite.
  • Model selection: During tuning, the validation objective is the harmonic mean of Memorization and Utility scores because these metrics are realistically available during development.Privacy scores are avoided because they require a retain model.
  • Benchmarking results: Table 6 aggregates only memorization and utility for TOFU, and its memorization-focused setup makes GradDiff perform best by enabling over-unlearning.Privacy scores are shown only for illustration.
  • Benchmarking limitations: Existing benchmarking remains limited because ES alone does not cover privacy protections, all utility facets, or the unresolved separation between tuning and leaderboard metrics.The authors identify these issues as an important direction for further research.
Loading 2506.12618v2…