PaperlayerView exact revision on arXiv

Source-linked AI summary

MUSE: Machine Unlearning Six-Way Evaluation for Language Models

Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A. Smith, Chiyuan Zhang

arXiv:2407.06460v2cs.CLcs.AI

TL;DR

Language-model unlearning must remove private or copyrighted training data, but exact retraining-based removal is intractable and existing evaluations are too narrow. MUSE introduces a six-dimension benchmark spanning data-owner and deployer expectations, then finds that current methods reduce memorization yet often leak privacy, damage utility, and fail at scale or over sequential requests. The benchmark is released for further evaluation.

  • Problem

    Exact removal from modern language models is intractable, while prior evaluations do not fully measure unlearning success and practicality for data owners and deployers.

  • Method

    MUSE evaluates eight unlearning algorithms on 7B-parameter language models using six dimensions across Harry Potter books and news articles.

  • Results

    Most methods reduce verbatim and knowledge memorization, but current methods often cause utility loss, severe privacy leakage, and failures on sustainable sequential or large-scale removal.

  • Takeaways & Limitations

    Current unlearning methods are not yet ready for meaningful real-world deployment, motivating more robust methods and further benchmark evaluations.

  • Takeaways & Limitations

    MUSE does not cover all possible expectations and currently evaluates language-model unlearning only on books and news articles.

Abstract

from arXiv · show

Language models (LMs) are trained on vast amounts of text data, which may include private and copyrighted content. Data owners may request the removal of their data from a trained model due to privacy or copyright concerns. However, exactly unlearning only these datapoints (i.e., retraining with the data removed) is intractable in modern-day models. This has led to the development of many approximate unlearning algorithms. The evaluation of the efficacy of these algorithms has traditionally been narrow in scope, failing to precisely quantify the success and practicality of the algorithm from the perspectives of both the model deployers and the data owners. We address this issue by proposing MUSE, a comprehensive machine unlearning evaluation benchmark that enumerates six diverse desirable properties for unlearned models: (1) no verbatim memorization, (2) no knowledge memorization, (3) no privacy leakage, (4) utility preservation on data not intended for removal, (5) scalability with respect to the size of removal requests, and (6) sustainability over sequential unlearning requests. Using these criteria, we benchmark how effectively eight popular unlearning algorithms on 7B-parameter LMs can unlearn Harry Potter books and news articles. Our results demonstrate that most algorithms can prevent verbatim memorization and knowledge memorization to varying degrees, but only one algorithm does not lead to severe privacy leakage. Furthermore, existing algorithms fail to meet deployer's expectations because they often degrade general model utility and also cannot sustainably accommodate successive unlearning requests or large-scale content removal. Our findings identify key issues with the practicality of existing unlearning algorithms on language models, and we release our benchmark to facilitate further evaluations: muse-bench.github.io

1 Introduction

MUSE addresses the privacy and copyright risks of training language models on large text corpora by evaluating unlearning across data-owner and deployer expectations. Its results show that current methods remove memorization inconsistently while often sacrificing utility, privacy, scalability, and sustainability.

  • Motivation: Training language models on vast text collections can expose private and copyrighted content, motivating requests for data removal.These requests arise from privacy and copyright concerns, including regulatory and legal pressure.
  • MUSE benchmark: MUSE evaluates six expectations: preventing verbatim and knowledge memorization, preventing privacy leakage, preserving utility, scaling to large removals, and sustaining sequential requests.The dimensions cover both data-owner and model-deployer perspectives.
  • Evaluation: MUSE applies its benchmark to eight unlearning algorithms on Harry Potter books and news articles.The benchmark compares methods on real-world unlearning cases.
  • Findings: Most methods reduce verbatim and knowledge memorization, but these gains often come with reduced utility and do not reliably prevent privacy leakage.NPO and task vectors are especially effective against memorization, while NPO often permits privacy leakage and both methods sharply reduce utility.
  • Benchmark scope: MUSE differs from TOFU by evaluating real-world corpora described as 22× larger and incorporating six desiderata for owners and deployers.TOFU evaluates unlearning on synthetic question-and-answer datasets.
  • Implications: The authors conclude that feasible unlearning methods are not yet ready for meaningful real-world deployment and release MUSE to support further evaluation.They also invite extensions to other modalities.

2 Machine Unlearning: Preliminaries and Notations

Machine unlearning transforms a trained model so it behaves as though it had not seen a specified forget set. The setting distinguishes removed data from retained and unseen hold-out data while seeking to preserve performance on the latter.

  • Setting: Machine unlearning addresses data-removal requirements arising from privacy or copyright concerns.It is framed as an important model capability for such scenarios.
  • Notation: Given a trained model ftarget and forget set Dforget, an unlearning algorithm U outputs an unlearned model funlearn using the forget set and optionally the retain set.The retain set is Dtrain \ Dforget, and a disjoint hold-out set is also evaluated.

3 The MUSE Evaluation Benchmark

MUSE evaluates machine unlearning across six facets spanning data-owner expectations, deployment needs, and representative textual corpora. Its metrics assess memorization, privacy leakage, retained utility, scalability, and sustainability using verbatim text and derived question-answer sets.

  • MUSE evaluates six facets: no verbatim memorization, no knowledge memorization, no privacy leakage, utility preservation, scalability, and sustainability.
  • Data-owner expectations: Verbatim memorization compares model continuations with original forget-set continuations using ROUGE-L F1.
  • Data-owner expectations: Knowledge memorization averages ROUGE scores for answers to question-answer pairs derived from forget-set examples.
  • Data-owner expectations: Privacy leakage uses membership inference to distinguish forget-set members from holdout non-members, with PrivLeak comparing unlearned and retrained-model AUC.A good unlearning algorithm should have PrivLeak close to zero; positive or negative values indicate over- or under-unlearning.
  • Deployer expectations: Deployment metrics measure retained-set utility, performance trends as forget-set size increases, and performance trends across sequential unlearning requests.The benchmark motivates these criteria because unlearning can degrade capabilities and must accommodate large-scale and successive requests.
  • Evaluation corpora: MUSE constructs NEWS and BOOKS corpora with forget, retain, and holdout material, plus verbatim text and derived knowledge sets.NEWS uses disjoint news-article splits; BOOKS uses Harry Potter books for forgetting and related FanWiki content for retention.

4 Unlearning Methods

The benchmark evaluates eight approximate unlearning methods drawn from four algorithmic families, including gradient-based, task-vector, and interpolation-based approaches. Gradient ascent and negative preference optimization are combined with retain-set regularizers, while Task Vector and WHP use different unlearning mechanisms.

  • Method families: Gradient Ascent reduces the likelihood of correct predictions on the forget set by ascending the cross-entropy loss.Its reported utility effects vary across prior datasets and unlearning settings.
  • Alternative mechanisms: Task Vector unlearning subtracts a weight-difference vector obtained by overfitting the target model to the forget set.This moves the model away from the direction associated with adapting to the forget set.
  • Alternative mechanisms: WHP defines the unlearned model by interpolating between the target and reinforced models, controlled by α.Task Vector and WHP are not combined with the retain-set regularizers because their procedures are incompatible with those optimization strategies.
  • Utility regularization: Negative Preference Optimization is paired with retain-set regularizers to form additional candidate methods.The regularizers are Gradient Descent on the Retain Set and KL Divergence Minimization on the Retain Set.
  • Method families: Eight candidate methods are evaluated: GA, GAGDR, GAKLR, NPO, NPOGDR, NPOKLR, Task Vector, and WHP.The methods combine GA and NPO with GDR or KLR, alongside Task Vector and WHP.

5 Experiments

The experiments evaluate eight unlearning methods against data-owner and deployer expectations, including memorization removal, privacy leakage, utility preservation, scaling, and sequential requests. Most methods reduce memorization but trade this against utility, leak forget-set membership, and degrade as removal demands grow.

  • 5.2 Results: Data Owner Expectations: Most methods reduce verbatim and knowledge memorization, with GA and NPO reaching 0 for both metrics.These reductions can come at the cost of significant utility loss on the retain set.
  • 5.2 Results: Data Owner Expectations: Most methods reveal forget-set membership through either under-unlearning or over-unlearning, whereas fretrain has AUC = 0.47, near random guessing.GA and NPOGDR without regularizers over-unlearn with AUC > 0.7; NPOKLR and GAKLR tend to under-unlearn.
  • 5.2 Results: Data Owner Expectations: Min-K% Prob distributions show that approximate methods typically under-unlearn or over-unlearn rather than align Dforget with Dholdout.GAKLR under-unlearns, while NPOGDR over-unlearns and raises the metric across datasets, especially for Dforget.
  • 5.3 Results: Deployment Considerations: All unlearning methods compromise utility by 24.2% ∼100%, and GA, GAGDR, and NPOGDR can cause complete utility loss.The desired outcome is low memorization on Dforget while preserving utility on Dretain, but most methods trade knowledge removal for utility.
  • 5.3 Results: Deployment Considerations: Utility preservation decreases as the forget-set size grows, reaching a minimum at the largest tested size.The NEWS corpus is scaled from 0.8M to 3.3M tokens and evaluated at four forget-set sizes.
  • 5.3 Results: Deployment Considerations: Utility performance decreases significantly with the number of sequential unlearning requests, indicating that current methods are not sustainable for repeated removal.The experiment partitions a 3.3M-token NEWS forget set into four disjoint 0.8M-token folds and applies unlearning sequentially.

6 Related Work

Related work covers exact, certifiable, tractable, and localization- or context-based unlearning methods, alongside applications and evaluations for language models. Prior LM evaluations often focus on specific tasks, while this paper studies forgetting phrasing or content knowledge while preserving utility on untargeted data.

  • Machine unlearning for non-language model applications: Exact unlearning aims to make the unlearned model identical to a retain-set retrained model but is feasible mainly for simple models.Certifiable unlearning instead seeks probabilistic indistinguishability, while rigorous algorithms are too costly for modern language models.
  • Machine unlearning for language models: methods and applications: Language-model unlearning includes parameter-optimization methods and non-training approaches such as localization-informed and in-context unlearning.In-context unlearning treats the model as a black box and modifies outputs using external knowledge.
  • Machine unlearning for language models: methods and applications: This paper evaluates forgetting specific examples or datasets, targeting phrasing or content knowledge while preserving utility on untargeted data.The stated motivation is privacy and copyright compliance.
  • Machine unlearning for language models: evaluation: Previous LM evaluations have largely focused on specific tasks such as question answering or sentence completion.The related work describes task-specific evaluations including Harry Potter forgetting experiments and completion-based or token-probability-based familiarity measures.
  • Survey papers: Surveys categorize LM unlearning methods and datasets, review effectiveness and efficiency, and emphasize defining the unlearning scope clearly.The broader literature also includes surveys and a NeurIPS 2023 image-classification unlearning competition.

7 Conclusion

MUSE benchmarks six desirable properties for machine unlearning from both data-owner and model-deployer perspectives. Its results show that current methods reduce memorization but remain inadequate for privacy, utility, and scalable deployment.

  • 7 Conclusion: MUSE evaluates unlearning across six properties spanning data-owner and model-deployer expectations.The benchmark covers memorization, privacy leakage, utility preservation, scalability, and sustainability.
  • 7 Conclusion: Current methods prevent model memorization of removed content, but this comes at a significant utility cost on retained data.
  • 7 Conclusion: Existing methods still produce severe privacy leakage and cannot sustainably handle successive requests or large-scale content removal.
  • 7 Conclusion: MUSE does not cover every unlearning consideration, including intermediate-activation probing, formal guarantees, fine-tuning, in-context learning, efficiency, and storage cost.
  • 7 Conclusion: The benchmark currently evaluates language-model unlearning on books and news articles, with extensions to other corpora, model sizes, and modalities left for future work.

Appendices

The appendices situate MUSE within legal and social pressure for effective unlearning and acknowledge broader evaluation limits. These include fairness across user groups and the possibility that results may be misinterpreted as rejecting machine unlearning itself.

  • Appendices: Legal and social pressure has increased demand for models that support effective data removal by owners.
  • Appendices: MUSE is presented as a systematic way to evaluate strengths and weaknesses of unlearning methods amid rapidly changing external incentives.
  • Appendices: The evaluation does not comprehensively measure effects on different user bases, especially underrepresented groups.
  • Appendices: Fairness evaluation in machine unlearning remains active future work, while the authors caution that current results should not be read as criticism of the broader paradigm.
  • Appendices: All experiments were conducted on eight NVIDIA A40 GPUs in a single node.

B.2 Experimental Setup

The experimental setup fine-tunes separate 7B language models on news and book data, then applies unlearning methods under shared hyperparameters. Method-specific stopping parameters are selected using retained-data utility.

  • B.2 Experimental Setup: For NEWS, the setup starts from LLaMA-2 7B and fine-tunes on BBC news articles for five epochs.The learning rate is 10^-5 and the batch size is 32.
  • B.2 Experimental Setup: For BOOKS, the setup starts from ICLM 7B and fine-tunes on the Harry Potter books with the same hyperparameters.The learning rate is 10^-5 and the batch size is 32.
  • B.2 Experimental Setup: All unlearning methods use a constant learning rate of 10^-5 and batch size 32.The passage also specifies additional fine-tuning for reinforced models, but the provided text is truncated after the duration begins.
  • B.2 Experimental Setup: Optimal unlearning epochs or α values are selected using stopping criteria based on unlearned-model utility on D_retain relative to f_retrain.
  • B.2 Experimental Setup: Table 4 reports the optimal epochs or α values chosen for each unlearning method.

B.3 Efficiency of Unlearning Methods

Efficiency is measured as wall-clock time per unlearning gradient-update step. The reported setup standardizes hardware, batch size, sequence length, and processed-token count, with task-vector and WHP steps defined as reinforced-model fine-tuning iterations.

  • B.3 Efficiency of Unlearning Methods: Efficiency is measured by wall-clock seconds per unlearning gradient-update step.
  • B.3 Efficiency of Unlearning Methods: Measurements use eight NVIDIA A40 GPUs, batch size 32, and input length 2048 tokens.
  • B.3 Efficiency of Unlearning Methods: Each efficiency step processes 65,536 tokens, computed as 32 × 2048 tokens.
  • B.3 Efficiency of Unlearning Methods: For Task Vector and WHP, one step is one fine-tuning iteration used to create the reinforced model.
  • B.3 Efficiency of Unlearning Methods: Table 5 reports wall-clock time for each unlearning method in seconds per step.

C More Experimental Results

The paper reports 95% confidence intervals for mean ROUGE-L F1 scores used in C1, C2, and C4, computed through bootstrap resampling.

  • 9,999 bootstrap resamples yield two-tailed 95% confidence intervals for each reported mean ROUGE-L score.The intervals use the percentage method.
  • Table 6 presents the 95% confidence intervals used for mean ROUGE-L scores in C1, C2, and C4.

D Dataset Details

MUSE constructs NEWS and BOOKS datasets with verbatim and knowledge-based evaluation sets, while using GPT-4 to generate excerpt-grounded question-answer pairs. The dataset also separates standard retain data from a disjoint regularization retain set.

  • Knowledge-set construction: GPT-4 generates QA pairs from randomly sampled 2048-token excerpts partitioned from each corpus’s Verbatim text.Each answer is extracted verbatim from its corresponding excerpt.
  • Knowledge-set construction: The generation procedure starts a new GPT-4 conversation for each QA pair and presents the selected excerpt as the user prompt.Two generated examples come from the NEWS Knowledge set.
  • Knowledge-set construction: Each generated QA pair asks a question answerable only from excerpt-specific information and returns a precise, minimal, phrase-level verbatim answer in JSON.The system prompt requires question and answer fields and excludes answers based only on common knowledge.
  • Knowledge-set examples: The NEWS examples ask for specific facts, including the US share of Israel’s arms imports and the moderator of a Twitter Spaces event.The corresponding answers are 69% and tech entrepreneur David Sacks.
  • Corpus organization: MUSE includes NEWS and BOOKS corpora with Verbatim and Knowledge sets for evaluating verbatim and knowledge memorization.NEWS uses disjoint news-article forget and retain sets; BOOKS uses the Harry Potter series for forgetting and related wiki articles for retention.
  • Corpus organization: The regularization retain set is disjoint from the evaluation retain set and is used with GDR and KLR regularizers to preserve utility during unlearning training.Dataset statistics report corpus and retain-set sizes in tokens.
Loading 2407.06460v2…