Source-linked AI summary
Selective Forgetting: Advancing Machine Unlearning Techniques and Evaluation in Language Models
Lingzhi Wang, Xingshan Zeng, Jinsong Guo, Kam-Fai Wong, Georg Gottlob
TL;DR
Neural models may retain personal or sensitive information from training data, while deleting that data from a model is difficult because its relationship to model weights is unclear. The paper proposes selective, fine-grained unlearning with sensitive-span annotation and evaluation metrics. SEUL reports better sensitive-information forgetting than KUL at matched general-information leakage measures, with comparable classification accuracy and stronger dialogue performance.
Problem
Neural models may retain sensitive training data, and deleting data from a model is challenging because the relationship between its weights and training data is unclear.
Source: Introduction · p. 1Method
SEUL selectively unlearns specific sequence spans and is supported by automatic online and offline sensitive-span annotation and the S-EL and S-MA metrics.
Source: Introduction · pp. 1–2Results
With EL10 and MA matched to KUL, SEUL reports better S-EL10 and S-MA scores, comparable classification accuracy, and significantly better average F1 and PPL on dialogue datasets.
Source: Main Results · pp. 5–6Takeaways & Limitations
The paper’s approach addresses concerns about inadvertent retention of personal or sensitive information by neural models.
Source: Conclusion · p. 7Online span selection assumes privacy-risk content is uncommon and therefore likely to have low language-model probability.
Source: How to Determine the Forgetting Span? · p. 3
Abstract
from arXiv · showhide
This paper explores Machine Unlearning (MU), an emerging field that is gaining increased attention due to concerns about neural models unintentionally remembering personal or sensitive information. We present SeUL, a novel method that enables selective and fine-grained unlearning for language models. Unlike previous work that employs a fully reversed training objective in unlearning, SeUL minimizes the negative impact on the capability of language models, particularly in terms of generation. Furthermore, we introduce two innovative evaluation metrics, sensitive extraction likelihood (S-EL) and sensitive memorization accuracy (S-MA), specifically designed to assess the effectiveness of forgetting sensitive information. In support of the unlearning framework, we propose efficient automatic online and offline sensitive span annotation methods. The online selection method, based on language probability scores, ensures computational efficiency, while the offline annotation involves a two-stage LLM-based process for robust verification. In summary, this paper contributes a novel selective unlearning method (SeUL), introduces specialized evaluation metrics (S-EL and S-MA) for assessing sensitive information forgetting, and proposes automatic online and offline sensitive span annotation methods to support the overall unlearning framework and evaluation process.
Introduction
Neural models can retain sensitive training data, while the connection between model weights and individual data makes deletion difficult. The paper proposes span-selective unlearning, sensitive-information metrics, and automatic span annotation methods.
Introduction
- Machine unlearning has gained attention because Internet-sourced training data may lead neural models to retain personal or sensitive information, amid data-erasure laws and concerns about language-model generation.Source: Introduction · p. 1
- The difficulty of neural-model deletion is that the relationship between model weights and training data is unclear, despite users’ right to request data erasure.Source: Introduction · p. 1
- Prior work has focused more on computer-vision classification than NLP generation, while fully reversing the training objective can harm language-model generation.Source: Introduction · p. 1
- SEUL targets specific sequence spans rather than whole instances, aiming to forget knowledge while better preserving language-model generation.The authors contrast this with fully reversed unlearning objectives.Source: Introduction · pp. 1–2
- The paper proposes online probability-based and offline LLM-based sensitive-span annotation, alongside S-EL and S-MA for evaluating sensitive-information forgetting.The offline process includes annotation and verification.Source: Introduction · p. 2
Related Work
Prior machine-unlearning work spans exact and approximate methods, varied datasets, and multiple evaluation metrics. The review notes limited dedicated datasets and the expense of some retraining-based evaluations.
Methodology
- Exact unlearning aims to eliminate deleted data’s impact but faces scalability and efficiency challenges, whereas approximate methods prioritize efficiency at the cost of exactness.Source: Related Work · p. 2
- Most unlearning methods were developed for computer vision, with more recent work addressing natural language.Source: Related Work · p. 2
Datasets
- No dedicated datasets exist for explicitly evaluating unlearning, so researchers use datasets selected for particular tasks and studies.Source: Related Work · p. 2
Evaluation Metrics
- Natural-language unlearning evaluations use measures including Δ-Acc, EL, MA, JSD, LPD, and PDLP; Wang et al. (2023) rely on retraining that can be prohibitively costly for large language models.Source: Related Work · p. 2
Selective Unlearning For LMs
SEUL targets designated sequence spans rather than reversing the training objective for entire instances. The section also describes span-selection and annotation procedures and sensitive-information-focused evaluation metrics.
Our Unlearning Methodology
- SEUL defines forget spans within each forget-set instance and minimizes a loss over those spans to avoid affecting other general knowledge.Each span is a continuous subsequence identified by its start index in the original sequence.Source: Our Unlearning Methodology · pp. 2–3
How to Determine the Forgetting Span?
- Online selection treats low-language-probability content as a privacy-risk signal and groups selected nearby tokens into forget spans.The method uses a predefined language-probability threshold or an average log-probability; tokens within two positions may be included together.Source: How to Determine the Forgetting Span? · p. 3
- Offline annotation uses LLM-generated sensitive spans followed by independent backward scoring, filtering out spans scored zero.The annotation uses few-shot prompts, and verification scores each generated span from 0 to 2 without showing the verifier the full sequence.Source: How to Determine the Forgetting Span? · p. 3
Evaluation of Unlearning
- S-EL measures a language model’s likelihood of generating sensitive information by assessing sensitive n-gram overlap in continuations from sequence prefixes.S-OVL, the overlap measure underlying S-EL, considers only n-grams containing sensitive spans.Source: Evaluation of Unlearning · pp. 3–4
- S-MA measures next-token prediction accuracy only for tokens within the sequence’s sensitive spans.The metric uses the model’s most probable next token and restricts evaluation to tokens marked as sensitive.Source: Evaluation of Unlearning · pp. 3–4
- The evaluation presents sensitive-focused S-EL and S-MA alongside extraction likelihood and memorization accuracy that do not distinguish sensitive tokens.The general metrics are denoted ELn and MA, respectively.Source: Evaluation of Unlearning · pp. 3–4
Experimental Setup
The setup evaluates unlearning on a shared forget set and tests whether general language-model capabilities are maintained. It compares SEUL with KUL under matched forgetting thresholds and reports model and training settings.
Datasets and Evaluation Metrics
- The forget set contains 15,000 samples of 200-token sequences from various Pile domains, matching the samples disclosed by Jang et al. (2023).The comparison table reports results for d=32 across the forget set, 8 classification datasets, and 4 dialogue datasets.Source: Datasets and Evaluation Metrics · p. 4, Table 1 · p. 5
- General capabilities are tested on 8 classification tasks and 4 dialogue tasks, using accuracy for classification and F1 and perplexity for dialogue.Unlearning effectiveness is assessed with S-ELn and S-MA, alongside ELn and MA.Source: Datasets and Evaluation Metrics · p. 5
Baselines and Implementation Details
- SEUL is primarily compared with Jang et al. (2023)'s KUL using their EL10- and MA-based forgetting thresholds; reported SEUL results use online selection.Offline-annotation results are labeled Oracle and compared in Table 4.Source: Baselines and Implementation Details · p. 5
- Experiments use GPT-Neo 125M, 1.3B, and 2.7B, Llama2-7B, and Mistral-7B; unless specified otherwise, results use d=32.Each setting is run five times and reported as the average; the learning rate is 5 × 10−5.Source: Baselines and Implementation Details · p. 5
Experimental Results
SEUL improves sensitive-information forgetting over KUL at matched general-leakage scores while preserving classification and generation performance. The experiments also examine forgetting over training and dataset sizes, adversarial attacks, and the reliability of sensitive-span annotations.
Main Results
- With EL10 and MA at the same levels for SEUL and KUL, SEUL scores better on S-EL10 and S-MA, indicating better protection against sensitive-information leakage.Source: Main Results · p. 5
Effectiveness Analysis of SEUL
- As unlearning progresses, Figure 3 shows minimal accuracy and F1 impact; with more forgetting instances, Figure 4 reports a declining accuracy trend and more stable F1.Source: Figure 3 · p. 6, Effectiveness Analysis of SEUL · p. 6, Figure 4 · p. 6
Main Results
- SEUL's average classification accuracy differs from KUL by 0.1 −0.3% across three GPT-Neo backbones, while SEUL significantly outperforms KUL on dialogue-dataset average F1 and PPL.Source: Main Results · p. 6
- SEUL and KUL require similar average epochs to reach the forgetting threshold, while GPT-Neo 125M has its largest performance drop when forgetting d = 32 instances.Source: Main Results · p. 6
Effectiveness Analysis of SEUL
- In the adversarial test, KUL cannot continue the injected commonsense sentence from the given prefix after unlearning, whereas SEUL can.Source: Effectiveness Analysis of SEUL · pp. 6–7, Table 2 · p. 6
- At S-EL10 and S-MA scores no higher than SEUL's, SEULOracle better maintains language-model performance, which the authors attribute to accurate annotation and fewer training epochs.Source: Effectiveness Analysis of SEUL · p. 7, Table 4 · p. 7
Reliability of Sensitive Span Annotation
- Across 50 instances, offline annotation matches human spans best; NER labels more spans with lower human overlap, while online selection may add inaccurate spans despite identifying sensitive information such as email addresses.Source: Reliability of Sensitive Span Annotation · p. 7, Table 5 · p. 7, Table 3 · p. 7
Conclusion
The conclusion presents SEUL as selective, span-focused unlearning intended to minimize capability impacts, alongside sensitive-information evaluation metrics and automatic span-annotation methods.
- SEUL focuses unlearning on specific sequence spans to minimize negative impacts on language-model capabilities.Source: Conclusion · p. 7
- S-EL and S-MA are specialized metrics for assessing the forgetting of sensitive information.Source: Conclusion · p. 7
- Automatic online and offline sensitive-span annotation methods support the unlearning framework.Source: Conclusion · p. 7