PaperlayerView exact revision on arXiv

Source-linked AI summary

In-Context Unlearning: Language Models as Few Shot Unlearners

Martin Pawelczyk, Seth Neel, Himabindu Lakkaraju

arXiv:2310.07579v4cs.LGcs.AIcs.CR

TL;DR

LLM unlearning is difficult when models are black boxes or too large for repeated parameter updates, motivating methods that avoid retraining. ICUL uses inference-time contexts with changed labels on forget points, and experiments report effective removal of their influence without parameter access. The method is presented as a practical complement to parameter-based unlearning, though larger question-answering deletion requests can substantially reduce accuracy.

  • Problem

    LLM unlearning is constrained by unavailable parameter access in black-box models and the computational infeasibility of repeatedly updating very large models.

  • Method

    ICUL constructs inference-time prompts containing targeted points with changed labels or answers alongside correctly labeled examples, without updating model parameters.

  • Results

    LiRA-Forget cannot reliably distinguish held-out points from training points subsequently unlearned by ICUL, while experiments report effective influence removal across established classification and question-answering datasets.

  • Takeaways & Limitations

    ICUL offers a black-box-compatible perspective on LLM unlearning and can serve as a practical patch until a model can be updated or retrained.

  • Takeaways & Limitations

    For larger deletion requests in question answering, ICUL produced more than a 15% post-unlearning accuracy drop, motivating improved prompt designs.

Abstract

from arXiv · show

Machine unlearning, the study of efficiently removing the impact of specific training instances on a model, has garnered increased attention in recent years due to regulatory guidelines such as the \emph{Right to be Forgotten}. Achieving precise unlearning typically involves fully retraining the model and is computationally infeasible in case of very large models such as Large Language Models (LLMs). To this end, recent work has proposed several algorithms which approximate the removal of training data without retraining the model. These algorithms crucially rely on access to the model parameters in order to update them, an assumption that may not hold in practice due to computational constraints or having only query access to the LLMs. In this work, we propose a new class of unlearning methods for LLMs called ``In-Context Unlearning.'' This method unlearns instances from the model by simply providing specific kinds of inputs in context, without the need to update model parameters. To unlearn specific training instances, we present these instances to the LLMs at inference time along with labels that differ from their ground truth. Our experimental results demonstrate that in-context unlearning performs on par with, or in some cases outperforms other state-of-the-art methods that require access to model parameters, effectively removing the influence of specific instances on the model while preserving test accuracy.

1. Introduction

Machine unlearning seeks to remove specific training points without costly retraining, but LLMs make parameter-based methods difficult because models may be black boxes or too large to update repeatedly. ICUL addresses this by constructing inference-time contexts with flipped labels, achieving competitive unlearning without parameter access.

  • LLM unlearning is challenging because black-box models hide parameters and updating very large models for each request may be computationally infeasible.
  • ICUL constructs contexts containing targeted training points with randomly flipped labels alongside correctly labeled examples, without accessing model parameters.
  • ICUL achieves competitive or sometimes superior performance to parameter-access unlearning methods while avoiding expensive gradient computations.
  • Experiments on AG-News, SST-2, SQUAD, and Amazon reviews show that ICUL practically eliminates training-point influence on classification and question-answering outputs.
  • ICUL requires less memory than Gradient Ascent; on Llama-2 7B it runs on a 32GB Tesla V100, whereas Gradient Ascent requires an 80GB A100.

2. Related Work

Related work has developed approximate unlearning to avoid retraining, but most prior methods target discriminative models and update parameters. This paper extends the area to LLMs and introduces a likelihood-ratio-based empirical evaluation of forgetting.

  • In-context learning adapts language models using examples in the input context rather than explicitly updating model weights.
  • Machine unlearning removes the influence of selected training points while avoiding computationally expensive full retraining.
  • Approximate unlearning methods for non-convex models generally lack theoretical guarantees and therefore require empirical evaluation.
  • The paper proposes LiRA-Forget, which adapts likelihood-ratio membership inference to test whether unlearned points resemble points absent from training.
  • Prior research mostly studied discriminative classifiers, often forgetting entire classes through gradient updates on deleted or retained points.

3. Preliminaries

The preliminaries formalize in-context learning and approximate unlearning as matching the distribution of an unlearned model to exact retraining without the forget set. LiRA-Forget operationalizes this goal through likelihood-ratio hypothesis testing and shadow models.

  • In-Context Learning: In-context learning supplies demonstrations and a query to a pretrained language model, which generates an output sequence without fine-tuning its weights.
  • Unlearning Definition: Approximate unlearning aims to minimize the distributional distance between a model produced by unlearning and one retrained without the forget set.
  • Operationalization: Shadow models trained on subsets with or without the forget set approximate the loss distributions needed for the likelihood-ratio test.
  • LiRA-Forget: The evaluation reframes distributional distance as testing whether an optimal hypothesis test can distinguish unlearned models from retrained models.
  • LiRA-Forget: LiRA-Forget uses likelihood ratios over losses on forget points rather than the intractable full model distribution.

4. Our Framework: In-Context Unlearning

ICUL uses inference-time prompts to make an LLM behave as though selected training points had not been seen. The prompts combine mislabeled forget examples with correctly labeled demonstrations before prediction.

  • ICUL avoids retraining and parameter updates by constructing an inference-time context that makes the LLM behave as if a targeted point were absent from training.
  • Changing labels or answers on targeted points reduces model confidence on those instances toward the level expected when they were never in training.
  • For a deletion request of size K, ICUL randomly flips the labels of the K forget points and places them at the start of the prompt.
  • ICUL then appends L randomly sampled correctly labeled examples, followed by the query input for prediction.
  • The final prompt is evaluated by next-token prediction with temperature t = 0.

5. Empirical Evaluation

Across classification and question-answering tasks, ICUL unlearns targeted instances using only inference-time contexts while retaining competitive test performance. Results cover multiple datasets and model families, and ablations show that flipped labels, sufficient context, and the actual forget point are important.

  • Experimental setup: ICUL evaluates in-context unlearning across SST-2, Amazon polarity, AG-News, and SQUAD using Bloom and Llama2 models.The experiments span classification and question-answering settings, with Bloom models from 560M to 3B and Llama2 at 7B.
  • Unlearning efficacy: ICUL achieves lower LiRA-Forget TPRs than GA at FPR=0.01 in 14 of 16 cases while requiring only black-box model access.Its curves remain close to the random-guessing Benchmark and consistently outperform the no-unlearning Baseline.
  • Model performance: ICUL maintains similar test accuracy across deletion-request sizes, whereas GA accuracy drops significantly for 20 deletions.On Amazon, ICUL test accuracy stays within 1% of the Baseline for deletion requests below 20 as well.
  • Sensitivity to model size: 54.05% improvement over the Baseline occurs for the 7.1B Bloom model, compared with 41.67% for the 560M model.Post-unlearning test accuracy also rises from 86.1% for 560M to 89% for 3B.
  • Sensitivity across LLMs: On Llama2 7B, TPR at FPR=0.01 closely aligns with the Benchmark for 1, 10, and 20 deletion requests.This indicates comparable forgetting efficacy across the evaluated Bloom and Llama2 model classes.
  • Question answering: On SQUAD, privacy leakage falls 24% for 5 deletions and 43% for 10, while accuracy drops 5.5% and 16.7%, respectively.ICUL performs close to the Benchmark, with the larger deletion request producing the larger accuracy reduction.

6. Conclusion

The paper presents ICUL as parameter-free in-context unlearning and reports that it can remove training-point influence, while identifying important limitations for task complexity, deletion size, prompt length, accuracy, privacy, and evaluation cost.

  • ICUL unlearns without model-parameter access by prompting with targeted data points, changed labels, and accurately labeled instances.
  • LiRA-Forget cannot reliably distinguish held-out points from training points subsequently unlearned by ICUL.
  • ICUL’s effectiveness and evaluation for open-ended data generation remain unclear and require further investigation.
  • Larger deletion requests can make the current context design infeasible and increase test-time runtime through longer prompts.
  • For question answering, deleting 10 points caused post-unlearning accuracy to drop by more than 15%.
  • LiRA-Forget requires multiple shadow models, creating computationally infeasible evaluation costs for large models.

Impact Statement

The paper frames ICUL as a tool for making finetuned LLMs behave as if selected data points were removed, with relevance to algorithmic decision-making and responsible practice.

  • ICUL is presented as a tool for decision makers seeking to make finetuned models behave as if specific data points had been removed.
  • Potential application domains include healthcare, education, insurance, credit scoring, recruitment, and criminal justice.
  • The authors caution that ICUL can fail under specific hyperparameter configurations, including too few correctly labeled examples.
  • The work provides practitioners with insights and tools for responsible unlearning in LLM-based classification and question answering.

A. Reproducibility Statement

The reproducibility statement reports the hardware, storage, and approximate compute used for the experiments, including separate requirements for shadow models, finetuning, and ICUL inference.

  • Reproducing all experiments requires setting aside 1500 GB of storage for model weights of LiRA-Forget shadow models.
  • Bloom experiments use Tesla V100 GPUs with 32 GB RAM, while Llama2 7B finetuning uses one A100 GPU with 80 GB RAM.
  • ICUL unlearning runs on Tesla V100 GPUs with 32 GB RAM.
  • LiRA-Forget evaluation finetunes 10 LLMs for each dataset and forget-set size, requiring roughly 40 GPU hours per dataset before additional search and inference costs.
  • SST-2 sensitivity experiments require approximately 5, 10, and 14 finetuning hours for 560M, 1.1B, and 3B models, respectively.
  • The Llama-2 7B effectiveness experiment required roughly 385 GPU hours in total.

B. Details on the Machine Unlearning Evaluation

The evaluation estimates likelihood-ratio distributions with shadow models and confidence scores, then compares computational resources and runtimes for parameter-updating GA and parameter-free ICUL.

  • The likelihood-ratio audit requires estimating loss distributions under null and alternative hypotheses, ideally from retrained and unlearned models.
  • The evaluation adapts sample splitting to forget-set sizes J = {1, 5, 10, 20} using shadow models trained with and without each forget set.
  • Ten shadow models are trained with p = 0.5, and Gaussian distributions are fitted to confidence scores from retain and unlearned models.
  • The confidence score compares probability assigned to the true class with probabilities assigned to all other vocabulary classes; higher values indicate greater confidence in the correct prediction.
  • Table 2 compares computational resources for updating Llama2 7B on SST-2 across unlearning methods, including memory and model-update time.
  • Table 3 reports inference runtimes for GA and ICUL across different numbers of deletions.

C.1. Additional Empirical Comparisons: Compute Times and Memory Requirements

The comparison evaluates GA and ICUL for a Llama2 (7B) model on SST-2 across memory requirements and computation times. It also notes that more efficient long-context transformers could improve ICUL's utility as inference becomes faster.

  • Compute and memory comparison: The comparison measures memory requirements and computation times for GA and ICUL using Llama2 (7B) on SST-2.The evaluation covers model-update memory, model-update time, and inference time with the updated model.
  • Compute and memory comparison: ICUL is designed for GPU RAM-constrained compute environments, trading increased computation time for lower fine-tuning memory demands.The passage frames additional computation time as potentially acceptable when increasing GPU RAM for fine-tuning is difficult.
  • Compute and memory comparison: The analysis separately identifies model-update memory, model-update time, and inference time as comparison dimensions.These dimensions correspond to Tables 2a, 2b, and 3, respectively.
  • Future efficiency: Transformer advances with linear computation complexity in sequence length may further improve ICUL as larger-context inference becomes faster.Ding et al. (2023) is cited as an example of improving computational efficiency for larger context lengths without sacrificing performance.

C.2. Additional Empirical Comparisons: Test Accuracy and Unlearning Efficacy

The additional experiments include Yelp polarity and examine test accuracy and unlearning efficacy through forget-set size, AUC, learning-rate, and context-length analyses. The supplied figure descriptions specify the evaluated conditions but do not report numerical outcomes.

  • Datasets: The additional evaluation includes the Yelp polarity dataset, originally introduced by Zhang et al. (2015).The passage identifies Yelp polarity as an additional dataset used in the experiments.
  • Unlearning efficacy: Figure 7 varies forget-set size and reports classification accuracy on train, forget, and test points for Bloom 1.1B across datasets.The figure focuses on how classification performance changes with the size of the forget set.
  • Unlearning efficacy: Figure 8 presents log-scaled complete AUC curves for competitive GA and ICUL hyperparameters on all datasets with Bloom 1.1B.The comparison uses the Bloom 1.1B model and includes both GA and ICUL.
  • Hyperparameter analysis: Figure 9 varies GA learning rate and reports train, forget, and test classification accuracy across datasets and model sizes.The figure excludes ±1 standard deviation for readability.
  • Hyperparameter analysis: Figure 10 varies ICUL context length and reports train, forget, and test classification accuracy across datasets and model sizes.The figure excludes ±1 standard deviation for readability.
Loading 2310.07579v4…