Source-linked AI summary
Unlearn What You Want to Forget: Efficient Unlearning for LLMs
Jiaao Chen, Diyi Yang
TL;DR
LLMs may memorize sensitive user data, creating a need to remove requested information without retraining the whole model or degrading predictive quality. EUL learns lightweight unlearning layers with a selective teacher-student objective and fuses layers across requests; experiments on classification and generation tasks demonstrate effectiveness against state-of-the-art baselines.
Problem
LLMs can memorize sensitive information, creating a need for efficient deletion of user-requested data while preserving predictive quality.
Method
EUL learns lightweight unlearning layers through a selective teacher-student objective and fuses layers learned for different data sets into one unified layer.
Results
Experiments on classification and generation tasks across datasets, model sizes, and forget-set sizes demonstrate EUL’s effectiveness compared with state-of-the-art baselines.
Takeaways & Limitations
EUL efficiently and dynamically handles sequences of deletion requests while leaving the original LLM unchanged during unlearning-layer training.
Takeaways & Limitations
Experiments mainly use T5-base/3b fine-tuned models and simulated settings, while more comprehensive evaluation and larger or closed-source backbones remain future work.
Abstract
from arXiv · showhide
Large language models (LLMs) have achieved significant progress from pre-training on and memorizing a wide range of textual data, however, this process might suffer from privacy issues and violations of data protection regulations. As a result, the ability to easily remove data related to individual users from such models while not deteriorating their predictive quality after the removal becomes increasingly important. To address these issues, in this work, we propose an efficient unlearning framework that could efficiently update LLMs without having to retrain the whole model after data removals, by introducing lightweight unlearning layers learned with a selective teacher-student objective into the transformers. In addition, we introduce a fusion mechanism to effectively combine different unlearning layers that learns to forget different sets of data to handle a sequence of forgetting operations. Experiments on classification and generation tasks demonstrate the effectiveness of our proposed methods compared to the state-of-the-art baselines.
1 Introduction
LLMs can memorize sensitive user information, creating privacy and regulatory pressures for efficient unlearning. EUL addresses this with lightweight selective teacher-student unlearning layers and fusion across deletion requests, evaluated on classification and generation tasks.
- Motivation: LLMs memorize broad textual data that may include sensitive information such as names, phone numbers, email addresses, and clinical notes.Prior work also showed that LLMs can generate private information.
- Motivation: GDPR and CCPA establish users’ right to request deletion of user-generated content, motivating efficient and effective LLM unlearning.
- EUL framework: EUL learns lightweight unlearning layers through a selective teacher-student formulation without tuning the large language model.The layers are plugged into transformer layers while the original LLM remains unchanged.
- EUL framework: EUL fuses layers learned for different forgotten-data sets into one unified layer, enabling a sequence of deletion operations.The fusion mechanism combines unlearning-layer weights by minimizing a regression objective.
- Evaluation: EUL is evaluated on classification and generation tasks with backbone models of different scales and under different settings.Experiments use IMDB and SAMSum and compare EUL with state-of-the-art unlearning and model-editing baselines.
2 Related Work
Related work frames machine unlearning as removing training-example contributions after user deletion requests, while model editing generally changes outputs for specified facts rather than forgetting required data.
- LLM development: LLM research has improved performance through scaling, longer training, instruction tuning, and human feedback.
- Machine unlearning: Machine unlearning includes exact retraining after data removal and approximate weight modification toward the retrained model.
- Model editing: Model-editing methods usually edit outputs for linguistic structures or world facts instead of forgetting requested data.
3 Efficient Unlearning for LLMs
EUL learns lightweight unlearning layers that preserve the original model’s behavior on retained data while disrupting predictions and extractability for forgotten data. A fusion mechanism then combines layers for sequential deletion requests without extra training or storing the forgotten data itself.
- Learning to Forget via Unlearning Layers: EUL models the updated LLM with a small adapter and updates only that adapter instead of retraining all parameters.The adapter has substantially fewer parameters than the original model.
- Learning to Forget via Unlearning Layers: The selective teacher-student objective makes the updated model follow the original model on retained data but diverge from it on forgotten data.Training minimizes output divergence on retained examples and maximizes it on forgotten examples.
- Learning to Forget via Unlearning Layers: Task loss on retained data helps maintain task performance while the original language-model objective is negated to remove knowledge from pretrained parameters.For T5 experiments, masked language modeling is modified with an instruction to predict the masked word.
- Learning to Forget via Unlearning Layers: The final objective balances forgetting, retention, task performance, and language-model forgetting through hyper-parameters and alternating updates on retained and forgotten data.The procedure alternates an epoch on retained data with an epoch on forgotten data for more stable optimization.
- Fusing Unlearning Layers: The fusion mechanism merges unlearning layers trained for different forgotten sets into one layer for sequential deletion requests.Its weights are obtained through a linear regression problem using precomputed hidden-representation inner products.
- Fusing Unlearning Layers: Fusion requires no extra training and stores an inner product matrix of representations rather than the forgotten data itself.This supports efficiency and privacy during repeated unlearning operations.
4 Experiments
Experiments evaluate EUL on privacy-related and mislabeled data across classification and generation tasks, comparing it with retraining, fine-tuning, SISA, Reverse-Gradient, and MEND. EUL generally forgets requested data while preserving task and retained-set performance with faster updates, and its components contribute distinct benefits.
- Datasets: IMDB sentiment classification and SAMSum conversation summarization evaluate unlearning privacy-related data associated with names, speakers, movies, actors, or other entities.The experiments use T5-base and T5-3b models, with names in development and test data replaced by special tokens.
- Evaluation Metrics: Evaluation measures test-set performance, retained-set performance, forgot-set performance, MLM loss, and model-updating time.Lower forgot-set performance indicates better forgetting, while higher MLM loss indicates that forgotten information is harder to extract.
- Privacy-related Data: EUL achieves 65.6% accuracy on the forgot set, 93.0% test accuracy, 100% retained accuracy, and one-sixth of retraining time when forgetting 0.5% of IMDB data.These are reported for T5-base and represent the paper’s example of EUL’s combined forgetting, retention, task-performance, and speed results.
- Privacy-related Data: On SAMSum, EUL consistently achieves the best overall performance by forgetting requested data, remembering retained data, preserving test ROUGE, and requiring less training time.The result supports applying the objectives beyond classification to generation tasks.
- Mislabeled Data: For 10% mislabeled IMDB data, EUL most effectively removes the mislabeled data’s negative impact and achieves the best test accuracy.The original model’s test accuracy is affected by the mislabeled data.
- Ablation Study: Removing individual EUL objectives causes performance drops: LKL primarily drives forgetting, LTASK supports test accuracy, and LLM supports higher MLM loss.The ablation evaluates EUL after removing 0.5% privacy-related IMDB data.
5 Conclusion
EUL efficiently unlearns user-requested data through lightweight unlearning layers and merges layers to handle sequential forgetting operations. Experiments across varied settings support its effectiveness against state-of-the-art baselines.
- EUL learns unlearning layers through a selective teacher-student objective to remove user-requested data efficiently.
- A fusion mechanism merges different unlearning layers into one unified layer for sequential data removal.
- Experiments across different datasets, model sizes, and forget-set sizes demonstrate EUL’s effectiveness compared with state-of-the-art baselines.
6 Limitations
The experiments mainly cover fine-tuned T5-base and T5-3B models, while broader model families, evaluation methods, and real-world applications remain future directions.
- Experiments mainly use T5-base and T5-3B models with fine-tuned tasks.
- The evaluation follows test-set, retained-set, and forgot-set performance together with MLM loss, leaving more comprehensive unlearning evaluation for future work.
- All experiments use simulated settings rather than actual use cases or new real-world benchmarks.