Source-linked AI summary
Privacy Adhering Machine Un-learning in NLP
Vinayshekhar Bannihatti Kumar, Rashmi Gangadharaiah, Dan Roth
TL;DR
Machine unlearning addresses the costly task of deleting user data from trained models, and NLP remains under-explored. This paper adapts SISA with SISA-FC and SISA-A for NLP tasks; SISA-A is reported to reduce training time while retaining BERT-base-level performance. The study also reports a few-shot performance drawback and limits its approach to encoder-based models.
Problem
Complete removal of user data from large models is costly, and machine unlearning remains under-explored for NLP tasks.
Source: §1 · p. 1Method
The paper adapts SISA for NLP with SISA-FC, which stores task-specific layers, and SISA-A, which stores adapter weights.
Source: §1 · pp. 1–2Results
100x lower training time is reported for SISA-A, which achieves the same level of performance as BERT-base.
Source: §5.2 · p. 4Takeaways & Limitations
The paper demonstrates SISA unlearning on encoder-based NLP models, including GLUE tasks such as QQP, SST, and MNLI.
Source: §1 · p. 2, §6 · p. 9In few-shot settings, SISA-A performs below its full-data setting and BERT-base; the authors leave addressing this shortcoming to future work.
Source: §5.6 · p. 6
Abstract
from arXiv · showhide
Regulations introduced by General Data Protection Regulation (GDPR) in the EU or California Consumer Privacy Act (CCPA) in the US have included provisions on the \textit{right to be forgotten} that mandates industry applications to remove data related to an individual from their systems. In several real world industry applications that use Machine Learning to build models on user data, such mandates require significant effort both in terms of data cleansing as well as model retraining while ensuring the models do not deteriorate in prediction quality due to removal of data. As a result, continuous removal of data and model retraining steps do not scale if these applications receive such requests at a very high frequency. Recently, a few researchers proposed the idea of \textit{Machine Unlearning} to tackle this challenge. Despite the significant importance of this task, the area of Machine Unlearning is under-explored in Natural Language Processing (NLP) tasks. In this paper, we explore the Unlearning framework on various GLUE tasks \cite{Wang:18}, such as, QQP, SST and MNLI. We propose computationally efficient approaches (SISA-FC and SISA-A) to perform \textit{guaranteed} Unlearning that provides significant reduction in terms of both memory (90-95\%), time (100x) and space consumption (99\%) in comparison to the baselines while keeping model performance constant.
1 Introduction
User-generated data may contain sensitive information, and right-to-be-forgotten requests make complete removal from large ML models costly and time-consuming. The paper adapts SISA for NLP through SISA-FC and SISA-A, targeting lower storage and retraining costs while preserving model quality.
1 Introduction
- Right-to-be-forgotten regulations require user-data deletion, but removing data efficiently from large-scale models while guaranteeing complete removal is expensive and time-consuming.User-generated content can include sensitive information such as names, account IDs, and email addresses.Source: §1 · p. 1
- Removing training points can substantially degrade model performance, a problem the paper calls catastrophic unlearning.The passage distinguishes unlearning, which targets removal, from differential privacy, which limits what can be inferred about training data.Source: §1 · p. 1
- SISA-FC stores only task-specific layers, while SISA-A stores adapter weights; both avoid storing entire model checkpoints.The approaches also partition data to reduce the number of slices affected by removal requests and retraining time.Source: §1 · p. 2
- The paper explores machine unlearning on GLUE tasks including QQP, SST, and MNLI.It presents this as an initial exploration of unlearning across NLP tasks.Source: §1 · p. 2
2 Related Work
Prior work describes machine unlearning as complete removal of training examples, while retraining from scratch after deletion is computationally expensive at high request rates. The paper situates its NLP work in a field whose recent approaches largely focus on computer vision and builds on SISA’s sharding and aggregation framework.
2 Related Work
- Retraining after each deletion is computationally expensive and not scalable when removal requests arrive frequently.Unlearning targets zero contribution from removed training examples, unlike privacy-preserving methods that bound their contribution.Source: §2 · p. 2
- Recent unlearning approaches largely target computer vision, while textual data can contain sensitive or personally identifiable information.The passage argues for exploring unlearning strategies on textual data.Source: §2 · p. 2
- SISA trains models on disjoint shards, retrains only the affected model after a deletion request, and aggregates predictions at inference; this paper extends the framework to NLP.The paper also proposes data partitioning intended to reduce affected shards or slices and retraining time.Source: §2 · p. 2
3 Datasets
The evaluation samples three GLUE task categories—SST-2, QQP, and MNLI—and uses accuracy to measure performance. The datasets provide sentiment, duplicate-question, and inference-labeling tasks, respectively.
3 Datasets
- The evaluation uses SST-2, QQP, and MNLI, representing single-sentence, similarity/paraphrase, and inference tasks.For each dataset, the study takes the first 60,000 samples and uses 20% as the test split.Source: §3 · p. 2
- Across SST-2, QQP, and MNLI, the study selected the first 60,000 samples from each dataset, used 20% as test data, and measured performance with accuracy to compare retraining times across tasks.Source: §3 · p. 2
- SST-2 pairs movie-review sentences with sentiment labels, while QQP pairs questions with duplicate labels.MNLI pairs a premise with a hypothesis labeled as entailment, contradiction, or neutral.Source: §3 · p. 3
4 Models
The models section presents SISA’s shard-and-slice retraining scheme and parameter-efficient NLP variants that reduce checkpoint storage and training costs. It describes the comparison baselines and implementation setup across the experiments.
4 Models
- The section frames SISA as a way to unlearn a datapoint without retraining a large model from scratch, an expensive approach for large models.The cited prior SISA work evaluated computer-vision datasets, while the paper applies the algorithm to NLP.Source: §4 · p. 3
4.1 SISA
- SISA trains one model per shard, checkpoints after each slice, and resumes from the checkpoint preceding the deleted datapoint’s slice.The dataset is divided into shards and slices, and predictions from the shard models are aggregated at inference.Source: §4.1 · p. 3
4.2 SISA modified with fully connected Layer (SISA-FC)
- SISA-FC fine-tunes only added fully connected layers on a pretrained base model, reducing backpropagation to those layers and storing only their weights.The change addresses the time and memory costs of storing and retraining full-model checkpoints.Source: §4.2 · p. 3
4.3 SISA modified with Adapters (SISA-A)
- SISA-A adds adapters in transformer encoder blocks to retain the benefits of linear-layer tuning while optimizing performance.The passage says adapters increase memory footprint but account for about 1 −5% of model parameters, providing 95− 99% memory benefits.Source: §4.3 · p. 3
4.4 Baselines:
- The baselines compare SISA-FC and SISA-A with fine-tuned BERT on the same samples and with a majority classifier.The stated comparison checks performance impact and improvement over the majority-classifier baseline.Source: §4.4 · p. 3
4.5 Implementation details:
- For the SISA-FC model, experiments used a batch size of 16, maximum sequence length of 256, Adam at learning rate 5e −3, and 10 training epochs.SISA-A used the same hyperparameters, added adapters using Adapterhub2, and updated only those parameters; experiments kept the number of shards fixed at 5 while varying slice sizes.Source: §4.5 · pp. 3–4
- The SISA-A architecture uses S slices and R shards, builds one model per shard, and aggregates labels by majority vote.The implementation fixes the number of shards at five while varying slice sizes; it uses adapters and updates only adapter parameters.Source: Figure 1 · p. 4, §4.5 · p. 4
5 Experiments and Results
Experiments assess accuracy, retraining time, and disk memory; results report a training-time gain for SISA-A against the deletion baseline and distinguish SISA-A’s accuracy advantage from SISA-FC’s lower-memory profile.
5.1 Evaluation Metrics:
- Experiments use shard size 5 and assess accuracy, retraining time, and on-disk memory, sampling 16 requests unless otherwise stated.Accuracy is the percentage of correctly predicted test samples; retraining time includes deleting requests and retraining, and memory measures the final model on disk.Source: §5.1 · p. 4, §5 · p. 4
5.2 Analysis of SISA-A
- With SISA-A, accuracy is nearly unchanged above slice size two, while more slices reduce retraining time relative to two slices across all three datasets.Retraining time does not increase linearly with additional unlearning requests, and is reported as constant across the three datasets.Source: §5.2 · p. 4, Figure 2 · p. 5
- 1 −6% worse accuracy and 100x training-time improvement: SISA-A is reported against the deletion baseline.The evaluation considers accuracy, retraining time, and memory occupied on disk.Source: §5.1 · p. 4, Table 1 · p. 5
5.3 Analysis of SISA-FC
- SISA-FC performance is 20 −30% lower than SISA-A across tasks, but it uses less memory and retrains the final layer faster.The paper attributes the lower performance to using fewer fine-tuned weights and notes SISA-FC may suit low-memory settings.Source: §5.3 · p. 6
5.4 Comparison Between SISA-FC and SISA-A
- SISA-A significantly outperforms SISA-FC in accuracy, with an absolute gain of 20-30% depending on the task.The reported gain is observed across different slice sizes.Source: §5.4 · p. 6
5.5 Profiling requests based on probability of occurrence
- 350% and 360%: inverse-Pareto training time is reported lower than uniform and Pareto sampling, respectively, while retraining stays flat across requests affecting the last few slices.The experiment simulates customers with different opt-out probabilities using Pareto sampling and its mirror image.Source: §5.5 · p. 6
5.6 How does SISA-A perform on a portion of the data?
- In the few-shot setting, SISA-A performs below its full-data setting and BERT-base; the authors hypothesize repeated slice exposure causes overfitting.The paper leaves further work to alleviate this shortcoming for future work.Source: §5.6 · p. 6
5.7 Memory profiling
- As slice count increases, memory grows linearly with vanilla SISA but far less drastically with SISA-A and SISA-FC.Source: §5.7 · p. 6
6 Conclusion and Future Work
The conclusion frames opting out of data already used to train models as an underexplored problem and demonstrates SISA approaches on encoder-based NLP models. It identifies the aggregation layer as a barrier to extending the approach to decoder-based models.
6 Conclusion and Future Work
- The figures cover SISA-FC performance and retraining time, plus SISA-A accuracy and retraining time under inverse-Pareto and Pareto distributions.They also include SISA-A accuracy on partial datasets.Source: Figure 4 · p. 7, Figure 5 · p. 7, Figure 6 · p. 7
- Other figures cover SISA-A accuracy and retraining time on partial datasets, Pareto-affected shards, and memory occupied by three approaches.Source: Figure 10 · p. 8, Figure 11 · p. 8, Figure 12 · p. 9
- The work addresses limited research on opting out of data already used to train models, within U.S. and EU “notice and choice” privacy frameworks.The passage says users can opt out of data collection when they deem it contrary to their privacy needs.Source: §6 · pp. 6, 9
- SISA-A and SISA-FC demonstrate SISA on encoder-based NLP models, but the aggregation-layer requirement limits extension to decoder-based models.The paper identifies this as a limitation of its work.Source: §6 · p. 9
A Example Appendix
The supplied passage labels this material as an appendix.
A Example Appendix
- The passage identifies this material as an appendix.Source: A Example Appendix · p. 10