PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey on Federated Unlearning: Challenges, Methods, and Future Directions

Ziyao Liu, Yu Jiang, Jiyuan Shen, Minyi Peng, Kwok-Yan Lam, Xingliang Yuan, Xiaoning Liu

arXiv:2310.20448v4cs.CR

TL;DR

Federated unlearning addresses privacy-driven data-erasure challenges and the need to support users’ right to be forgotten. This survey proposes a unified workflow and taxonomy organized around who unlearns and what is unlearned, while highlighting verification gaps and client dynamics.

  • Problem

    Privacy concerns and the right to be forgotten create new targets and challenges for federated unlearning.

  • Method

    The survey presents a unified federated-unlearning workflow and taxonomy based on who initiates unlearning and what needs to be unlearned.

  • Results

    The survey formalizes federated-unlearning targets and challenges while reviewing verification and client-dynamics considerations.

  • Takeaways & Limitations

    Client-side verification can help participants assess whether unlearning succeeded, while client dynamics remain relevant to process integrity and effectiveness.

  • Takeaways & Limitations

    There are no standard or widely adopted methods for proof of unlearning.

Abstract

from arXiv · show

In recent years, the notion of ``the right to be forgotten" (RTBF) has become a crucial aspect of data privacy for digital trust and AI safety, requiring the provision of mechanisms that support the removal of personal data of individuals upon their requests. Consequently, machine unlearning (MU) has gained considerable attention which allows an ML model to selectively eliminate identifiable information. Evolving from MU, federated unlearning (FU) has emerged to confront the challenge of data erasure within federated learning (FL) settings, which empowers the FL model to unlearn an FL client or identifiable information pertaining to the client. Nevertheless, the distinctive attributes of federated learning introduce specific challenges for FU techniques. These challenges necessitate a tailored design when developing FU algorithms. While various concepts and numerous federated unlearning schemes exist in this field, the unified workflow and tailored design of FU are not yet well understood. Therefore, this comprehensive survey delves into the techniques and methodologies in FU providing an overview of fundamental concepts and principles, evaluating existing federated unlearning algorithms, and reviewing optimizations tailored to federated learning. Additionally, it discusses practical applications and assesses their limitations. Finally, it outlines promising directions for future research.

1 INTRODUCTION

Federated unlearning extends machine unlearning to federated learning, where data erasure involves distributed clients and servers and may target entire clients or partial client data. This survey organizes FU concepts, methods, workflows, optimizations, limitations, challenges, and future directions.

  • Right-to-be-forgotten regulations motivate removing personal data and its influence from training datasets and trained models.
  • Machine unlearning removes specific data impacts while preserving overall model performance, offering a more cost-effective alternative to naive retraining.
  • Federated unlearning removes an entire client or identifiable information from partial client data while maintaining the privacy guarantees of decentralized learning.
  • Federated learning introduces new unlearning targets and challenges, leaving FU design, cross-setting methodology, unified workflows, and security and privacy issues insufficiently understood.
  • The survey presents a unified workflow and taxonomy, summarizes methods by who-unlearn and unlearn-what, reviews FL-tailored optimizations and limitations, and identifies future research directions.

2 TARGETS AND CHALLENGES OF FEDERATED UNLEARNING

Federated unlearning targets removal of entire clients or partial client data while preserving model quality, reducing cost, protecting privacy, and enabling verification. Compared with machine unlearning, FL creates challenges from distributed data, knowledge permeation, multiple possible unlearners, and differing verification requirements.

  • Targets: Model consistency requires the unlearned model to perform similarly to a retrained model while preserving accuracy and reliability.This target indicates whether specific data can be removed without diminishing the model’s overall quality.
  • Targets: Unlearning efficiency requires costs to be significantly lower than retraining, including runtime, participating clients, and communication overhead.Retraining in FL starts training from scratch, making these cost dimensions central to efficiency.
  • Targets: Privacy preservation requires federated unlearning to maintain FL’s protection of clients’ local data.FU must operate without undermining the privacy guarantees provided by sharing models or gradients rather than local data.
  • Targets: Certified removal requires clients or the server to verify that an entire client or partial data was removed according to the request.Verification must be robust and reliable for both client-initiated and server-initiated requests.
  • Challenges: Knowledge permeation spreads a target client’s information through FL training, complicating model consistency relative to centralized machine unlearning.The server aggregates client updates and redistributes the global model, allowing targeted knowledge to reach other clients.
  • Challenges: Data isolation limits direct access to unlearned data, hindering adaptation of machine-unlearning methods and efficient federated-unlearning design.Only gradients or global models are publicly shared, while local datasets remain under individual clients’ control.
  • Challenges: FU must address who unlearns, what is unlearned, and who verifies removal because clients and servers may initiate or execute different requests.Targets may cover an entire client or partial data, and the responsible entity’s information access affects algorithm choice and verification.
  • Challenges: Removing an entire client can make local retraining, fine-tuning, and multi-task unlearning inapplicable because that client’s data becomes inaccessible.Different unlearning targets therefore affect model consistency, efficiency, and certified removal.

3 PRELIMINARIES AND BACKGROUNDS

The preliminaries define machine and federated unlearning concepts, review common unlearning principles, and summarize metrics and attacks used to verify removal. They contrast exact retraining with more efficient approximate approaches that face computational or forgetting-related limitations.

  • Foundations: Machine unlearning separates forgotten data D_u from remaining data D_r and seeks a model matching one trained only on D_r.The target distribution is to make M(D) identical to M(D_r).
  • Unlearning principles: Retraining starts from scratch on D_r and removes information about D_u, but it is time-consuming and resource-intensive.Its cost arises partly because the original model trained on D is discarded.
  • Unlearning principles: Fine-tuning optimizes the existing model on D_r to reduce D_u’s impact, but multiple iterations increase computational and communication costs.This approach retains the original model while repeatedly adapting it using remaining data.
  • Unlearning principles: Gradient ascent reverses gradient descent by maximizing loss on targeted data, but it can cause catastrophic forgetting.This limitation motivates approaches that balance erasure and retention effects.
  • Unlearning principles: Model scrubbing transforms M(D) so it approximates a retrained model using D_r, often through a quadratic loss approximation and Newton step.Computing the Hessian can be infeasible for high-dimensional models, so some methods approximate it.
  • Unlearning principles: Synthetic-data methods replace or relabel targeted data to disentangle its influence while retaining the model’s remaining knowledge.One example generates synthetic labels for D_u and combines them with D_u during training.
  • Verification: Unlearning verification evaluates attack performance, model performance, model discrepancy, and execution efficiency.Metrics include attack success rate, accuracy, loss, statistical errors, distance measures, runtime, rounds, speed-up, and memory consumption.

3.2 Federated Learning

Federated learning distributes local training across clients and uses a central server to aggregate and redistribute models. Federated unlearning removes whole clients or partial data while comparing the resulting model with retraining and limiting added privacy leakage.

  • Federated learning workflow: FL separates n clients with local datasets from a central server, sharing trained models or gradients rather than local data.This design supports privacy but creates constraints for unlearning and aggregation security.
  • Federated learning workflow: A typical FL system repeatedly trains local client models, uploads them, aggregates them at the server, and distributes the updated global model.The cycle continues until training stops.
  • Federated learning security: Privacy-preserving aggregation protects the aggregation process, while malicious-client detection addresses poisoning updates that can degrade performance or inject backdoors.These mechanisms protect the standard FL workflow surrounding federated unlearning.
  • Federated unlearning: Federated unlearning maps a global FL model M to an unlearned model M̄ using unlearned clients U_u and remaining clients U_r.For a target client, D̄_j = D_j denotes full-client removal, while D̄_j ⊂ D_j denotes partial-data removal.
  • Federated unlearning targets: Model consistency compares predictions from the unlearned model with those from a retrained global model.Full consistency holds when the predictions are equal, Ȳ = Ŷ.
  • Federated unlearning targets: Unlearning efficiency requires FU cost to be lower than obtaining the retrained model, while privacy preservation limits additional leakage beyond FL.Certified removal additionally confirms that the process followed the initiating participant’s request.

3.4 Attacks to ML Models

Attacks on ML models provide ways to test whether target data or backdoors remain after unlearning. Membership inference and backdoor attacks use attack success rate as an indicator, with lower attack performance suggesting reduced retained influence.

  • Membership inference attacks: Membership inference attacks determine whether a record was used to train a target model by exploiting differences between member and non-member outputs.They can operate without knowing the target architecture or training-data distribution, using shadow models to train an attack classifier.
  • Membership inference attacks: Applying MIA to unlearned data evaluates removal: a lower attack success rate suggests that the data’s influence on the global model was reduced.The ASR serves as an indicator of how effectively the data has been unlearned.
  • Backdoor attacks: Backdoor attacks implant visible or covert triggers that cause malicious behavior when triggered while leaving standard-input behavior typical.Their dormancy can allow them to remain undetected until exploitation.
  • Backdoor attacks: For unlearning evaluation, injecting a backdoor into target data tests whether unlearning disrupts the trigger-to-class relationship.A lower ASR indicates that the backdoor has been successfully eliminated.

4 FEDERATED UNLEARNING METHODS

The survey introduces a unified federated unlearning workflow covering learning, unlearning, and verification, then uses it as the basis for organizing existing FU techniques. Requests may lead to passive or active unlearning, with responsibilities distributed among clients and the server.

  • Unlearning requests can originate from the target client or server, and unlearning or verification may involve servers, target clients, remaining clients, or combinations.
  • The unified workflow defines a timeline spanning learning, unlearning, and verification.
  • The workflow provides the basis for a taxonomy of FU techniques, with existing works summarized separately.
  • Passive unlearning lets the target client exit immediately, whereas active unlearning lets it remain and participate.
  • Some target clients may initiate requests and transmit information, while others remain only for verification.

4.1 Passive unlearning

Passive unlearning removes a target client without keeping it in the FU process, using server-side or remaining-client procedures and often stored historical information. These methods trade participation and adaptability against storage, computation, and model-specific reconstruction costs.

  • Passive unlearning: Passive unlearning removes the entire target client rather than partial data, and the target client does not remain to participate.
  • Server-standalone unlearning: Server-standalone methods use stored gradients, global models, contributions, or intermediate information to eliminate the target client’s influence.
  • Client-aided unlearning: Client-aided methods retain the standard FL workflow and iteratively refine the global model by aggregating improved information from remaining clients.
  • Passive unlearning: Local retraining, local fine-tuning, and multi-task unlearning are unavailable when passive unlearning removes the target client’s access to data.
  • Limitations: Server-standalone unlearning can require substantial server memory and may perform worse or adapt less effectively than client-aided unlearning in complex or Non-IID settings.
  • Server-standalone unlearning: RevFRF removes affected random-forest nodes and reconstructs branches from stored intermediate information instead of retraining remaining clients.
  • Server-standalone unlearning: In the worst case, RevFRF must reconstruct the entire random decision tree when the revoked node is the root.
  • Client-aided unlearning: FedEraser calibrates historical gradients iteratively using stored gradients and client-derived calibration directions to obtain a model eliminating the target client’s influence.

4.2 Active unlearning

Active unlearning involves the target client in the unlearning process, with the option to stay or leave and with or without verification. Reviewed methods address partial-data or client-level removal through retraining, rollback, fine-tuning, model scrubbing, synthetic data, gradient ascent, and Bayesian updating.

  • Active unlearning lets the target client participate in removal and optionally stay or leave, with or without verification.
  • Partial-data unlearning: Retraining-based methods remove partial data by recomputing client updates and either restarting or rolling back to a suitable historical global-model state.ViFLa stores historical states and lets remaining clients continue training from a server-identified previous state; SCMA recalculates and re-aggregates local clustering vectors.
  • Partial-data unlearning: Fine-tuning and multi-task methods combine local forgetting with global learning, attention-based filtering, embedding optimization, or penalties linking local and global models.
  • Partial-data unlearning: Model-scrubbing methods use Hessian or Fisher approximations, dummy gradients, posterior matching, auxiliary models, feature-sensitivity reduction, or randomized smoothing.
  • Partial-data unlearning: Synthetic-data and gradient-ascent approaches train on perturbed or synthetically labeled forgotten data, while FedAF adds a task to retain remaining-data memory.FedAF obtains synthetic labels from ensemble predictions of random teacher models created by a trusted third party.
  • Bayesian federated unlearning: Bayesian federated unlearning approximates posteriors with variational inference or uses parameter self-sharing to balance forgetting requested data with retained knowledge.

4.3 Verification

Verification gives the requester a way to assess whether unlearning succeeded, but it remains limited and inconsistently assigned across federated unlearning systems. Existing methods use markers, accuracy-related metrics, attacks, model comparisons, convergence, or certified budgets.

  • The right to verify allows a target client or server to actively check the unlearning effect after completion.
  • Verification mechanisms: EMA ensembles correctness, confidence, and negative entropy on an audited dataset against a predefined threshold for judging whether target data was unlearned.
  • Verification mechanisms: Marker-based verification monitors unlearned-model performance on Forgettable Memory or Erroneous Memory samples to assess removal.VERIFI uses non-invasive markers rather than injecting backdoors or watermarks into the original data.
  • Verification mechanisms: Other works stop or assess unlearning using global-model differences, validation accuracy and deviation, convergence, knowledge-based predictions, or certified removal budgets.
  • Evaluation practice: Accuracy-based metrics over unlearned data are the most commonly used verification metrics, alongside time, backdoor attacks, membership inference, and retrained-model differences.
  • Limitations and directions: Few studies address who verifies, most assume server verification, and no standard or widely adopted proof-of-unlearning method exists.The survey identifies client verification as especially important for unlearning services.

4.4 Lessons Learned

The survey’s lessons emphasize matching unlearning principles to data-access conditions and aligning the requester with the party that performs unlearning or verification. It also identifies unresolved request-structure and proof-of-unlearning problems.

  • Who-unlearn and who-verify: The alignment between who requests unlearning and who performs verification is not optimal in existing literature.
  • Who-unlearn and who-verify: Systems should allow requesters to conduct unlearning or verification themselves so results are credible to those participants.
  • Selection of unlearning principles: Unlearning principles differ in their reliance on training-data access, so methods should be selected according to the federated scenario and access level.
  • Structure of unlearning requests: The structure of unlearning requests remains difficult to determine because direct access by an unlearner can conflict with federated learning’s privacy foundation.
  • Structure of unlearning requests: Unclear request structures and their integration into federated systems may hinder adoption of unlearning services in federated settings.
  • Proof of unlearning: Federated unlearning lacks standard or widely adopted metrics for proving that removal occurred.

5 FL-TAILORED OPTIMIZATIONS, LIMITATIONS, AND APPLICATIONS

Federated unlearning inherits resource, heterogeneity, privacy, and security constraints from federated learning while adding further storage, communication, and computation demands. The survey reviews FL-tailored optimizations and applications, but notes unresolved security and defense gaps.

  • Overview: FU faces constrained resources, participant heterogeneity, and security and privacy threats, with additional risks introduced by its extra processes.
  • Resource constraints: Resource-constrained clients and dropout make FU resource requirements important, while historical gradients, models, residuals, states, or intermediate results increase server memory use.
  • Resource constraints: Storage can be reduced by selectively retaining important historical updates, storing them at intervals, or compressing them with coding-based techniques.
  • Communication constraints: FU requires extra client-to-server information and communication, which can be mitigated through quantization, fewer rounds, rollback, clustering, or partial participation.
  • Computation constraints: Additional computation includes dummy gradients, reinforcement learning, seed-model generation, and Hessian calculations, while approximations, outsourcing, precomputation, and dataset distillation reduce overhead.
  • Participant heterogeneity: Heterogeneity-aware designs use vertical federated learning, weighted aggregation, knowledge distillation, clustering, asynchronous aggregation, and incentives.
  • Security, privacy, and applications: Gradient leakage can recover images with pixel-wise accuracy and text with token-wise matching; privacy-preserving aggregation and FU can support poisoning recovery, backdoor removal, and data-misuse response.
  • Security, privacy, and applications: Extended investigation of crafted unlearning requests and defense strategies remains lacking, while existing work independently optimizes different efficiency dimensions.

6 DISCUSSIONS AND PROMISING DIRECTIONS

The survey identifies emerging challenges and open problems in federated unlearning and proposes directions for expanding its privacy, security, adaptability, application scope, fairness, and explainability.

  • Open challenges: Federated unlearning faces emerging challenges and open problems that require additional properties and broader capabilities.The survey extends its discussion to potential research directions for enhancing FU capabilities.
  • Privacy-preserving FU: Client models and global models can enable pixel-wise reconstruction of image data or token-wise matching for text data.Privacy-preserving techniques such as HE, MPC, and DP can protect aggregation, but may significantly affect FU performance when target-client gradients are unavailable.
  • Verification: FU requires verification mechanisms that let target clients determine whether their data and its impact on the FL model were removed.The survey highlights client-side verification and non-invasive mechanisms as ways to support privacy and avoid altering model performance through invasive marking.
  • Verification: Cryptographic and hardware-based approaches such as ZKPs and TEEs are promising for verifiable FU when existing verification methods are impractical or trust is limited.These approaches are described as offering stronger cryptographic security guarantees in sensitive or distrustful settings.
  • Security: FU systems face privacy leakage and security threats from model differences, crafted unlearning requests, utility degradation, and backdoor injection.The survey calls for defense mechanisms that preserve the integrity and security of FU schemes.
  • Client dynamics: Random client and data selection, dropped or newly joined clients, and the need to recall past clients make FU more nondeterministic and motivate client-dynamics-aware algorithms.This challenge is especially difficult for privacy-preserving FU integrating privacy-enhancing technologies.
  • Applications, fairness, and explainability: FU remains insufficiently adapted to applications such as LLMs, recommendation systems, GNNs, health, IoT, blockchain, metaverse, and digital twins.The survey also identifies fairness and explainability challenges, including adverse effects on clients that share overlapping data.

7 CONCLUSIONS

The survey formalizes federated unlearning through a unified workflow and taxonomy, examines tailored optimizations and their limitations, and synthesizes current challenges with future research directions.

  • Contributions: The survey formalizes federated unlearning targets and challenges and introduces a unified federated unlearning workflow.This provides the paper’s organizing framework for the field.
  • Contributions: It derives a taxonomy of federated unlearning methods based on who initiates unlearning, what is unlearned, and how results are verified.These factors structure the classification of existing methods in federated settings.
  • Contributions: The survey reviews federated-learning-specific optimizations, critically examines their limitations, and outlines current challenges and promising future research directions.It presents the survey as a resource for researchers and practitioners.
Loading 2310.20448v4…