PaperlayerView exact revision on arXiv

Source-linked AI summary

VeriFi: Towards Verifiable Federated Unlearning

Xiangshan Gao, Xingjun Ma, Jingyi Wang, Youcheng Sun, Bo Li, Shouling Ji, Peng Cheng, Jiming Chen

arXiv:2205.12709v1cs.CRcs.LG

TL;DR

Federated unlearning alone does not let a participant verify that their data has been forgotten, a gap that matters to the right to be forgotten. VERIFI combines server-side unlearning with participant-led marking and checking, and the study identifies uS2U with vEM as its most promising combination. The framework and evaluation provide empirical understanding of verifiable federated unlearning.

  • Problem

    In FL, it remains difficult to verify reliably that a participant’s data was deleted after unlearning, and backdoor verification can pose security risks.

    Source: §I · p. 1, B. Federated Unlearning and Verification · p. 4

  • Method

    VERIFI combines server-side unlearning with a leaver’s local marking and checking, and supports systematic analysis of combinations of unlearning and verification methods.

    Source: §I · p. 2, A. Unlearning-Verification Mechanism · p. 4

  • Results

    The authors identify vEM with uS2U as the most promising approach for verifiable federated unlearning without tampering with FL, requiring white-box model access, or raising new security risks.

    Source: §VI · pp. 13–14

  • Takeaways & Limitations

    VERIFI supports systematic analysis of the verifiable amount of unlearning across different combinations of unlearning and verification methods.

    Source: §I · p. 2, §VI · p. 13

    Some verification methods may raise security concerns or become less effective when secure FL algorithms are used.

    Source: 1) Marking · p. 7

Abstract

from arXiv · show

Federated learning (FL) is a collaborative learning paradigm where participants jointly train a powerful model without sharing their private data. One desirable property for FL is the implementation of the right to be forgotten (RTBF), i.e., a leaving participant has the right to request to delete its private data from the global model. However, unlearning itself may not be enough to implement RTBF unless the unlearning effect can be independently verified, an important aspect that has been overlooked in the current literature. In this paper, we prompt the concept of verifiable federated unlearning, and propose VeriFi, a unified framework integrating federated unlearning and verification that allows systematic analysis of the unlearning and quantification of its effect, with different combinations of multiple unlearning and verification methods. In VeriFi, the leaving participant is granted the right to verify (RTV), that is, the participant notifies the server before leaving, then actively verifies the unlearning effect in the next few communication rounds. The unlearning is done at the server side immediately after receiving the leaving notification, while the verification is done locally by the leaving participant via two steps: marking (injecting carefully-designed markers to fingerprint the leaver) and checking (examining the change of the global model's performance on the markers). Based on VeriFi, we conduct the first systematic and large-scale study for verifiable federated unlearning, considering 7 unlearning methods and 5 verification methods. Particularly, we propose a more efficient and FL-friendly unlearning method, and two more effective and robust non-invasive-verification methods. We extensively evaluate VeriFi on 7 datasets and 4 types of deep learning models. Our analysis establishes important empirical understandings for more trustworthy federated unlearning.

I. INTRODUCTION

Federated learning supports collaboration without sharing private data, but participants may struggle to exercise the right to be forgotten when unlearning cannot be independently verified. VERIFI combines unlearning with participant-led verification and supports a systematic study of different method combinations.

I. INTRODUCTION

  • FL participants have the right to request deletion of their private data, but existing frameworks make exercising this right difficult and verification after unlearning remains understudied.Ordinary performance measures may barely change when only a few participants leave, while backdoor-based verification can introduce security threats.Source: §I · pp. 1–2
  • VERIFI integrates federated unlearning, marking and checking, and a mechanism that specifies when and by whom these steps occur.The framework grants a leaving participant the right to verify unlearning and quantify its effectiveness across method combinations.Source: §I · p. 2
  • VERIFI proposes uS2U, which scales down a leaver’s gradients or parameters, and two non-invasive memory-based verification methods, vEM and vFM.Checking measures performance change on marked data using loss, accuracy, influence function, or KL divergence.Source: §I · p. 2
  • The study examines 7 unlearning methods and 5 verification methods across CNNs and RNNs on 7 datasets spanning natural images, facial images, audio, and medical images.The authors analyze the necessity, potential, and limitations of different federated unlearning and verification methods.Source: §I · p. 2

II. PRELIMINARIES

The paper introduces federated learning, federated unlearning, and verification before detailing VERIFI’s timeline and procedure. Its mechanism separates early departures from later departures and gives a leaving participant a local way to check the server’s unlearning.

A. Federated Learning

  • In horizontal FedAvg, participants train locally and send accumulated updates for the server to aggregate into the global model; their private data stays on their devices.The paper focuses on horizontal FL with FedAvg.Source: A. Federated Learning · p. 3

B. Federated Unlearning and Verification

  • Federated unlearning explicitly forces a model to forget requested data, unlike natural forgetting; existing approaches include retraining, partition-based learning, noise, and gradient subtraction.Gradient-based methods may harm the original task or introduce privacy threats in FL.Source: B. Federated Unlearning and Verification · pp. 3–4
  • Prior verification approaches check performance on leaving data or use backdoored samples, while the paper also adapts watermarking and fingerprinting for federated unlearning verification.Backdoor techniques are not ideal for FL because they can create security risks.Source: B. Federated Unlearning and Verification · p. 4

A. Unlearning-Verification Mechanism

  • VERIFI divides training into a free stage, when departures do not activate unlearning, and an unlearning-enabled stage, when departure triggers verification and unlearning.The paper focuses on leaving participants and assumes a trusted server with an unlearning method in place.Source: A. Unlearning-Verification Mechanism · pp. 4–5
  • A leaver notifies the server, marks data for checking, requests server-side unlearning, and locally checks the markers over subsequent rounds before deciding whether privacy is assured.VERIFI focuses on one leaver per round; longer checking periods increase certainty but can allow more model downloads, so the leaving time should be agreed upon.Source: A. Unlearning-Verification Mechanism · p. 4

B. Unlearning

VeriFi studies seven federated unlearning methods, combining existing, adapted, and proposed approaches. It introduces uS2U, a one-step method intended to be more efficient and FL-friendly than existing approaches.

Proposed Scale-to-Unlearn (uS2U)

  • uS2U scales down the leaving participant’s updates and scales other participants’ updates to encourage the global model to forget the leaver’s contribution.It can use the global model at the unlearning-enabled round as an approximate stand-in for other participants’ local models and is compatible with aggregation rules such as FedAvg and Krum.Source: 1) Proposed Scale-to-Unlearn (uS2U) · p. 5

Existing or Adapted Unlearning Methods

Discussion

  • uDP adds noise to the leaving participant’s updates, with a smaller ε adding more noise, but noise may harm the original task’s convergence.The server adjusts ε and δ to blur memorization without degrading global-model performance too much.Source: 2) Existing or Adapted Unlearning… · p. 6
  • uRT and uRTB are described as effective but costly; the five multi-step methods require multiple rounds and stored gradients, which may raise privacy concerns.The methods ideally use as many unlearning rounds as the leaver’s past contribution rounds.Source: 2) Existing or Adapted Unlearning… · p. 6
  • uS2U and uDP are one-step methods that use only current-round gradient information and do not require storing local or global gradients.The paper characterizes uS2U as more FL-friendly than uDP because it has minimum or even positive impact on other participants’ updates after aggregation.Source: B. Unlearning · p. 5, 2) Existing or Adapted Unlearning… · p. 6

C. Verification

VeriFi’s verification is performed by the leaving participant through marking and checking, and the paper considers five marking methods. It proposes non-invasive methods based on unique model memories and adapts other marking approaches.

Verification

  • Verification chains marking, which creates markers, with checking, in which the leaving participant tracks changes in global-model performance on those markers.Checking can continue for a few communication rounds until the leaving time.Source: C. Verification · p. 6, 1) Marking · p. 6, 2) Checking · p. 8
  • VeriFi includes five marking methods; vFM and vEM are proposed non-invasive methods, while vBN adapts backdoor-based verification and can raise security risks.The paper also adapts watermarking and fingerprinting methods from deep-learning intellectual-property protection.Source: C. Verification · p. 6, 1) Marking · p. 7

Proposed Unique Memory Markers

  • The proposed vFM method selects the leaving participant’s examples with the highest local-training-loss variance across communication rounds as markers.These forgettable examples are described as hard and unique to the participant; effective unlearning should restore high loss variation on them.Source: 1) Marking · p. 6
  • The proposed vEM method uses erroneous samples as markers, relabels them to their mostly predicted class, and fine-tunes the local model on them.Effective unlearning should recover high losses on these markers.Source: 1) Marking · pp. 6–7

Verification

  • Marking methods differ in whether they tamper with the FL process or rely on training controllability, external data, or white-box access; vFM and vEM require none of those properties.The paper notes these properties may introduce security or privacy risks.Source: 1) Marking · p. 8

Checking

  • Checking can use accuracy, loss, influence function, or KL divergence; effective unlearning on markers is associated with low or zero KL divergence.The paper defines KL divergence against an ideal with-unlearning probability distribution and describes the uniform distribution as an ideal case.Source: 2) Checking · p. 8

IV. EXPERIMENTS

The experiments use seven datasets spanning image, speech, face, and medical-image tasks. The setup specifies participant data distribution and the main experimental parameters.

Experimental Setup

  • Experiments cover MNIST, CIFAR-10, SpeechCommand, VGGFace_mini, ImageNet_mini, ISIC, and COVID datasets.The set includes low- and high-resolution images, speech recognition, face and natural-object recognition, and medical-image diagnosis.Source: §IV · p. 8
  • The dataset/model and test-accuracy pairings are summarized in Table III.The supplied table caption identifies datasets, models, and test accuracies as its contents.Source: Table III · p. 8
  • Each participant receives an equal, non-overlapping share of each dataset, and the setup defines the rounds and training parameters used in experiments.The early-stage testing experiment uses two highlighted hyperparameters; the setup distinguishes its unlearning and leaving rounds from standard testing.Source: §IV · p. 8, Table IV · p. 8

A. Is Federated Unlearning Necessary?

At convergence, natural forgetting leaves the global model's performance on departing-participant data nearly unchanged, so active unlearning is necessary to remove information about that data.

A. Is Federated Unlearning Necessary?

B. Are Markers Necessary for Verification?

Checking metrics on leaving data alone do not reliably verify all unlearning methods, while marker-based verification distinguishes methods with varying effectiveness, robustness, and cost.

B. Are Markers Necessary for Verification?

  • 5 of 7 unlearning methods are not properly verified by the four leaving-data metrics without specialized markers.uDP and uRT are relatively easier to verify, but uRT is costly and uDP causes the most performance drop; the radar charts show metric differences across methods and datasets.Source: Figure 6 · p. 9, B. Are Markers Necessary for… · p. 9

C. Federated Unlearning Verification with Markers

  • vEM generally verifies better than vFM, vME, and vBN, while vBF ranks last.vEM distinguishes stronger methods uRT, uRTB, uDP, and uS2U from milder methods across datasets; vFM misses distinctions on MNIST, vME misses uDP, vBN fails on high-resolution datasets, and vBF distinguishes only uRT and uDP.Source: C. Federated Unlearning Verification… · p. 10
  • vEM maintains a stable metric difference under Krum and Median aggregation, unlike the sharp drops for vBN and vME.The robustness experiment is on CIFAR-10; Table VI describes absolute metric differences before and after unlearning under different aggregation rules.Source: C. Federated Unlearning Verification… · p. 10, Table VI · p. 10
  • uS2U has the least overall computational overhead and minor impact on the original task, while uRT takes the most time and uRTB the most space.uCGS is time- and space-consuming, and uDP has low time and storage cost but causes the most performance drop; Table VII reports costs on CIFAR-10.Source: C. Federated Unlearning Verification… · p. 10, Table VII · p. 11
  • All five marking methods cause tolerable original-task performance drops; vFM costs less time and space than vEM and vME, while vBN takes the most time.vBF also requires substantial time and space; the verification-cost measurements are reported for CIFAR-10.Source: C. Federated Unlearning Verification… · p. 10, Table VIII · p. 11
  • vBN, vME, and vBF work best with accuracy, while vEM and vFM work best with loss and loss variance, respectively.Fig. 7 compares normalized metric differences across five marker types, seven unlearning methods, and datasets.Source: Figure 7 · p. 11
  • Marker and leaving-data performance trends are strongly correlated before and after unlearning, except for vBN markers.The comparison uses uRT around the unlearning round; the authors conclude that unlearning markers can largely reflect how much the leaving data is unlearned.Source: C. Federated Unlearning Verification… · p. 11

D. Unlearning-Verification: The Combinations

Across the evaluated combinations, uS2U with vEM is the most promising strategy given its effectiveness, efficiency, task impact, and verifiability. In the adversarial setting, continued checking can detect some attempts to restore a leaver’s memory, but detection is not guaranteed.

D. Unlearning-Verification: The Combinations

  • uS2U with vEM is the most promising combination, given uS2U’s relatively high effectiveness, efficiency, limited FL impact, and verifiability.The general unlearning ranking is uRT > uRTB > uS2U > uDP > uCGS ≈ uGGS ≈ uIGS; Fig. 9 compares normalized effects for seven unlearning and five marking methods.Source: D. Unlearning-Verification: The… · pp. 11–12, Figure 9 · p. 12
  • The general verification ranking is vEM > vFM > vME > vBF > vBN; vME, vBF, and vBN are placed last because invasive marking may introduce security risks.The deeper blue cells in Fig. 9 indicate more effective combinations.Source: D. Unlearning-Verification: The… · p. 12, Figure 9 · p. 12

A. Adversarial Setting

  • The adversarial experiment assumes ideal uRT and uses vME, which checks unlearning through bits extracted from model parameters.The setting considers a malicious participant who captures and uploads the global model before or after the marking stage.Source: A. Adversarial Setting · p. 12
  • When an attacker restores a captured model from the marking stage, marker accuracy rises after unlearning and continued checking can expose the deception.If the attacker captures the model outside the marking stage, marker accuracy does not change and the leaver deems unlearning effective; VERIFI detects deception only with some probability.Source: A. Adversarial Setting · p. 12, Figure 10 · p. 12

B. More Unlearning and Verification Parameters

The parameter analyses show that marking time, marking strength, participant count, and data heterogeneity all affect verification. The conclusion identifies VERIFI as a test bed and vEM with uS2U as a promising combination.

B. More Unlearning and Verification Parameters

VI. CONCLUSION AND FUTURE WOK

  • VERIFI studies seven unlearning and five verification methods; the authors identify vEM with uS2U as the most promising approach without altering the FL process, requiring white-box access, or raising new security risks.The conclusion also presents VERIFI as an open test bed for future methods and benchmarking.Source: §VI · pp. 13–14

A. Parameter

This material specifies datasets and method parameters, then assesses parameter differences, membership inference, and the security risk of vBN as verification approaches or concerns.

A. Parameter

  • The study uses seven named datasets and specifies method settings, including uDP’s (ε,δ), uGS’s λ, and uS2U’s α and β; verification parameters are tabulated.The dataset descriptions include MNIST, CIFAR-10, ISIC, COVID, VGGFace_mini, ImageNet_mini, and Speechcommand.Source: A. Parameter · p. 16, Table IX · p. 16

B. Federated Unlearning Verification by Comparing Parameter Differences

C. Federated Unlearning Verification by Membership Inference

  • Membership inference is infeasible for verifying unlearning in these experiments: even uRT’s complete removal and retraining did not consistently lower the deduced membership ratio versus uNT.The authors attribute the difficulty to similar data belonging to other participants.Source: C. Federated Unlearning Verification… · p. 16, Figure 15 · p. 17

D. Security risk of vBN

  • Some unlearning methods leave vBN’s backdoor security risk, with attack success reaching 50%; robust aggregation such as Krum and Median can remove the threat.Figure 16 depicts the security threat of vBN.Source: D. Security risk of vBN · p. 17, Figure 16 · p. 17

E. Verification effect difference between unique memory samples and leaving data

The selected unique memory markers maintain better unlearning verification effects than using the leaving data alone, identifying the performance of methods beyond uRT and uDP.

E. Verification effect difference between unique memory samples and leaving data

F. Unlearning verification visualization

Grad-CAM visualizations show how attention to the vBN trigger changes after unlearning, with the clearest effect under uRT.

F. Unlearning verification visualization

  • After uRT, attention on the vBN trigger disappears; uRTB, uDP, and uS2U reduce it, while uNF, uCGS, uGGS, and uIGS show less obvious decreases.Before unlearning, the backdoor sample is classified as airplane and attention is mainly on the trigger; its original label is dog.Source: F. Unlearning verification visualization · p. 17

G. Theoretical Explanation

This section explains how uS2U changes vBF-based boundary verification and how uDP limits membership information about a participant. Its arguments use assumptions including FedAvg aggregation and a binary classifier.

  • vBF uses perturbed local training samples near the decision boundary as markers, assuming these remain boundary samples for the converged global model after verification is enabled.The paper identifies high, close top-two class probabilities as characteristic of boundary samples.Source: G. Theoretical Explanation · p. 18
  • Under FedAvg, uS2U reduces participant a's local update; the resulting global-model reduction enlarges the decision-boundary distance and changes results on boundary samples.The paper uses those result changes to verify unlearning.Source: G. Theoretical Explanation · p. 18, Figure 19 · p. 18
  • For uDP, membership advantage is the inferer's false-positive-rate gap and is bounded by e^ε−1, limiting information about whether z belongs to a's data.Source: G. Theoretical Explanation · p. 19
Loading 2205.12709v1…