PaperlayerView exact revision on arXiv

Source-linked AI summary

Federated Unlearning via Class-Discriminative Pruning

Junxiao Wang, Song Guo, Xin Xie, Heng Qi

arXiv:2110.11794v3cs.CVcs.CRcs.DCcs.LG

TL;DR

The paper addresses category-level unlearning in federated learning, where global training data are unavailable and non-IID local data challenge existing centralized methods. It proposes channel pruning guided by TF-IDF class discrimination, followed by fine-tuning, and reports faster unlearning than retraining from scratch without accuracy degradation.

  • Problem

    Federated learning requires removing a target category's influence from a trained model without globally accessing distributed, non-IID training data.

  • Method

    The method quantizes channel class discrimination with TF-IDF, prunes channels most relevant to the target category, and fine-tunes the pruned model.

  • Results

    Compared to retraining from scratch, unlearning speedups reach 8.9× and 7.9× on CIFAR10 and 9.9× and 8.4× on CIFAR100 for ResNet and VGG, respectively, without accuracy degradation.

  • Takeaways & Limitations

    The approach provides an early complementary block for federated unlearning aimed at supporting legal and ethical compliance without global training-data access.

  • Takeaways & Limitations

    The study focuses on class-level unlearning; sample-level unlearning remains a more challenging problem for future work.

Abstract

from arXiv · show

We explore the problem of selectively forgetting categories from trained CNN classification models in the federated learning (FL). Given that the data used for training cannot be accessed globally in FL, our insights probe deep into the internal influence of each channel. Through the visualization of feature maps activated by different channels, we observe that different channels have a varying contribution to different categories in image classification. Inspired by this, we propose a method for scrubbing the model clean of information about particular categories. The method does not require retraining from scratch, nor global access to the data used for training. Instead, we introduce the concept of Term Frequency Inverse Document Frequency (TF-IDF) to quantize the class discrimination of channels. Channels with high TF-IDF scores have more discrimination on the target categories and thus need to be pruned to unlearn. The channel pruning is followed by a fine-tuning process to recover the performance of the pruned model. Evaluated on CIFAR10 dataset, our method accelerates the speed of unlearning by 8.9x for the ResNet model, and 7.9x for the VGG model under no degradation in accuracy, compared to retraining from scratch. For CIFAR100 dataset, the speedups are 9.9x and 8.4x, respectively. We envision this work as a complementary block for FL towards compliance with legal and ethical criteria.

1 Introduction

The paper addresses category-level unlearning in federated learning, where global training data are unavailable and retraining from scratch is costly. It proposes TF-IDF-guided channel pruning followed by federated fine-tuning to remove target-category information efficiently.

  • Motivation: Federated learning requires selectively removing categories from trained CNNs without globally accessing private training data.The target model should behave as if the target category had never been observed.
  • Challenges: Retraining from scratch can make model unlearning prohibitively expensive, while existing centralized approaches assume access to global training data.Non-IID and incomplete local data further limit the accuracy of centralized unlearning approximations in FL.
  • Method: After pruning, clients fine-tune the model with target-category-excluded data using normal federated training.The pruning and fine-tuning workflow avoids retraining from scratch and does not require global raw-data access.
  • Insight: Different CNN channels contribute differently to image categories, motivating channel-level identification of category-specific information.Feature-map visualizations show distinct channels consistently activating head or textual information.
  • Method: The method treats channel outputs as words and category feature maps as documents, using TF-IDF to quantify class discrimination and prune channels most relevant to the target category.The workflow aggregates feature-map scores from devices at the server before pruning.
  • Results: 8.9× and 7.9× speedups are reported on CIFAR10 for ResNet and VGG, respectively, with no degradation in accuracy versus retraining from scratch.The corresponding CIFAR100 speedups are 9.9× for ResNet and 8.4× for VGG.

2 Preliminaries

Federated learning trains shared models without exchanging clients’ training data, but category unlearning is difficult because centralized methods rely on global data that FL does not provide. Existing approaches approximate retraining through Fisher-, influence-, or gradient-based updates, while retraining from scratch remains costly.

  • Federated Learning: Federated learning collaboratively trains a shared model while keeping client training data local.Clients perform local training and send model updates to a federated server for aggregation.
  • Machine Unlearning: Retraining from scratch can provably forget erased data but imposes substantial time, computation, and energy costs.Approximate unlearning methods instead seek models approximating those produced by full retraining.
  • Centralized Unlearning: Existing centralized unlearning methods use Fisher information, data influence, or corrected SGD steps to update trained models.These approaches respectively use remaining data, erased-data influence with second derivatives, or periodically recomputed exact gradients.
  • Federated Unlearning: Category unlearning removes a target category from a model trained on distributed data.The objective maps a learned model and target category to an updated model whose output distribution corresponds to the classification space without that category.
  • Challenges: Centralized methods are poorly suited to FL because they assume global training data, while FL exposes incomplete and non-IID local data.The resulting updating directions can produce inaccurate approximations of the global model.

3 Pathway to Channel Pruning

Channels in CNNs encode category-relevant local features with differing contributions across classes, motivating channel pruning for category forgetting. The method adapts TF-IDF to score channel–category discrimination, while targeting computational efficiency for large-scale FL.

  • 3 Pathway to Channel Pruning: The proposed pathway probes internal channel influence to forget categories without globally accessing the training data.It seeks channels whose class discrimination is greatest for the target categories and prunes them.
  • 3.1 Class Discrimination of Channels: Different CNN channels contribute unequally to different image categories by activating localized, category-relevant feature maps.In VGG16, one channel highlights heads for animal recognition, while another highlights text and deactivates for categories lacking textual semantics.
  • 3.2 TD-IDF based Channel Scoring: TF-IDF quantizes how discriminative each channel is for a category by treating channel outputs as words and category feature maps as documents.The score increases with channel relevance to a category while discounting channels common across categories.
  • 3.2 TD-IDF based Channel Scoring: Channels with high TF-IDF scores are selected for pruning because they carry greater class discrimination.This scoring design avoids the iterative optimization or heavy matrix operations used by some prior class-discriminative pruning methods.
  • 3 Pathway to Channel Pruning: Channel pruning can remove selected channels and shrink CNNs into thinner models supported by general-purpose hardware and BLAS libraries.Its structured simplification operates at the channel level rather than removing individual activations.

4 Federated Unlearning Framework

The framework uses client-generated channel–class representations and server-side TF-IDF scoring to prune channels discriminative for a target class. The pruned federated model is then fine-tuned, while the approach remains focused on class-level rather than sample-level unlearning.

  • Local Processing in FL Clients: Clients generate local channel–class representations from private images by recording layer feature maps, applying ReLU and average pooling, and averaging across classes.The resulting representations are uploaded to the federated server after local processing.
  • Processing in the Federated Server: The server averages uploaded representations to construct a global representation between classes and channels.This global representation supports subsequent channel-discrimination scoring.
  • TF-IDF-Based Pruning: TF-IDF measures channel relevance by combining category-specific contribution with how uncommon that contribution is across classes.Channels with higher TF-IDF scores are more relevant to the target class and are selected for pruning.
  • TF-IDF-Based Pruning: The server prunes the most discriminative target-category channels using a one-shot threshold, removing corresponding filters in adjacent layers.The pruning percentage is controlled by a predefined threshold R.
  • Fine-Tuning: After pruning, clients fine-tune the model until the target accuracy is restored, using a prune-once-and-retrain strategy across multiple layers.Fine-tuning follows the normal federated learning training procedure without additional regularization.
  • Discussion and Limitations: The framework is designed for class-level unlearning in federated settings, whereas sample-level unlearning remains an open challenge because individual data-point contributions are difficult to evaluate without raw data.Its advantages are also significantly diminished in centralized learning, where global raw-data access enables more diverse unlearning designs.

5 Experimental Evaluation

Experiments evaluate federated unlearning on CIFAR10 and CIFAR100 with ResNet and VGG models under biased, incomplete participant data. The method is compared with retraining and Fisher-based unlearning across speed, retained accuracy, information erasure, multi-class removal, privacy leakage, and class-wise accuracy distribution.

  • FL Settings: Experiments use CIFAR10 and CIFAR100 with ResNet and VGG models, including biased participant data and comparisons against retraining and Fisher-based unlearning.The evaluation models federated conditions in which participant data are incomplete and biased toward certain classes.
  • Unlearning speedup: 8.9–11.0× speedup is achieved for CIFAR10 ResNet, 7.9–9.1× for CIFAR10 VGG, and 9.9× and 8.4× for CIFAR100 ResNet and VGG.The reported speedup is measured against fully retraining the model from scratch.
  • Baseline comparison: The proposed method performs significantly better than Fisher unlearning because it is insensitive to training-data distribution and remains accurate with severely biased participant data.The Fisher method depends heavily on accurate global access to training data, whereas unavailable global data can make its corrective step imprecise.
  • Information erasure: 0% U-set accuracy matches full retraining while R-set precision is not lost, indicating comparable information erasure without sacrificing retained-category accuracy.The R-set contains retained categories and the U-set evaluates information about the target category.
  • Multi-class removal: Multi-class removal retains comparable speedups, and pruning the second class reduces after-pruned U-set accuracy to 0%.The two-category CIFAR10 experiment uses ResNet20.
  • Additional evaluations: On CIFAR10, attack success rates differ from full retraining by less than 0.63%, class-wise accuracy differs by at most 0.59%, and KL divergence remains below 5.27e-6.These evaluations compare privacy leakage and accuracy distributions after unlearning with those obtained by full retraining.

6 Related Work

Prior unlearning work spans statistical-query, clustering, linear-classification, and gradient-based approaches, while the paper distinguishes class-level forgetting from stricter sample-level removal.

  • Machine unlearning was first proposed in statistical query learning, but that approach is not applicable to neural networks.
  • Table 4 reports differences in membership-inference attack success rates between fully retrained and proposed models on CIFAR10.
  • Table 5 reports distances between classification-accuracy distributions for fully retrained and proposed models on CIFAR10.
  • Sample-level unlearning removes particular samples within a class, whereas this paper defines class-level unlearning for entire categories.
  • Existing unlearning methods also include data-removal procedures for k-means clustering and corrective updates for supervised linear classifiers.

7 Conclusion

The paper defines selective category forgetting for trained CNNs in federated learning and proposes scrubbing without global training-data access or retraining from scratch.

  • The method scrubs information about particular categories from trained CNN classification models without globally accessing training data or retraining from scratch.
  • Compared with a gold-standard model retrained without target-category data, the method achieves faster unlearning with no degradation in accuracy.
  • The paper positions this work as an early complementary block for federated learning compliance with legal and ethical criteria.
  • Future work will study fine-grained removal of subsets of samples within a class while preserving output knowledge of that class.

A.1 Multi-class removal: extensive cases

The appendix examines simultaneous removal of two CIFAR10 categories and extends evaluation across additional pretrained ResNet and VGG models.

  • Multi-class removal: extensive cases: Two CIFAR10 categories are deleted simultaneously from ResNet20 in the multi-class removal experiment.
  • Multi-class removal: extensive cases: Additional CIFAR10 experiments use pretrained ResNet20, ResNet56, VGG11, and VGG19 models.
  • Multi-class removal: extensive cases: The experiments use FedSGD with learning rate 0.1 and bias probability 0.1(0.01).
  • Multi-class removal: extensive cases: The appendix evaluates these cases for unlearning acceleration and information erasure.

A.2 Term frequency inverse document frequency (TF-IDF): formal definition

The paper adapts TF-IDF by treating channels as terms and category feature maps as documents, then combines term frequency and inverse document frequency into a relevance score.

  • Term frequency inverse document frequency (TF-IDF): formal definition: TF-IDF for term t in document e from document set E is defined as TF(t, e, E) = TF(t, e) ∗ IDF(t, E).
  • Term frequency inverse document frequency (TF-IDF): formal definition: Inverse document frequency uses the document-set size and the number of documents containing term t, with smoothing in the paper’s formula.
  • Term frequency inverse document frequency (TF-IDF): formal definition: The inverse document frequency is calculated as IDF(t, E) = log[(|E| + 1)/(count(e, e ∈E ∩t ∈e) + 1)].
  • Term frequency inverse document frequency (TF-IDF): formal definition: Term frequency divides the occurrence count freq(t, e) by document length |e| to reduce document-length effects.

A.3 ReLU activation and avg pool2d operation: more details in Section 4.1

The local procedure derives class–channel representations from private client images by recording layer feature maps, reducing them through ReLU and average pooling, and aggregating class-wise activations.

  • Each client uses local private images to generate a representation linking channels with categories.
  • The procedure records the feature map produced by the local model in each layer.
  • ReLU and average pooling reduce each layer’s feature-map tensor for category images before channel activation values are formed.
  • Local representations encode the relationship between classes and channels.
  • Global representations for each category are obtained by averaging local representations across images of that class.

A.4 Algorithm of the framework

The framework coordinates clients and the server to compute representations, prune channels associated with requested classes, and fine-tune the resulting model. Its algorithms specify the local representation procedure and the federated unlearning workflow.

  • Algorithm organization: Algorithm 2 contains the server-side procedure, while Algorithm 3 specifies the overall framework and Algorithm 1 specifies local processing.
  • Algorithm 1: Local procedure: Algorithm 1 takes local private images and a pretrained local FL model, then returns a class–channel representation.
  • Algorithm 1: Local procedure: For every model layer, the client records the layer output feature map as part of local processing.
  • Algorithm 3: Federated framework: Algorithm 3 takes participating clients, a pretrained FL model, and target classes to forget, returning a model with those classes unlearned.
  • Algorithm 3: Federated framework: Upon an unlearning request, the server notifies clients, distributes the unlearning program, and collects their locally calculated representations.
  • Algorithm 3: Federated framework: After receiving all client uploads, the server performs server-side pruning, followed by fine-tuning of the pruned model.
Loading 2110.11794v3…