Source-linked AI summary
Not All Retained Clients Are Equal: Client Selection for Federated Unlearning
Thanh Linh Nguyen, Quoc-Viet Pham
TL;DR
Federated unlearning seeks to remove designated clients’ influence while preserving retained-client knowledge, but existing approaches overlook path-dependent client influence and face resource or availability constraints. PDF uses CLP-weighted histories to select retained clients for retraining-based unlearning, and experiments report effective forgetting, strong retained utility, and reduced participation and costs.
Problem
Existing federated-unlearning methods often treat removal as stateless, although client influence depends on when contributions occur, and retraining can require substantial resources and continuously available retained clients.
Source: §1 · p. 2Method
PDF uses lightweight, target-agnostic CLP-weighted training histories to rank retained clients by global informativeness and coherence with the target set, then retrains using only the top-k clients.
Source: §1 · p. 2Results
Experiments across FMNIST, CIFAR-10, and CIFAR-100 under IID and non-IID settings report effective forgetting and strong retained utility with substantially reduced client participation, communication, and computation.
Source: §1 · p. 2Takeaways & Limitations
PDF’s selected-client retraining achieves effective forgetting and strong retained utility while reducing participation and resource use, with the forgetting–utility trade-off adjustable through client selection.
Source: §6 · p. 8The stated unlearning setup covers only a nonempty target-client set that is a proper subset of all clients.
Source: §3.2 · p. 3
Abstract
from arXiv · showhide
Federated unlearning (FU) aims to remove a designated target client's influence from a trained model without the prohibitive cost of retraining from scratch. Existing methods largely treat unlearning as a post-hoc, stateless operation, overlooking that client influence is path-dependent: contributions made during training, particularly during critical learning periods (CLPs), can have disproportionate and persistent effects. Moreover, knowledge overlap and intermittent client availability make relying on all retained clients inefficient and impractical. We introduce PDF, a path-dependent client-selection framework based on the insight that retained clients are not equally valuable for unlearning. During training, PDF maintains lightweight, target-agnostic, CLP-weighted histories summarizing each client's accumulated update direction and global-model alignment. Upon an unlearning request, these histories identify and rank retained clients that are globally informative yet weakly coupled to the target, enabling retraining-based unlearning with only the top-$k$ clients over a small number of rounds. Experiments on FMNIST, CIFAR-10, and CIFAR-100 under IID and non-IID settings show that PDF achieves effective forgetting and strong retained utility with substantially fewer retained clients. In particular, PDF achieves $0\%$ forgetting accuracy with retraining-like MIA AUC on non-IID CIFAR-10, improves retained accuracy by up to $6.91$ percentage points over the strongest competing FU baseline on CIFAR-100, and reduces communication by approximately $69\%$ and computation by $57$--$61\%$ relative to Retrain.
1 Introduction
PDF treats client-level federated unlearning as path-dependent: client value depends on when and how its influence accumulated and its coupling with targets. It uses training histories to select retained clients, and experiments report effective forgetting and strong retained utility with fewer clients and reduced costs.
1 Introduction
- Existing FU methods overlook when client influence develops, while retraining incurs high resource costs and depends on retained-client availability; model manipulation can trade forgetting efficacy against retained utility.Critical learning periods can have disproportionately persistent effects, and may emerge beyond a fixed early-training window.Source: §1 · pp. 1–2
- PDF frames client-level unlearning as path-dependent because retained clients differ in when and how they contributed and in their historical dependence on target clients.Its selection objective uses historical global-model alignment and target dependence rather than uniform selection or availability alone.Source: §1 · p. 2
- PDF ranks retained clients using lightweight, target-agnostic histories of global-model alignment, update direction, and CLP participation, then retrains with only the top-k clients.The selection favors globally informative clients weakly coupled with the target set.Source: §1 · p. 2
- Experiments across FMNIST, CIFAR-10, and CIFAR-100 under IID and non-IID settings show effective forgetting and strong retained utility with reduced client participation, communication, and computation.Ablations also validate the ranking and CLP-aware design.Source: §1 · p. 2
2 Related work
Related work describes machine unlearning methods that modify training or learned models, while federated unlearning extends removal to client-, sample-, or class-level contributions. The discussion identifies a gap between CLP-aware training research and using temporal training signals to guide client-level unlearning.
2 Related work
- Machine-unlearning methods modify training or learned models, while federated unlearning targets client-, sample-, or class-level removal; client-level methods approximate retraining or manipulate model components.FL poses distinct constraints because servers lack access to local datasets and clients may be intermittently available.Source: §2 · p. 2
- Critical learning periods are intervals when updates have persistent effects on representations and final performance; analogous critical rounds reflect temporally non-uniform client influence in FL.Prior CLP-aware FL work uses these dynamics to improve participation, aggregation, or training efficiency.Source: §2 · p. 3
- The discussion identifies an open question: whether temporal patterns captured during training can help identify retained clients valuable for unlearning.PDF investigates this by incorporating CLP-weighted signals into client-level unlearning and client selection.Source: §2 · p. 3
3 Methodology
The system model describes federated training with sampled clients performing local gradient steps and a target-agnostic training history. The problem definition then specifies a post-training target set and uses retraining on retained clients as the reference for unlearning.
3.1 System Model
- The FL system has a central server and N clients with private, record-disjoint datasets and a shared model architecture.Training uses FedAvg over Tlearn rounds, sampling participating clients and having them perform E local gradient steps; the resulting local update is denoted by h_i.Source: §3.1 · p. 3
- After training, the server has a target-agnostic history that depends on neither the target set nor a later unlearning request.The passage identifies this history as recorded during training; its details are deferred to Section 4.Source: §3.1 · p. 3
3.2 Problem Definition
- Client-level unlearning designates a nonempty proper subset of clients for removal, leaving the remaining clients as the retained set.The retraining reference reruns the original training algorithm from scratch using only retained clients; unlearning seeks retraining-like behavior for retained and target clients.Source: §3.2 · pp. 3–4
4 Path-Dependent Client Selection for Unlearning
PDF uses training histories to select retained clients for unlearning, rather than treating all retained clients as equally useful. It detects critical learning periods, tracks client update direction and global alignment, and ranks clients by global alignment and historical decoupling from the target set.
4 Path-Dependent Client Selection for Unlearning
- PDF uses target-agnostic, CLP-weighted training histories to rank retained clients by their suitability for unlearning, without requiring the target set during training or storing per-round model snapshots.Its statistics are computed on the classification layer.Source: §4 · p. 4
4.1 Learning stage
- FGN-based detection flags critical rounds when the previous round's proxy meets a threshold, allowing early and late critical periods to be identified.PDF uses the resulting CLP information to emphasize clients' accumulated update directions and historical alignment with the global trajectory.Source: §4.1 · pp. 4–5
4.2 Unlearning stage
- The unlearning score favors retained clients that are globally aligned and historically decoupled from the target set, with CLP contributions emphasized.A larger β favors retained-model utility, while a smaller β favors target forgetting; only the top-k clients participate, scaling unlearning communication and computation with k rather than all retained clients.Source: §4.2 · pp. 5–6
5 Experimental evaluation
Experiments compare PDF with eight FU baselines across three datasets and IID/non-IID settings. PDF preserves retained accuracy while forgetting target data, and the ablations examine client-count and ranking choices.
5.1 Experiment setup
- The evaluation uses FedAvg with 64 clients, samples 16 per round, and tests FMNIST, CIFAR-10, and CIFAR-100 under IID and Dirichlet non-IID partitions.It compares PDF with Retrain, CONDA, FedRecovery, FOUL, FedEraser, Exact-Fun, FUSED, and GDFA, using forgetting accuracy, retained accuracy, MIA AUC, communication, computation, and space complexity.Source: §5.1 · p. 6
5.2 Main result
- On CIFAR-100, PDF reaches 49.13% RA versus 42.22% under α = 0.1 and 54.38% versus 47.50% under α = 1.0; communication falls approximately 69% and computation 57–61% versus Retrain.Table 1 evaluates three PDF variants using k = 25; β controls the forgetting–utility trade-off.Source: §5.2 · pp. 6–7, Table 1 · p. 7
- PDF’s learning cache uses O(Ndcls) storage independent of learning rounds, and unlearning tensors scale with O(kP) rather than all retained clients.Table 2 compares methods’ space complexity; PDF stores one accumulated classification-layer update direction and two scalar statistics per client.Source: §5.2 · p. 7, Table 2 · p. 7
5.3 Ablation study
- Increasing k raises retained accuracy with diminishing gains while communication and computation rise; selection-variant ablations also favor PDF, especially under α = 0.1.At β = 0.5, RA rises from 79.26% at k = 10 to 85.97% at k = 25, then to 86.40% at k = 30; removing CLP weighting lowers CIFAR-10 RA by 4.88 points.Source: §5.3 · pp. 7–8, Table 3 · p. 8
6 Conclusion
The conclusion frames client-level federated unlearning as path-dependent: retained clients differ in usefulness according to when and how their influence accumulated and their coupling to targets. PDF ranks clients using CLP-weighted history and retrains with a selected subset.
6 Conclusion
- PDF combines CLP-weighted historical global-model alignment and client-target coherence to rank retained clients.Retraining-based unlearning then uses only a selected top-k subset.Source: §6 · p. 8
- The paper argues retained-client usefulness depends on when and how influence accumulated and how strongly it is coupled to target clients.Source: §6 · p. 8
- Experiments report effective forgetting and strong retained utility with reduced client participation, communication, and computation; adjusting the forgetting–utility trade-off supports different application requirements.Source: §6 · p. 8
A.1 CNN Architecture for FMNIST
The FMNIST CNN has a feature extractor with three convolutional blocks and a classifier. Its flattened feature representation is 576-dimensional before classification.
A.1 CNN Architecture for FMNIST
- The feature extractor uses three convolutional blocks with channel counts 32, 64, and 64, combining convolution, normalization, ReLU, pooling, and dropout as specified.The resulting feature map is 64 × 3 × 3 and is flattened to 576 dimensions.Source: §A.1 · p. 11
- The classifier maps the 576-dimensional representation to 256 units with ReLU and dropout, then outputs 10 classes.Source: §A.1 · p. 11
- All convolutional and linear layers use Kaiming uniform initialization with zero biases.Source: §A.1 · p. 11
A.2 Federated Unlearning Baselines
The baselines range from full retraining to replay-based and server-side model-manipulation methods, trading unlearning approach, client participation, cost, and retained-knowledge preservation.
A.2 Federated Unlearning Baselines
- Retrain is the gold-standard baseline: it removes target clients and trains from scratch on the remaining clients, guaranteeing exact unlearning but requiring substantial computation and their availability.Its cost and participation requirements can make it impractical.Source: §A.2 · p. 12
- FedEraser approximates retraining by replaying cached periodic updates from remaining clients through calibrated aggregation, reducing time and storage overhead.Source: §A.2 · p. 12
- Exact-Fun uses quantized FL and local retraining after deletion; it retrains the global model only when the quantized model state changes.Source: §A.2 · p. 12
- CONDA tracks client parameter contributions and dampens parameters most associated with the target, without requiring client participation.Source: §A.2 · p. 12
- FedRecovery subtracts accumulated target-client gradient residuals and adds calibrated Gaussian noise to make the recovered model statistically indistinguishable from ideal retraining.Source: §A.2 · p. 12
- FUSED fine-tunes sparse adapters on task-critical layers using remaining clients, reducing unlearning cost while preserving reversibility.Source: §A.2 · p. 12
- FOUL disentangles causal and non-causal components, then optimizes client-gradient coefficients to align with retain-client gradients and reduce alignment with target-client gradients.It does not require the server to access client data.Source: §A.2 · p. 12
- GDFA derives target-knowledge task vectors from perturbed models fine-tuned on forgetting data, then negates directionally consistent components merged by sign consensus.It aims to preserve retained knowledge but requires request-time access to forgetting data and extra computation.Source: §A.2 · p. 12