Source-linked AI summary
Split Federated Unlearning
Jia Chen, Yecheng Lv, Xinrong Hu, Mingdi Huang, Jun Bai
TL;DR
SFL combines partitioned execution with federated aggregation, but existing unlearning formulations do not capture their combined effects. The paper formulates SFU and proposes SURE, an initial importance-guided approach. Across experiments, SURE achieves competitive agreement with mode-matched retraining at modest additional cost, while the paper identifies scope boundaries including retained-data needs and unexplored formal guarantees.
Problem
Existing unlearning formulations do not capture SFL’s combination of cross-partition coupling and historical influence propagation through dual-side aggregation.
Source: §1 · p. 1Method
The paper formulates client-wise and class-wise SFU with counterfactual retraining and partition-specific update constraints, then proposes SURE using dual-importance estimation, selective perturbation, and guided recovery.
Source: §1 · pp. 1–2Results
Takeaways & Limitations
The formulation and theoretical characterization establish SFU as a distinct research problem and provide a foundation for unlearning in partitioned federated systems.
Source: §7 · p. 9SURE’s recovery requires retained data, formal forgetting guarantees remain unexplored, and agreement with mode-matched retraining under restricted updates need not imply equivalence to full retraining.
Source: §6 · p. 8
Abstract
from arXiv · showhide
Split federated learning (SFL) combines partitioned model execution with federated aggregation to support collaborative learning across distributed Web and edge environments. However, removing learned contributions in SFL is challenging due to cross-partition coupling and historical influence propagation through dual-side aggregation. To the best of our knowledge, we provide the first systematic formulation of Split Federated Unlearning (SFU), encompassing client-wise and class-wise removal, counterfactual retraining, and partition-specific update constraints. We theoretically characterize historical influence propagation and mode-dependent unlearning feasibility, revealing structural limitations of restricted model updates. As an initial solution, we propose SURE, integrating dual-importance estimation, selective parameter perturbation, and importance-guided recovery. Extensive experiments with existing unlearning methods adapted to SFL reveal substantial forgetting-utility trade-offs across model architectures, data distributions, and update modes. SURE achieves competitive agreement with mode-matched retraining at modest additional cost, while further analyses highlight the impact of split position and update constraints on unlearning behavior. These findings establish SFU as a distinct research problem and provide a foundation for unlearning in partitioned federated systems.
1 Introduction
SFL combines split model execution with federated aggregation, but existing unlearning formulations do not capture both mechanisms operating together. The paper formulates SFU, characterizes its structural constraints, and proposes SURE, which achieves competitive agreement with mode-matched retraining at modest additional cost.
1 Introduction
- SFL’s split execution and separate aggregation leave learned contributions intact when users withdraw data or leave; deleting records or ending participation cannot undo accumulated model influence.Existing federated and split unlearning formulations do not directly capture SFL, where both mechanisms operate together.Source: §1 · p. 1
- SFU’s challenges are cross-partition coupling through activations and gradients, historical influence propagation across clients and rounds, and update-mode restrictions that may limit forgetting.The figure frames client-wise and class-wise removal against full retained-data retraining under different update modes.Source: §1 · p. 1, Figure 1 · p. 2
- The paper characterizes historical influence propagation and mode-dependent feasibility using prediction-discrepancy bounds and excess retained-data risk under a quadratic approximation.The resulting structural limitations arise from model partitioning, independently of any particular unlearning algorithm.Source: §1 · pp. 1–2
- SURE achieves competitive agreement with mode-matched retraining at modest additional cost across evaluations spanning datasets, data distributions, and model update modes.The adapted unlearning baselines show substantial forgetting–utility trade-offs.Source: §1 · p. 1
- The paper presents the first systematic SFU formulation for client-wise and class-wise removal, partition-specific update constraints, and cross-partition coupling under dual-side aggregation.Full retained-data retraining is the ideal counterfactual reference; mode-matched retraining captures the effects of restricted updates.Source: §1 · pp. 1–2
2 Related Work
Prior work studies federated unlearning, split unlearning, and SFL training, but does not jointly address dependencies between split partitions and repeated federated aggregation. The paper positions SFU as addressing that combined setting and its partition-specific update constraints.
Federated Unlearning
- Federated unlearning methods address contributions propagated through model aggregation, but do not explicitly account for dependencies between separately aggregated partitions.Prior approaches include update calibration and selective retraining [20], privacy-preserving recovery, and methods for class-specific pruning, subgraph removal, gradient conflicts, and task-vector alignment [31] [16] [22] [24].Source: §2 · p. 2
Split Learning and Unlearning
- Split unlearning addresses training dependencies through isolation-based removal and gradient-cache projection with retained-data fine-tuning, but does not jointly characterize repeated federated aggregation.Parameter-selective SalUn [5] demonstrates the role of weight saliency in targeted unlearning.Source: §2 · p. 2
Split Federated Learning
- SFL research has studied server-side parallelism and instability under heterogeneous data [28], primarily focusing on training efficiency and knowledge preservation rather than deliberate removal.Unlearning in SFL must additionally account for cross-partition dependencies, dual-side aggregation, and partition-specific update constraints.Source: §2 · p. 2
3 Split Federated Unlearning
The section models SFL as coupled client/server partitions with independent dual-side aggregation, then formalizes SFU against retained-data retraining under partition-specific update modes. Its analysis characterizes historical influence propagation and structural feasibility and excess-risk limits under restricted updates.
3.1 System Model
- Each client prediction composes a client-side model with its server-side replica; split computation couples the partitions while dual-side aggregation propagates contributions across rounds.Client and server partitions are aggregated independently, and both aggregated partitions initialize the next round.Source: §3.1 · pp. 2–3
3.2 Problem Formulation
- SFU separates client- or class-wise removal requests from permitted Client-only, Server-only, or Joint updates; full retained-data retraining is ideal, while mode-matched retraining isolates update restrictions.For client-wise removal, the withdrawing client is excluded from subsequent participation; for class-wise removal, clients with retained data may remain.Source: §3.2 · p. 3
- SFU seeks to approximate full retraining while preserving retained-data utility and limiting computation and communication, using predictive discrepancy, retained-data risk, and unlearning cost as objectives.Neither full retraining nor mode-matched retraining is assumed available to the unlearning method.Source: §3.2 · p. 3
3.3 Theoretical Characterization
- Theorem 3.1 decomposes historical influence into round-wise discrepancies propagated through later updates; mixed derivatives couple partition dynamics, so removing only a client's final-round update generally misses retained-only training.Round-wise discrepancies also capture changes in client participation and aggregation weights.Source: §3.3 · p. 3
- The local feasibility bound measures the retraining shift inaccessible to first-order updates within permitted partitions; if d_M > Lρ^2/2, exact predictive agreement is impossible within the update ball.The bound assumes a continuously differentiable predictor with an L-Lipschitz derivative near the original model.Source: §3.3 · pp. 3–4
- For the same radius, Joint updates include both single-partition feasible sets; the bounds describe attainable rather than algorithm-specific performance.The feasible-set inclusion is stated as a mode hierarchy.Source: §3.3 · p. 4
- Under a local positive-definite quadratic surrogate, single-partition updates have nonnegative minimum excess retained-data risk, strictly positive for nonzero displacement in the frozen partition.Cross-partition adaptation can reduce, but not necessarily eliminate, this excess risk.Source: §3.3 · p. 4
4 SURE: An Initial Solution
SURE approximates split federated unlearning through dual-importance estimation, selective perturbation, and retained-data recovery, with updates constrained by the permitted mode. Its stages identify target- and retention-relevant parameters, perturb a selected layer, then recover retained utility while preserving frozen states.
4 SURE: An Initial Solution
- SURE starts from the synchronized checkpoint, estimates forgotten- and retained-data importance, selectively perturbs a target-relevant layer, then recovers retained utility through constrained split-federated training.Only partitions permitted by the update mode may change.Source: §4 · p. 4
4.1 Dual-Importance Estimation
- Dual-importance estimation scans forgotten and retained data without updates, using gradient-squared proxies and a variance discount that favors parameters consistently important across retained clients.The importance maps are normalized for forgotten data and for each retained client.Source: §4.1 · p. 4
4.2 Selective Parameter Perturbation
- SURE selects one layer with the highest mean forgotten-data importance among layers in permitted partitions, then scales parameters using target-specific scores discounted by retained-data importance.Other layers and frozen partitions remain unchanged; perturbation alone does not establish unlearning.Source: §4.2 · pp. 4–5, Figure 2 · p. 5
4.3 Retained-Data Recovery
- Recovery trains on retained data through the original SFL protocol, modulating gradients to reinforce retained-relevant updates and attenuate forgotten-relevant ones while aggregating permitted partitions separately.Frozen parameters, replicas, and persistent buffers remain fixed; SURE targets approximate, not certified, unlearning.Source: §4.3 · p. 5
5 Experimental Evaluation
Across client-wise and class-wise removal, adapted methods show forgetting–utility trade-offs, while SURE achieves competitive agreement with mode-matched retraining. The evaluation examines how data settings, update modes, split position, method components, and system costs shape these outcomes.
5.1 Experimental Setup
- Client-wise experiments cover four dataset–model configurations, IID and extreme non-IID client partitions, and Client-only, Server-only, and Joint update modes.Class-wise removal uses Dirichlet non-IID partitions with α=0.1; evaluation reports Retain, Forget, Test, MIA, agreement with mode-matched Retrain, communication, and computation.Source: §5.1 · pp. 5–6
5.2 Unlearning Performance Comparison
- SURE records 0.40% Forget, 56.30% Test, and a 1.76 pp Avg. Gap for CIFAR-10/ResNet-18 client-wise removal with Server-only updates.For class-wise removal on MNIST/LeNet with Server-only updates, SURE matches Retrain’s 0.00% Forget and 88.96% Test, with a 0.06 pp Avg. Gap; on CIFAR-10/ResNet-18 Joint updates, it has 83.72% Retain, 71.33% Test, 0.10% Forget, and a 1.15 pp gap, while FedOSD has a smaller 0.61 pp gap.Source: Table 1 · p. 6, §5.2 · pp. 6–7, Table 2 · p. 7
5.4 Ablation Study
- Full SURE has the lowest Avg. Gap under Server-only (1.63 pp) and Joint (1.90 pp), and ties A2 under Client-only (0.41 pp).A1 and A4 consistently increase the gap; under Joint updates, A6 yields 72.42% Test and A7 yields 70.11% Forget, versus SURE’s 77.36% Test and 65.34% Forget.Source: Table 3 · p. 8, §5.4 · p. 8
5.5 Efficiency Analysis
- On CIFAR-10/ResNet-18 with Joint updates, SURE requires 218 GB and 1,370 TFLOPs, versus Fine-tune’s 213 GB and 1,370 TFLOPs and MoDe’s 449 GB and 2,890 TFLOPs.FedRecovery has substantially lower costs through training-free correction using stored historical updates, but historical storage costs are excluded.Source: Table 4 · p. 8, §5.5 · p. 8
5.3 Impact of Split Position on Unlearning
- Across nine split positions, deeper cuts lower Client-only Test and Retain accuracy but raise Server-only accuracy; Joint behavior is non-monotonic and Forget varies by mode.Final-stage CKA relative to Original changes from 0.980 to 0.908 between splits 1 and 8 for Client-only, and from 0.954 to 0.980 for Server-only.Source: Figure 3 · p. 7, Figure 4 · p. 7, §5.3 · p. 8
6 Limitations
SURE’s effectiveness varies across update modes, split positions, and forgetting scenarios. Its recovery requires retained data, formal forgetting guarantees remain unexplored, and agreement with mode-matched retraining under restricted updates does not ensure equivalence to full retraining.
6 Limitations
- SURE’s effectiveness varies across model update modes, split positions, and forgetting scenarios.Source: §6 · p. 8
- SURE’s recovery stage requires retained data, and formal forgetting guarantees remain unexplored.Source: §6 · p. 8
- Agreement with mode-matched retraining under restricted updates does not necessarily imply equivalence to full retraining.Source: §6 · p. 8
7 Conclusion
The paper establishes Split Federated Unlearning as a distinct problem and presents SURE as an initial approach. In an IID client-wise setting, SURE’s CIFAR-10/ResNet-18 test accuracy under Joint updates is close to Retrain’s, while FedEraser and MoDe score lower.
7 Conclusion
- The paper establishes SFU as a distinct research problem, with a unified formulation and theoretical characterization of historical influence propagation and mode-dependent feasibility.Source: §7 · p. 9
- SURE integrates dual-importance estimation, selective perturbation, and guided recovery as an initial SFU solution.Source: §7 · p. 9
A.1 Client-Wise Unlearning under IID Partitions
- SURE achieves 88.01% Test accuracy on CIFAR-10/ResNet-18 under Joint updates, close to Retrain (87.60%).Source: §A.1 · p. 10
- On CIFAR-10/ResNet-18 under Joint updates, FedEraser reaches 57.83% and MoDe 46.78% Test accuracy, compared with SURE’s 88.01% and Retrain’s 87.60%.Source: §A.1 · p. 10
- Deviations in Forget accuracy and MIA, particularly on CIFAR-100, distinguish utility preservation from agreement with retraining.Source: §A.1 · p. 10
A.2 Additional Representation Analysis
Representation analyses show that responses vary with split placement and update permissions, while larger client-removal requests challenge utility and retraining consistency. No single tested hyperparameter configuration is optimal across update modes.
A.2 Additional Representation Analysis
- Under Client-only updates, forgotten samples remain intermixed at split 1 but form a more concentrated target-sample distribution at split 8; Server-only and Joint updates produce more distinct neighborhoods at split 1.The authors caution that t-SNE separation alone does not establish forgetting effectiveness.Source: §A.2 · p. 10
- At the client partition, class distributions remain relatively mixed, while server-partition neighborhoods are more structured; SURE and Retrain also show different target-sample arrangements.Similar prediction-level forgetting behavior does not necessarily imply identical learned representations.Source: §A.2 · p. 10
A.3 Multiple Forgetting Clients
- Under Joint updates, increasing the forgetting fraction from 10% (10–1) to 50% (10–5) drops Test accuracy from 77.36% to 56.42% and raises Avg. Gap from 1.90 to 4.36 pp.Source: §A.3 · p. 10
A.4 Hyperparameter Sensitivity
- The minimum Avg. Gap occurs at γ=1.25 for Client-only, γ=1.0 for Server-only, and γ=0.75 for Joint updates.Source: §A.4 · p. 10
- No single recovery-coefficient configuration is uniformly optimal across modes, so the main experiments use the shared default (γ, β_R, β_F) = (1, 0.25, 0.75) as a practical compromise.Source: §A.4 · p. 10
B.1 Proof of Theorem 3.1
The proof passages derive Eq. (8) by iterating a trajectory recurrence from shared initialization, establish Eq. (12) by taking a nonnegative part and infimum, and show that Joint updates do not increase the feasibility residual.
B.1 Proof of Theorem 3.1
- Iterating the trajectory recurrence from the shared initialization δ0 = 0 gives Eq. (8).Source: §B.1 · p. 10
B.2 Proof of Theorem 3.2
- Taking the nonnegative part and infimum over admissible updates proves Eq. (12).Source: §B.2 · p. 10
B.3 Proof of Corollary 3.3
- Because the Client-only and Server-only feasible sets are included in the Joint set, the projection residual cannot increase under Joint updates.The proof extends the hierarchy to nonlinear feasible sets for a common ρ.Source: §B.3 · p. 10
B.4 Proof of Proposition 3.4
The proof uses positive-definite Schur complements to establish nonnegativity and strict positivity for nonzero frozen-partition displacement. The supplied figure and table captions identify client-wise visualizations and IID evaluation measures.
B.4 Proof of Proposition 3.4
- Positive-definite Schur complements establish nonnegativity and strict positivity for nonzero frozen-partition displacement.The proof states that both Schur complements are positive definite because H_R ≻ 0.Source: §B.4 · pp. 10–11
- Table 5 reports client-wise unlearning under IID partitions using Ret., For., Test, MIA, and Avg. Gap relative to mode-matched Retrain.Avg. Gap is the mean absolute distance of those four measures from mode-matched Retrain, in percentage points.Source: Table 5 · p. 11
- Figure 5 compares Client-only, Server-only, and Joint updates at split 1 and split 8 for client-wise unlearning.It uses t-SNE; colors distinguish retained-training, non-member, and forgotten samples before and after SURE.Source: Figure 5 · p. 11
C.1 Experimental Settings
The experiments vary data partitions, model cut positions, training configurations, and permitted update modes. They also specify how baselines, membership inference, metrics, and computational costs are evaluated.
C.1 Experimental Settings
- Extreme non-IID partitions assign one exclusive class per client on MNIST and CIFAR-10, and ten classes per client on CIFAR-100.Class-wise, ablation, sensitivity, and multiclient experiments use Dirichlet α=0.1.Source: §C.1 · p. 11
- Client partitions use architecture-specific early layers, while ResNet-18 experiments examine nine cut positions separately.The stated client partitions are LeNet’s first convolution-pooling block, CNN’s first two convolutional blocks, or the ResNet-18 stem and first residual block.Source: §C.1 · p. 11
- Original and Retrain share round budgets, and checkpoint selection uses validation accuracy without using test data.Unlearning selects the highest-validation-accuracy checkpoint satisfying the forgetting constraint; otherwise it reports the lowest-target-validation-accuracy checkpoint as failing the constraint.Source: §C.1 · pp. 11–12
- The class-wise representation figure compares Original, Retrain, and SURE at client cut-layer and server pre-classifier representations.Colors denote classes, and black triangles mark forgotten class-0 samples.Source: Figure 6 · p. 12
- Table 6 covers SURE with multiple forgetting clients on CIFAR-10/ResNet-18 under Dirichlet α=0.1.Its configurations are denoted by total clients–forgetting clients, and Avg. Gap is computed against the corresponding Retrain reference.Source: Table 6 · p. 12
- SURE uses batch size 32, at most 40 recovery rounds, and coefficients (γ, β_R, β_F) = (1, 0.25, 0.75).The sensitivity figure varies perturbation protection and recovery coefficients across Client-only, Server-only, and Joint updates; lower Avg. Gap is better.Source: §C.1 · p. 12, Figure 7 · p. 12
C.2 Baseline Adaptation to SFL
- Baselines share SURE’s partitions, architectures, and requests, with operations adapted to split computation and dual-side aggregation.Parameter changes respect permitted update modes; FedEraser and FedRecovery are evaluated only for client-wise unlearning.Source: §C.2 · p. 12
C.3 Membership Inference Attack
- The membership attacker is fitted and calibrated on Original, then held fixed across Retrain and unlearning methods.ROC-AUC measures membership distinguishability; 50% indicates chance-level discrimination, not certified forgetting.Source: §C.3 · p. 13