FedASAP: Activation Statistics-driven Structured Adaptive Pruning for Efficient Personalized Federated Learning for Lesion Segmentation on brain MRI

1 Introduction

• Existing federated-pruning methods are mostly developed for classification, while unstructured pruning produces sparse masks that are inefficient on standard hardware.The introduction also notes that gradient-based importance scores or mean activations may not provide comprehensive task-relevant criteria for segmentation.Source: §1 · p. 2
• FedASAP first ranks filters by importance, then uses a lightweight GLM trained on per-filter activation features to select filters for pruning or retention.The method removes whole filters and dependent parameters to produce structured compression.Source: §1 · p. 2
• FedASAP is reported to achieve significant model compression and improved segmentation accuracy in heterogeneous federated settings, including low-data clients, with minimal training communication overhead.The introduction also reports faster inference, lower peak memory, and reduced client-level divergence from structured pruning.Source: §1 · p. 2
• Clients upload only indices of newly pruned filters, reducing the communication payload compared with sending full model updates.The introduction describes this as a communication-efficiency contribution.Source: §1 · p. 2
• The paper presents structured pruning as a form of implicit regularization that reduces client-level divergence by removing unimportant filters.The introduction also identifies data heterogeneity as a source of divergence between locally trained models.Source: §1 · pp. 1–3

2 Related Works

• Earlier personalized FL methods fine-tune global models or share selected parameters, while pruning offers a route to lightweight models aligned with client distributions.Source: §2 · p. 3

2.1 Neural network pruning

• Structured pruning removes grouped components for practical size reduction, but consistently ranking unimportant groups is harder because important weights may lie within them [33] [35].Source: §2.1 · p. 3
• Activation-based filter ranking has used only a few, often single, handcrafted statistics that may miss task-relevant characteristics, motivating more representative features alongside gradient scores.Source: §2.1 · pp. 3–4

2.2 Pruning in Federated Learning

• PruneFL ignores client-specific characteristics under non-IID data, while AutoFLIP better accommodates heterogeneity but still yields a costly single global model.Source: §2.2 · p. 4
• FedMef [29] uses scaled activation pruning for memory efficiency but may blur lesion boundaries, while FedDUAP [40] requires server-side data access.Source: §2.2 · p. 4
• FedDST aggregates sparse-mask votes into a global sparse network, prioritizing global sparsity over client-specific models.Source: §2.2 · p. 4
• Prior personalized pruning [41] [43] is limited to unstructured weight masking rather than removing entire components.Source: §2.2 · p. 4
• Existing FL pruning approaches often lack segmentation evaluation, impose impractical computational overhead for resource-constrained healthcare deployment, and have not explored integrating activation features with gradient-based scores.Source: §2.2 · p. 4
• FedASAP addresses these shortcomings with two-stage structured pruning for federated medical-image segmentation, reducing computational overhead and using activation statistics to identify redundant structures in a feature-informed manner.Source: §2.2 · p. 4

3 Methodology

• FedASAP uses a two-stage pruning pipeline: score-based screening followed by GLM-based final selection using multiple activation statistics.Figure 1 depicts local training, adaptive pruning, and recovery training as the overall process.Source: §3 · p. 4, Figure 1 · p. 5

(FedASAP)

• Client data distributions can differ, making a single generalized global model difficult to train; the stated aim is personalized lightweight models that leverage shared federated knowledge.The passage gives demographic and scanner differences as examples of client variation.Source: (FedASAP) · p. 5

3.1.1 Training flow with pruning

• Clients adapt personalized architectures by applying accumulated pruning decisions, removing selected filters and dependent parameters while retaining architectural consistency.The server transmits a client-specific reduced model to lower communication cost.Source: §3.1.1 · p. 5

3.1.2 Recovery training post pruning

• After pruning, clients fine-tune the reduced model during the latter half of the epochs to adapt remaining weights and recover performance lost through filter removal.This recovery phase follows the pruning step within each round.Source: §3.1.2 · p. 6

3.1.3 Model aggregation

• The server reconstructs each client model using its pruned-filter indices, aligns parameter indices, and aggregates the non-masked weights.Zero-masking is used during aggregation to preserve index alignment.Source: §3.1.3 · p. 6

3.2 Adaptive Pruning strategy - Stage I: Score-based Initial Screening

• Clients use a first-order Taylor expansion of the loss to score filters, then flag the lowest-scoring fraction as pruning candidates according to a predefined percentage.The pruning percentage is determined empirically based on performance and parameter conservation.Source: §3.2 · p. 6

3.3 Adaptive Pruning strategy - Stage II: GLM-based Final Selection

• The client-specific GLM combines complementary filter attributes to decide which candidate filters to prune or retain, preserving filters with distinctive activation patterns.This stage is intended to provide a more relevant compression decision than a single heuristic score.Source: §3.3 · p. 6

3.3.1 Generalized Linear Model formulation

• The GLM is selected for computational efficiency and interpretability, and its link-transformed linear predictor yields a pruning probability compared with threshold τ.Filters below τ are pruned; those above it are retained.Source: §3.3.1 · pp. 6–7

3.3.2 Feature design via multi-component filter profiling

• Each filter is represented by seven activation features—L2 norm, mean, entropy, DoG mean, DoG entropy, skewness, and kurtosis—that form a signature of its behavior.The low-dimensional feature space is described as having negligible computational overhead and supporting client adaptation even with limited samples.Source: §3.3.1 · p. 7, §3.3.2 · p. 8

3.3.3 GLM-guided final pruning

• After a no-pruning warm-up, the GLM predicts keep probabilities for candidate filters; pruning also removes their dependent channels and normalization parameters.The pruning aggressiveness setting controls the decision, with higher values producing more aggressive pruning.Source: §3.3.3 · p. 8
• Sampling filters and computing identical features yields a balanced dataset for training.Source: §3.3.3 · p. 8

4.1 Dataset Details

• WMH and FeTS are multi-centric FLAIR MRI datasets, with only 3–12 training examples per client.WMH spans five scanners; the datasets represent privacy-sensitive, low-data federated settings.Source: §4.1 · p. 9

4.2 Implementation Details

• Experiments use a U-Net with four encoder and decoder blocks and 16 initial features, trained for 100 rounds with batch size 4, N = 10, and p = 1.For GLM prediction, τ is 0.8 for WMH and 0.9 for FeTS.Source: §4.2 · p. 9

4.3.1 Comparison with the state-of-the-art methods

• The comparison includes FedProx for non-IID settings, FedBN [14], MOON, FedAvg [3], centralized training, and pruning approaches PruneFL and AutoFLIP.Pruning baselines use target sparsities of 73.0% for WMH and 45.6% for FeTS.Source: §4.3.1 · pp. 9–10

4.3.2 Ablation study: Effect of two-stage adaptive pruning in FedASAP

• Ablations compare FedASAP with FedAvg + SBIS and replace GLM final pruning with lesion-informed layer-adaptive thresholding (LAT).LAT varies pruning by layer based on lesion size and the differing fine-grained versus structural features captured across network depth.Source: §4.3.2 · p. 10

4.3.3 Effect of pruning on computational efficiency

• The efficiency evaluation compares explicit structural pruning with unpruned dense and unstructured-sparse full-sized architectures, measuring resource use including FLOPs.Source: §4.3.3 · p. 10
• Efficiency is assessed by forward-pass and per-sample inference time and peak inference memory on CPU for WMH and FeTS resource-constrained deployment scenarios.Source: §4.3.3 · p. 10

4.3.4 Extensibility to deeper architectures

• FedASAP, including SBIS and GBFS, is adapted from the standard 4-layer U-Net to a deeper 5-layer U-Net in federated settings.Source: §4.3.4 · p. 10

4.3.5 Effect of the pruning parameter p

• The study varies SBIS pruning percentage p across [0.5, 1, 2] and measures model-size reduction during training and segmentation accuracy for WMH and FeTS clients.Source: §4.3.5 · pp. 10–11

5.1 Comparison with the state-of-the-art methods

• FedASAP transmits 2 KB of pruning-related information per client, reducing communication overhead by a factor of 108 without extra downloads.Clients send pruned-filter indices once per filter; standard model-weight updates remain separate.Source: §5.1 · p. 11

5.2 Ablation study: Effect of two-stage adaptive pruning in FedASAP

• Compared with the best ablation variant, FedASAP adds 1.1% Dice and 9.3% LL-F1 on FeTS, and 1.5% Dice and 2.8% LL-F1 on WMH.Source: §5.2 · p. 11

5.3 Effect of pruning on computational efficiency

• Pruning reduces inference time per sample by 19.0% on WMH (1.287s to 1.043s) and 13.4% on FeTS (1.270s to 1.100s).The figure compares inference time, peak memory, and computational load for pruned and unpruned models; computational load falls by 50% on WMH and 32.2% on FeTS.Source: §5.3 · pp. 11, 13, Figure 3 · p. 13

5.4 Extensibility to deeper architectures

• Adding GBFS after SBIS further increases DSC and LL-F1 to 0.746 and 0.603, respectively (+4.9% and +7.5% relative to SBIS), with both improvements statistically significant (p < 0.001).The GBFS stage ultimately outperforms FedAvg by 2.0% in DSC and 5.3% in LL-F1, with 58.6% of parameters pruned.Source: §5.4 · p. 13

5.5 Effect of the pruning parameter p

• Higher pruning percentages accelerate model-size reduction but reduce segmentation accuracy, while lower values preserve performance at the cost of limited compression.The reported analysis suggests intermediate p values offer a favorable compactness–accuracy trade-off on both WMH and FeTS.Source: §5.5 · p. 13

• FedASAP consistently outperforms unstructured federated-pruning baselines FedDST, AutoFLIP, and PruneFL.The discussion attributes the comparison to the effectiveness of two-stage fine-grained pruning in balancing segmentation performance and computational efficiency.Source: §6 · p. 15
• Incremental pruning requires no preset sparsity level and progressively reduces model size, helping maintain stable gradients and avoid sudden loss spikes.Weights adapt as sparsity increases.Source: §6 · p. 15
• Activation-based statistics help prioritize important filters explainably, while client-side pruning tailors models to each client's data distribution.The discussion presents this as useful when lesion characteristics are not known in advance.Source: §6 · p. 15
• Progressive structured pruning reduces computational overhead during training, whereas unstructured sparsity alone does not yield practical runtime or memory reductions on standard hardware.The discussion points to Figure 3 for this hardware/software distinction.Source: §6 · p. 15
• Communication overhead is modest relative to PruneFL, FedDST, and AutoFLIP, with upload cost scaling with filter count rather than client count or rounds; early-round costs remain front-loaded.The model remains large early in progressive pruning, until pruning substantially reduces its footprint.Source: §6 · p. 15
• For WMH, NLAT yielded 2.9M versus PLAT's 1.3M parameters despite similar performance, while NLAT outperformed PLAT on large-lesion FeTS.The discussion links small-lesion segmentation to retaining fine-detail filters in shallow layers and large-lesion segmentation to preserving coarse structural features in central layers.Source: §6 · p. 15
• FedASAP improves accuracy and reduces runtime and memory costs in the presence of lesion heterogeneity.The discussion also reports tangible runtime and memory benefits.Source: §6 · pp. 15–16
• Performance degrades in pooled WMH–FeTS client setups, likely because domain shift and mixed lesion attributes affect GLM filter-retention decisions; future work proposes additional radiomics features and multimodal settings.The suggested multimodal extension includes FLAIR and T1.Source: §6 · p. 16

7 Conclusions

• FedASAP is a two-stage, fine-grained structured adaptive pruning framework for personalized federated medical image segmentation.It is presented as optimizing segmentation accuracy and model compactness.Source: §7 · p. 16
• Parameter reductions reach up to 73.0% while FedASAP shows superior segmentation performance to existing federated learning approaches.The conclusion also reports reduced model size.Source: §7 · p. 16
• FedASAP requires minimal communication overhead and reports gains in training time, memory usage, and computational load over unstructured approaches.Source: §7 · p. 16
• The authors describe FedASAP as suitable for resource-constrained medical institutions and integrable with standard federated learning frameworks.Source: §7 · p. 16

Declaration of competing interest

• The authors report no known competing financial interests or personal relationships and disclose a pending patent on two-stage adaptive structured pruning in federated medical image segmentation.The patent is assigned to the Indian Institute of Science, Bangalore, India.Source: Declaration of competing interest · p. 16

Ethics statement

• The FeTS dataset includes multi-institutional data.Source: Ethics statement · p. 16
• Subjects gave informed consent at their respective institutions, and the data-contributing institution's review board approved the data-release protocol.Source: Ethics statement · p. 16
• The statement cites Amsterdam UMC and UMCU approval 14-083/C and says no additional ethics approval was required because both datasets are publicly available.It identifies WMH as part of the MICCAI 2017 WMH segmentation challenge.Source: Ethics statement · p. 16

writing process

• The authors say all manuscript content, including analysis, interpretations, and conclusions, resulted solely from their research and intellectual effort.Source: writing process · p. 16