Source-linked AI summary
MUC-FL: Block-Wise Marginal Utility Contribution for Communication-Efficient Federated Learning
Akshay Mhatre, Vikram Karthick, Deepti Gupta, Jia Zou
TL;DR
Federated learning faces substantial communication overhead, while existing approaches do not distinguish routine from high-value data blocks. MUC selectively transmits blocks according to their contribution to model performance, with reported results showing lower communication and higher macro F1 than standard federated optimization.
Problem
Federated learning incurs high communication overhead, and existing methods may treat data blocks uniformly or require costly valuation procedures.
Method
MUC evaluates block-level contributions to model performance and selectively transmits impactful blocks while filtering redundant or low-impact updates.
Results
A macro F1 score of 0.8566 versus 0.8155 for standard federated optimization was achieved, with potential communication reduction of 45–50%.
Takeaways & Limitations
Selective block transmission can improve model performance, particularly for underrepresented classes, while reducing communication costs.
Abstract
from arXiv · showhide
Federated Learning (FL) enables distributed model training without centralizing data but suffers from high communication overhead. To address this, we propose Block-Wise Marginal Utility Contribution (MUC), a framework that selectively transmits only the most impactful data blocks based on their contribution to model performance. To evaluate our framework, we apply it to a multimodal dataset integrated from multiple MIMIC clinical datasets and show that only 24 out of 1,135 candidate blocks (1.76%) carry meaningful improvement signals, enabling a potential communication reduction of 45-50% while maintaining or improving model quality. Our deduplication-based block selection achieves a macro F1 score of 0.8566 compared to 0.8155 for standard federated optimization, demonstrating that selective transmission can improve performance, particularly in underrepresented classes.
I. INTRODUCTION
MUC addresses FL’s communication burden by selecting data blocks according to their contribution to model improvement. On integrated MIMIC clinical data, the framework reports sparse useful blocks, potential communication savings, and improved macro F1.
- MUC selectively transmits high-utility data blocks by quantifying each block’s contribution to model performance.
- The framework targets redundant updates because blocks contribute unequally across training iterations, with rare or underrepresented patterns potentially yielding larger gains.
- Only 24 of 1,135 candidate blocks, or 1.76%, are incorporated into the final optimized model after validation across five clients.
- A potential 45–50% communication reduction is reported while maintaining model quality.
- Macro F1 reaches 0.8566 versus 0.8155 for standard federated optimization, particularly benefiting underrepresented classes.
II. RELATED WORK
Prior FL communication-efficiency methods compress gradients or select clients, but they do not adequately distinguish valuable data blocks. These approaches therefore leave block-level utility and data quality insufficiently addressed.
- Gradient compression can achieve 90–99% compression by transmitting large gradients, but it may treat routine and rare-case blocks identically.
- Existing approaches motivate block-level selection because gradient magnitude and participant choice do not directly identify valuable data contributions.
- Client-selection methods operate at the participant level rather than selecting individual data blocks, limiting optimization granularity.
C. Data Valuation Methods
Data valuation methods seek to quantify contribution, but Shapley-based approaches become computationally impractical for large federated systems. MUC instead frames block-wise deduplication as a utility-aware selection mechanism.
- Shapley-based FL valuation captures participation-order effects but requires exponential 2^n or quadratic O(n^2) retraining.
- With hundreds of changing blocks across rounds, Shapley values are described as computationally unrealistic in federated settings.
- The research gap is the lack of scalable methods that distinguish high-value blocks from routine ones at block-level granularity.
- MUC addresses this gap by reconstructing an improved global model from the most useful client-side block updates.
B. Block-Wise Deduplication in Federated Learning
The framework treats the broadcast global model as a target and client checkpoints as donors, then constructs a hybrid by transferring only beneficial aligned blocks. Candidate ranking and forward sparse selection reduce unrestricted checkpoint mixing.
- The method constructs a hybrid global model by selectively transferring donor blocks that improve the target in the global evaluation context.
- The block-wise deduplication framework is presented as the core mechanism for reconstructing improved global models from selected client updates.
- Eligible parameter tensors are flattened into fixed-size blocks, so block counts reflect parameter segments rather than neural-network layers.
- A lightweight proxy ranks donor candidates before explicit evaluation because exhaustive multi-block combination testing is computationally prohibitive.
- Forward sparse selection incrementally retains a compact subset of donor blocks that provide useful round-specific signals.
D. Communication-Aware Interpretation
The framework extends block-wise deduplication into a communication-aware FL design that avoids unnecessary evaluation and retrieval when block changes are weak. It also supports selective triggering policies for client participation and aggregation.
- Selective block transmission lets the server retain G_t and reconstruct an improved hybrid model using only high-utility donor blocks.This avoids transmitting the full client update when round-specific utility is concentrated in a small subset of block substitutions.
- The broader design space combines block-wise deduplication with lightweight utility proxies, metadata exchange, and selective triggering before explicit evaluation.These mechanisms prioritize promising block positions while preserving deduplication as the core selection mechanism.
- Smart triggering skips a full deduplication pass when lightweight block-change statistics remain weak across most positions.The round can instead fall back to the standard aggregated model, avoiding unnecessary evaluation and retrieval overhead.
- Communication-aware deployment can trigger only selected clients based on bandwidth, resources, completion status, deadlines, or accumulated high-utility information.The supplied policies include dynamic client selection, asynchronous triggering, adaptive deadlines, condition-based local triggering, and incentive-based triggering.
G. Metadata Exchange and Leadership
The protocol exchanges compact block metadata to identify leaders, retrieves gradients from the strongest proposals, and evaluates utility changes before retaining or restoring updates. This creates a staged leadership and filtering process for block-wise deduplication.
- Clients transmit block identifiers and MUC scores so the server can select the maximum-MUC leader for each block position.The selection operates over client datasets partitioned into logical blocks.
- The server requests gradients from leader blocks and broadcasts high-MUC gradients to clients for further evaluation.This separates compact metadata exchange from full gradient transmission.
- Clients use gradient-norm proxies and top-k selection to identify changed blocks, then retain gradients with positive MUC changes and discard the rest.The procedure evaluates whether applying server gradients produces utility improvements.
- Tables I and II present performance comparisons for the MultimodalClassifier and LLaVA classifier branches.The supplied table labels identify the two classifier branches but do not provide cell values or a comparison outcome.
- Weak utility changes reported by multiple clients flag a leader as potentially poisonous and trigger restoration of previous gradients.This provides a safeguard against retaining harmful donor updates.
A. MUC Computation
MUC combines gradient magnitude, sample size, and average loss to estimate each block’s marginal utility. The resulting proxy correlates with actual utility while reducing evaluation complexity.
- MUC combines gradient norm, sample size, and average loss as complementary signals of block-level learning utility.The supplied passages identify these three components but do not include the displayed approximation equation itself.
- Gradient norm indicates learning potential, with high norms signaling under-fitting that may require larger updates.The norm is presented as a measure of gradient magnitude.
- Sample size scales contribution by the square root of batch size to reflect diminishing returns.This scaling follows the statistical rationale stated in the passage.
- Average loss weights the estimate by current model fit, so higher loss indicates greater marginal learning utility.The weighting emphasizes blocks on which the current model performs poorly.
- The proxy achieves high correlation with actual utility in O(n) rather than O(n^2), eliminating expensive retraining.The passage reports the complexity contrast and its stated computational consequence.
A. MIMIC Dataset and Federated Setup
The experiments use a linked multimodal MIMIC pipeline combining imaging, reports, and structured clinical information, with pathology labels defined for a 14-category multilabel task. Federated clients are formed through subject-level partitioning to prevent leakage.
- The dataset links chest radiographs, radiology reports, and structured clinical information into clinically aligned multimodal examples.Sources include MIMIC-CXR-JPG, MIMIC-CXR reports, and MIMIC-IV admission-level and patient-level tables.
- The pathology task uses 14 chest pathology categories with multiple labels potentially active for each study.Labels are restricted to cases where NegBio- and CheXpert-derived annotations agree.
- Subject-level partitioning keeps records from the same patient within one client, preventing subject leakage across federated partitions.The setup includes both even and uneven subject-level splits.
- The reported FL training corpus contains 222,554 subject-partitioned samples and uses a fixed held-out test set.The even split distributes approximately [44,614, 44,481, 45,074, 43,708, 44,677] samples across five clients.
B. Models Used in the FL Pipeline
The FL pipeline evaluates block-wise deduplication on classifier-stage multimodal models, using sparse donor-block selection to improve a weaker global checkpoint. The MultimodalClassifier benefits more clearly than the LLaVA branch, whose compressed hybrid does not surpass the next vanilla checkpoint.
- Model setup: The experiments operate on classifier-stage models built over cached multimodal embeddings rather than deduplicating a full end-to-end vision-language backbone.The LLaVA branch replaces conventional image and text representations with LLaVA-derived features while retaining demographic inputs.
- MultimodalClassifier: 24 of 227 eligible blocks were accepted into the MultimodalClassifier hybrid, which achieved macro F1 0.8549 and improved over the weaker base model across all reported metrics.The hybrid slightly exceeded vanilla FedYogi on micro F1 and achieved noticeably higher macro F1, but remained below it on macro AUROC.
- LLaVA classifier branch: The LLaVA hybrid retained 102 of 205 eligible blocks but did not improve upon the next vanilla global checkpoint.Its compression–utility tradeoff was therefore less favorable than the MultimodalClassifier result.
A. Communication-Savings Interpretation
Communication savings are estimated structurally from the number of retained blocks rather than measured directly over a network. The reported conclusion is that compression is useful only when the selected blocks preserve model utility.
- Block-count estimates: 24 of 227 MultimodalClassifier blocks imply an estimated 89.43% block-count payload reduction, while retaining 102 of 205 LLaVA blocks implies 50.24%.The block-count proxy reports 10.57% of positions retained for the MultimodalClassifier branch.
- Interpretation and limitation: These savings are structural communication proxies rather than direct network measurements, and compression is meaningful only when the retained subset preserves utility.The LLaVA configuration illustrates this boundary because its block reduction did not recover the stronger FedYogi round.
- Reported conclusion: The reported 86.2% communication reduction and 69.1% computation savings were accompanied by a 4.4% accuracy improvement over FedAvg on MIMIC-IV clinical data.The conclusion also reports a gradient norm proxy correlation of 0.89 with ground-truth utility.