Source-linked AI summary
Measure Contribution of Participants in Federated Learning
Guan Wang, Charlie Xiaoqian Dang, Ziye Zhou
TL;DR
Federated machine learning needs fair participant contribution measures because parties collaborate without sharing raw data, while data quantity alone does not capture data quality. The paper develops grouped instance deletion for horizontal FML and grouped-feature Shapley values for vertical FML, reporting effective and privacy-preserving contribution measurements and a model-agnostic framework.
Problem
Federated learning requires fair contribution measures for credit allocation because data quantity alone may not reflect data quality or model contribution.
Method
The paper uses group instance deletion to measure horizontal-FML influence and Shapley values to measure grouped feature importance in vertical FML.
Results
The experiments indicate that the methods provide effective, fair, and reliable contribution measurements without disclosing participants’ data.
Takeaways & Limitations
The contribution measurement is model agnostic and could serve as a general framework for integration into federated-learning toolsets.
Abstract
from arXiv · showhide
Federated Machine Learning (FML) creates an ecosystem for multiple parties to collaborate on building models while protecting data privacy for the participants. A measure of the contribution for each party in FML enables fair credits allocation. In this paper we develop simple but powerful techniques to fairly calculate the contributions of multiple parties in FML, in the context of both horizontal FML and vertical FML. For Horizontal FML we use deletion method to calculate the grouped instance influence. For Vertical FML we use Shapley Values to calculate the grouped feature importance. Our methods open the door for research in model contribution and credit allocation in the context of federated machine learning.
I. INTRODUCTION
Federated learning lets parties jointly build models without transferring raw data, but fair participation requires measuring data quality and contribution beyond data quantity. The paper proposes contribution measures for both horizontal and vertical FML.
- Federated learning keeps data at each party while enabling collaborative model building through transferred model parameters or encrypted communication.
- Insurance companies and other entities can collaborate on models without disclosing their data and share the resulting machine-learning benefits.
- Fair credit and reward allocation requires measuring contribution because large quantities of data may not improve the model substantially.
- The paper measures horizontal-FML contribution through grouped instance deletion and vertical-FML contribution through grouped feature Shapley values.Horizontal contribution is based on prediction changes after retraining without one party’s instances; vertical contribution uses grouped feature importance.
- The paper organizes its approach around federated learning background, Federated Deletion, Federated Shap, experiments, and discussion.
II. FEDERATED LEARNING
Federated learning addresses privacy and data-sharing barriers by coordinating models across parties, with horizontal and vertical settings defined by different overlaps in instances and features. These settings support distinct collaboration patterns and applications.
- Federated learning avoids transmitting sensitive data by exchanging model parameters or using secure communication while parties train collaboratively.
- Federated learning emerged partly from efforts to train models on user devices while limiting bandwidth and centralized data transfer.
- Horizontal federated learning applies when parties share many features but few instances, as in the Gboard use case.
- Vertical federated learning applies when parties share many instances but few features, combining their feature spaces for machine-learning tasks.The insurance example involves overlapping users whose separate feature spaces are merged, with homomorphic encryption protecting privacy.
- Federated transfer learning targets settings with limited overlap in both features and instances.
- In insurance, horizontal FML models collaboration between primary insurers and reinsurers, whereas vertical FML links a reinsurer with an online retailer holding additional policyholder features.
III. DELETION METHOD FOR HORIZONTAL FML
For horizontal FML, the paper adapts deletion diagnostics to estimate each party’s contribution from the influence of its training instances. The method retrains models after removing party-held data and compares predictions.
- The method supports interpreting prediction changes on training or new data, for individual predictions or batches.
- The approach builds on deletion diagnostics and related influence-function methods for identifying influential training instances.
- Deletion diagnostics measure instance influence by retraining after omitting an instance and measuring the resulting prediction change.
- A party’s horizontal-FML contribution is defined as the total influence of the instances in its subset.
- An approximation algorithm processes a batch of instances together during each deletion to estimate party influence.
IV. SHAPLEY VALUES FOR VERTICAL FML
For vertical FML, the paper represents a party’s contribution as the combined importance of its shared features. It therefore extends feature-level contribution analysis to grouped feature sets.
- In vertical FML, each party contributes features, so its contribution is represented by the combined contributions of its shared features.
- The method first distributes contribution among individual features and then extends the measure to grouped features.
A. Shapley Values for Individual Feature
The paper explains individual feature influence through situational importance and Shapley values, then approximates Shapley estimation with Monte Carlo sampling for black-box models.
- Situational importance compares a feature’s contribution at value x_i with its expected contribution.
- For black-box models, feature influence is the difference between an instance’s prediction and its expected prediction without feature i.
- Shapley values assign each feature a number representing its influence while accounting for all feature sub-coalitions.
- Exact Shapley computation has exponential time complexity, making it infeasible for practical scenarios.
- Monte Carlo sampling approximates Shapley estimation by repeatedly sampling instances and feature permutations.
B. Shapley Values for Grouped Features
For vertical federated learning, the paper groups each party’s private features into a united federated feature and estimates its Shapley contribution without exposing individual features.
- Vertical federated learning divides the feature space across parties, making contribution measurement subject to privacy concerns.
- The united federated feature combines the features owned by one party and treats them as a single feature because individual features remain private.
- The Shapley group value sums individual Shapley values to represent a subset’s contribution.
- Under either stated assumption, Proposition 4 establishes the Shapley group value for a party’s feature set.
- The proposed vertical method estimates a party’s contribution by computing the Shapley value of its united federated feature alongside other parties’ features.
- The approach relies on strong assumptions, while experiments report that its approximation algorithm works well in real scenarios.
V. EXPERIMENT
The experiments evaluate participant-contribution methods on the Cervical cancer dataset in horizontal and vertical federated-learning setups, including encrypted party interaction for vertical estimation.
- The experiments use the Cervical cancer dataset to calculate participant contributions in horizontal and vertical federated-learning setups.
- The horizontal setup uses deletion to measure grouped instance importance across participants.
- The vertical setup uses Shapley values to evaluate feature importance for features shared by different participants.
- Algorithm 3 sends an encrypted instance ID or special ID to a party, sets a federated feature to a reference value when needed, and runs the federated prediction.
- The algorithm saves prediction results and computes the Shapley value for the federated feature as the party’s contribution measure.
A. Deletion Method (Horizontal FML)
The deletion method generalizes single-instance influence to party-level instance groups in horizontal FML by retraining without a party’s instances and measuring prediction changes.
- A. Deletion Method (Horizontal FML): Deletion diagnostics evaluate an instance’s influence by retraining after omission and measuring the resulting prediction change.The approach is generalized from individual instances to groups supplied by different parties.
- A. Deletion Method (Horizontal FML): Horizontal FML experiments used the Cervical dataset with binary Biopsy labels and an SVM classifier.The full dataset served as training data without a separate split.
- A. Deletion Method (Horizontal FML): Five simulated parties contributed equally sized instance groups, whose horizontal FML importance values were plotted in Fig. 1.The figure’s vertical axis represents the instance group importance value.
B. Shapley Value (Vertical FML)
For vertical FML, the paper applies Shapley values to feature importance, first for individual predictions and then for grouped features representing multiple participants.
- B. Shapley Value (Vertical FML): The vertical FML experiment used the Cervical cancer dataset with 70% training data and 30% testing data, reaching 95.42% test accuracy.The algorithm setup matched the preceding experiment.
- B. Shapley Value (Vertical FML): A single training instance was predicted to demonstrate Shapley-based feature importance for an individual prediction.The result is presented in Fig. 2 as a feature-importance explanation.
- B. Shapley Value (Vertical FML): Whole-feature-space Shapley values directly reflect the importance of different features across predictions.Fig. 3 presents scatter plots for predictions and bar plots for total feature contributions.
- B. Shapley Value (Vertical FML): In the multi-party setup, 15 features were divided into five participant groups of three features each for federated Shapley evaluation.The grouped features were evaluated together with individual features from other participants.
- B. Shapley Value (Vertical FML): The experiments report federated Shapley value as a good quantity for indicating each participant’s contribution in multi-party vertical FML.The paper contrasts grouped evaluation with calculating all federated features together, which it expects to be less accurate.
VI. CONCLUSION
The paper concludes that grouped instance deletion and grouped Shapley values can measure participant contributions in horizontal and vertical FML while preserving data privacy.
- VI. CONCLUSION: Grouped instance deletion and grouped Shapley values provide contribution measurements for participants in horizontal and vertical FML.The experiments indicate that these measurements are fair and reliable without disclosing participant data.
- VI. CONCLUSION: The contribution-measurement work is model agnostic and is intended to function across almost any machine-learning algorithm.The authors describe it as a potential general framework for the task.
- VI. CONCLUSION: The authors identify influential functions and sampling-based Shapley calculations as future directions for greater accuracy, fairness, and computational efficiency.These directions target horizontal and vertical FML respectively.
- VI. CONCLUSION: Average feature importance was evaluated for 855 instances after dividing 15 features into five groups of three.The grouped-feature analysis appears in the bar plot of Fig. 5.
- VI. CONCLUSION: Feature-importance scatter plots were also produced for 855 instances under the same five-group, three-features-per-group organization.The grouped-feature analysis appears in Fig. 4.