PaperlayerView exact revision on arXiv

Source-linked AI summary

Collaborative Machine Learning with Incentive-Aware Model Rewards

Rachael Hwee Ling Sim, Yehong Zhang, Mun Choon Chan, Bryan Kian Hsiang Low

arXiv:2010.12797v1cs.LGcs.GTcs.MAstat.ML

TL;DR

Collaborative ML needs contribution-based incentives because parties incur data-collection costs and may not share altruistically. The paper values contributions with information gain and Shapley value, distributes model rewards under adapted cooperative-game incentives, and realizes them through optimized Gaussian noise. Empirically, smaller ρ and more valuable model rewards are associated with higher predictive accuracy, although information gain does not always track MNLP.

  • Problem

    Parties incur nontrivial data-collection costs, motivating a fair incentive-aware scheme that rewards contributions without relying on monetary compensation.

  • Method

    The paper combines information gain and Shapley value for data valuation, adapts cooperative-game incentives to freely replicable model rewards, and realizes rewards with optimized Gaussian noise.

  • Results

    Smaller ρ and more valuable model rewards translate to higher predictive accuracy in the empirical evaluations.

  • Takeaways & Limitations

    The scheme can trade off fairness-related incentives while distributing only model rewards and maintaining fairness.

  • Takeaways & Limitations

    Information gain does not always translate to lower MNLP; MNLP improvement is negative around 15% of the time.

Abstract

from arXiv · show

Collaborative machine learning (ML) is an appealing paradigm to build high-quality ML models by training on the aggregated data from many parties. However, these parties are only willing to share their data when given enough incentives, such as a guaranteed fair reward based on their contributions. This motivates the need for measuring a party's contribution and designing an incentive-aware reward scheme accordingly. This paper proposes to value a party's reward based on Shapley value and information gain on model parameters given its data. Subsequently, we give each party a model as a reward. To formally incentivize the collaboration, we define some desirable properties (e.g., fairness and stability) which are inspired by cooperative game theory but adapted for our model reward that is uniquely freely replicable. Then, we propose a novel model reward scheme to satisfy fairness and trade off between the desirable properties via an adjustable parameter. The value of each party's model reward determined by our scheme is attained by injecting Gaussian noise to the aggregated training data with an optimized noise variance. We empirically demonstrate interesting properties of our scheme and evaluate its performance using synthetic and real-world datasets.

1. Introduction

The paper develops incentive-aware collaborative ML using model rewards rather than monetary compensation. It values data contributions, adapts cooperative-game incentives to freely replicable model rewards, and realizes rewards through optimized Gaussian noise.

  • Motivation: Parties share data when rewards provide guaranteed benefits and recognize more valuable contributions; the scheme therefore gives each party a separate ML model.The motivation includes applications where parties primarily seek improved model quality for future predictions.
  • Data valuation: The paper values data using information gain on model parameters, avoiding assumptions about current or future test-query distributions.More informative data is associated with greater reductions in parameter uncertainty.
  • Incentive design: The reward scheme adapts fairness, stability, individual rationality, and group welfare because model rewards are freely replicable.These incentives cannot all be provided simultaneously, so the scheme uses an adjustable parameter to trade them off while maintaining fairness.
  • Reward realization: Reward values are realized by injecting Gaussian noise into aggregated data and optimizing the noise variance rather than searching over discrete data subsets.The continuous optimization avoids an exponential number of subsets.
  • Evaluation: The paper empirically evaluates the scheme on synthetic and real-world datasets and reports properties of its model rewards.The introduction presents this as an empirical demonstration of the proposed scheme.
  • Novelty: The authors claim their scheme is the first collaborative ML approach to formally consider incentives beyond fairness using only model rewards.Existing cited work considered fairness and used monetary compensation when needed.

2. Problem Formulation

The problem is to design a collaborative ML scheme in which a trusted central party aggregates parties’ data and distributes model rewards whose values encourage collaboration. Unlike monetary-value sharing, freely replicable models can make total rewards exceed the aggregated-data value.

  • Setting: The setting assumes n honest, nonmalicious parties and a trusted central party that aggregates data, measures value, and distributes trained models.Parties own data and may organize into coalitions.
  • Notation: For coalition C, the aggregated data has value v_C, while r_i denotes the value of party i’s received model reward.The notation distinguishes coalition data values from individual model-reward values.
  • Objective: The central party must decide and realize model-reward values that satisfy incentives such as fairness and stability.The objective is explicitly incentive-aware rather than merely value partitioning.
  • Challenge: Unlike monetary rewards, freely replicable model rewards permit the sum of received rewards to exceed v_N.This difference prevents direct application of standard cooperative-game-theory value partitioning.

3. Data Valuation with Information Gain

The paper values coalition data by information gain on model parameters, interpreting larger uncertainty reduction as more valuable training data. This avoids dependence on a mutually agreed validation dataset but relies on assumptions for some formal properties.

  • Motivation: Existing valuation methods use validation accuracy, but selecting a validation dataset acceptable to all parties can be tedious or impossible.Different parties may have different data characteristics and preferred test-query demographics.
  • Information gain: Information gain measures the reduction from prior entropy H(θ) to posterior entropy H(θ|D_C) after training on coalition data D_C.The paper uses this information-theoretic quantity as a surrogate for trained-model quality.
  • Predictive interpretation: Reducing uncertainty in θ places greater posterior weight on plausible model parameters, improving predictive accuracy in expectation.The predictive distribution averages over model parameters weighted by p(θ|D_C).
  • Properties: Information gain assigns zero value to empty data, nonnegative value to every coalition, and monotonic value when parties are added.These are stated properties of the data-value function.
  • Scope: Validation accuracy remains preferred if all parties can agree on a common validation dataset.The information-gain approach addresses settings where that agreement is unavailable.
  • Assumptions: The submodularity property assumes conditional independence of parties’ data given θ.The paper attributes other listed properties to the information-never-hurts entropy bound.

4. Incentive-Aware Reward Scheme with Model Rewards

The paper defines incentive conditions for freely replicable model rewards and develops a parameterized Shapley-based scheme that balances fairness with rationality, stability, and welfare. It realizes the resulting reward values by injecting optimized Gaussian noise into aggregated data.

  • Model rewards are evaluated through feasibility, individual rationality, and fairness-related conditions adapted from cooperative game theory.Because model rewards are freely replicable, the paper modifies standard cooperative-game requirements for this setting.
  • Fairness: Directly rewarding individual data value or grand-coalition marginal loss can violate strict desirability and strict monotonicity because these schemes ignore other parties’ data.The paper gives these schemes as examples whose rewards do not change appropriately when marginal contributions to other coalitions change.
  • Fairness: Shapley fairness assigns rewards proportional to Shapley values, linking each party’s reward to its expected marginal contribution across coalitions.The paper argues that Shapley values capture marginal contribution and shows that purely Shapley-fair rewards satisfy R5.
  • Reward Scheme: ρ ≤ ρr ensures individual rationality, while ρ ≤ ρs ensures grand-coalition stability and individual rationality; ρ = 0 maximizes group welfare but loses fairness.The theorem defines ρr and ρs from individual and coalition values, respectively.
  • Reward Realization: The scheme realizes model-reward values by injecting Gaussian noise into aggregated data and optimizing the continuous noise variance.The paper states that the added noise affects reward-parameter variance and predictive distributions, with predictive-accuracy effects evaluated later.

5. Experiments and Discussion

Experiments on synthetic and real-world datasets show that the reward scheme preserves fairness while allowing the adjustable parameter ρ to trade off incentive properties. Collaboration usually improves predictive performance, though the relationship between information gain and MNLP has limitations.

  • Experimental settings: The experiments evaluate the reward scheme with GP, sparse GP, and neural-network regression on Friedman, DiaP, and CaliH datasets.The settings include three-party collaborations and a ten-party Friedman experiment.
  • Reward properties: In the Friedman GP experiment, larger Shapley values always yield higher-valued model rewards, while decreasing ρ increases rewards for parties with smaller Shapley values.At ρ = 0, all parties receive equally valuable rewards; at ρs, stability is reached for party 3’s reward.
  • Experimental settings: Partitioning partially sorts data by a randomly selected feature before dividing it into consecutive blocks, allowing differing quantities of unique or overlapping data.Each party is constrained to own a minimum share of the dataset in the illustrated partitions.
  • Predictive performance: The improvement in MNLP is usually positive, so collaboration generally benefits predictive performance, and larger φi is associated with more valuable rewards and lower MNLP.This pattern can fail when maximal MNLP improvement is very small or negative.
  • Reward properties: When ρ decreases from 1 to 0.5, points move closer to the diagonal, indicating more valuable rewards for parties with smaller φi and more cases satisfying R4.The scheme therefore trades off proportionality in Shapley fairness against stability and group welfare.
  • Limitations: A limitation is that higher information gain does not always translate to lower MNLP: MNLP improvement is negative around 15% of the time in some experiments.The scheme works best with a suitable model and a prior that is not sufficiently informative for any party to achieve high predictive accuracy alone.

6. Conclusion

The paper values data using information gain and Shapley value, distributes model rewards, and proposes a fairness-preserving scheme with an adjustable trade-off parameter. Experiments report that smaller ρ and more valuable model rewards correspond to higher predictive accuracy.

  • Data valuation uses information gain on model parameters, while each party’s marginal contribution is computed with the Shapley value.
  • The scheme adapts fairness, Shapley fairness, stability, individual rationality, and group welfare to freely replicable model rewards.
  • An adjustable parameter trades off among these incentives while maintaining fairness.
  • Smaller ρ and more valuable model rewards translate to higher predictive accuracy.

A. Proof of Submodularity of IG (1)

The information-gain value function is shown to be submodular under conditional independence of parties’ data given model parameters. The proof uses mutual-information identities and the fact that conditioning cannot increase entropy.

  • Conditional independence of Di and Dj given θ enables rewriting marginal information gain through conditional entropies.The proof equates mutual-information differences with entropy differences using symmetry and conditional independence.
  • More conditioning reduces the entropy term, yielding diminishing marginal information gain as coalitions grow.The key inequality invokes the information-never-hurts property for entropy.

B. Proof of Proposition 1

The proof establishes that Shapley-value rewards satisfy the stated fairness and monotonicity properties. It verifies symmetry, strict desirability, and strict monotonicity by comparing permutation-based marginal contributions.

  • Shapley-value rewards satisfy properties F1 through F4 for all parties.The proof explicitly states that ri = Shapleyv(i) satisfies each property.
  • F2 Symmetry: Symmetry follows by swapping two parties’ positions in a permutation and matching their corresponding coalition values.The proof considers whether either party precedes the other.
  • F3 Strict Desirability: Strict desirability follows when one party’s coalition value is everywhere at least as large and strictly larger for at least one coalition.The condition compares marginal values across all coalitions excluding the two parties.
  • F4 Strict Monotonicity: Strict monotonicity compares two coalition-value systems and derives corresponding inequalities for each party’s model reward.The argument uses that the predecessor coalition Sπ,i excludes party i.

C. Enforcing Shapley Fairness may not satisfy Individual Rationality (R4) for Submodular Value of Data

For submodular data value, enforcing Shapley fairness can violate individual rationality R4. The conflict arises because Shapley values may not cover each party’s standalone data value under the reward scaling required by other properties.

  • Submodularity implies that each party’s standalone data value vi is at least its Shapley value φi.The marginal value from the empty coalition bounds every coalition marginal, and Shapley efficiency gives Σi∈N φi = vN.
  • Satisfying R3 and R5 requires setting k = vN/φ∗, where φ∗ is the largest Shapley value.Under this definition, the party with φ∗ receives the most valuable reward, valued at vN.
  • R4 may fail when vi/φi exceeds vN/φ∗, because the resulting reward ri is smaller than vi.Although vN/φ∗ ≥ 1, submodularity gives vi ≮ φi, so the sufficient condition vi/φi < 1 does not hold.

D. Proof of Theorem 1

The proof establishes that the model-reward values inherit non-negativity, feasibility, weak efficiency, fairness, rationality, and stability properties from the Shapley-based construction. It also shows how the adjustable exponent ρ controls these reward guarantees.

  • Reward properties: The reward values are non-negative, feasible, and weakly efficient because they scale powered Shapley values by vN/φ*^ρ.The party with the largest Shapley value receives reward vN.
  • Fairness: For ρ > 0, the scheme is ρ-Shapley fair, with the common scaling factor k = vN/φ*^ρ.The same factor preserves the relevant Shapley comparisons across parties.
  • Incentives: The construction satisfies individual rationality and stability because each model reward is at least the party’s standalone and coalition-based data value.The proof states ri ≥ vi and ri ≥ vCi for every party.
  • Fairness: Symmetry, strict desirability, and strict monotonicity follow from corresponding Shapley-value properties and the common positive scaling factor.Equal or strictly ordered coalition contributions induce equal or strictly ordered rewards.
  • Reward realization: The reward scheme is realized by optimizing each party’s Gaussian-noise variance with the TOMS Algorithm 748 root-finding method.The variance parameter ηi is optimized by finding a root of the information-gain equation.

E.1. Gaussian Process (GP) Regression with Synthetic Friedman Dataset

The synthetic Friedman experiment evaluates the information-gain-based reward scheme with Gaussian-process and Bayesian-linear-regression components across varied data partitions and noise settings. The setup includes optimized party-specific noise and repeated test evaluation.

  • Synthetic dataset: The Friedman function uses six independent uniform input features, with the sixth feature irrelevant to the output.Outputs are standardized and modeled with a squared-exponential-kernel GP.
  • Reward realization: For each tested ρ, the study draws 20 Gaussian noise samples using an optimized ηi while fixing the random seed for fair comparisons.The standardized noise samples remain identical across ρ values despite differing variances.
  • Experimental design: The experiment evaluates five random 80%-20% train-test splits and ten training-data partitions per split.This partitioning design is used for the synthetic Friedman evaluation.
  • Information gain: The information-gain calculations cover Bayesian linear regression and full-rank or sparse Gaussian-process models.Sparse GP calculations replace KXX with the inducing-point approximation K⊤UUKUX.
  • Information gain: Heteroscedastic observations are handled by weighting with the inverse diagonal noise-covariance matrix instead of dividing by a common σ2.Each diagonal entry represents the corresponding data point’s noise variance.

F.1. Additional Results for Synthetic Friedman and BosH Datasets

Additional Friedman and Boston-housing experiments show generally positive MNLP improvement from the reward scheme. Lower ρ values and relatively larger Shapley values yield more accurate model rewards.

  • Friedman dataset: The Friedman experiments usually produce positive MNLP improvement, with smaller ρ moving most points closer to the diagonal.For ρ = 0.5, parties with smaller φi can receive more valuable rewards with higher predictive accuracy.
  • BosH dataset: The BosH setup uses 506 houses with ten selected features, while three listed features are removed before standardization and partitioning.The first two neural-network layers are trained on 80% of the dataset.
  • BosH dataset: The BosH experiments likewise show that most model rewards achieve positive MNLP improvement across multiple three-party partitions.The results are reported for the Boston housing dataset with neural-network and Bayesian-linear-regression components.
  • Reward behavior: Smaller ρ or higher relative Shapley values produce more valuable rewards, reflected by lower MNLP and points closer to the identity line.The comparison applies to the BosH scatter plots for ρ = 1 and ρ = 0.5.

F.2. Empirical Analysis of Relationship between IG and MNLP

The experiments find that information gain often tracks MNLP but is not universally predictive. Its advantage is reduced dependence on a shared validation dataset, while unsuitable models or highly informative priors can break the relationship.

  • Limitations: IG improvements can fail to produce higher predictive accuracy because MNLP also depends on the scaled squared-error term, not only predictive variance.The paper therefore studies poor use cases where IG and MNLP diverge.
  • Limitations: With an unsuitable GP model, positive MNLP improvement occurs for 631 of 750 suitable-model points but only 388 of 750 unsuitable-model points.The unsuitable model’s squared error remains high relative to the variance in the data.
  • Limitations: An informative prior with mean near the true mean and variance 0.01 produces maximal MNLP improvement around 0.2, versus 1.0 under the comparison setting.The informative-prior setting also yields more negative MNLP improvements.
  • Scope: The paper concludes that IG-to-accuracy alignment is expected when the model is suitable and parties cannot already train highly accurate models independently.This is the collaboration regime assumed to make additional shared data beneficial.
  • Validation sensitivity: IG-based valuation avoids choosing a common validation dataset, whereas MNLP-based valuation is highly sensitive to which validation data are used.MNLP-based values decrease substantially when a party’s data differ from the test dataset.
  • Empirical relationship: For Friedman data, normalized IG-based data and Shapley values closely approximate MNLP-based values across different validation datasets on average.This supports IG as a surrogate for MNLP-based valuation in that setting.
Loading 2010.12797v1…