Source-linked AI summary
FairMean: Promoting Fairness in Distributed Learning under Label Poisoning Attacks
Huigan Zheng, Jiaojiao Zhang, Yongxiang Liu
TL;DR
Label poisoning can create large losses that complicate fairness-aware prioritization, and the behavior of fairness methods under protocol-following poisoned updates remains unclear. FairMean applies bounded, nondecreasing loss-based weights to client gradients; the paper reports improved fairness and worst-client accuracy in experiments.
Problem
Label poisoning can create large losses, so prioritizing high-loss clients for fairness may also amplify poisoned gradients; the behavior of fairness methods when poisoned clients follow protocol remains unclear.
Source: §1 · p. 1Method
FairMean scales client gradients by a nondecreasing marginal weight of local loss that is bounded by 1 + κ.
Source: §3 · pp. 2–3Results
FairMean reports improved fairness and worst-client accuracy under both attacks, with competitive average accuracy.
Source: §1 · p. 2Takeaways & Limitations
Without label poisoning, minimizing the FairMean objective is more conducive to solution fairness than minimizing the standard average-loss objective.
Source: §4 · p. 3The stated theory assumes cross-entropy local losses, bounded logit Jacobians, and a bound on poisoned-client gradient deviation from the regular-client objective gradient.
Source: §4 · p. 3
Abstract
from arXiv · showhide
Fairness-aware distributed learning prioritizes clients with large losses to reduce performance disparities, but label poisoning can create large losses, thereby inducing a fairness--robustness conflict. We propose FairMean to manage this conflict. FairMean weights client gradients using a bounded, nondecreasing function of local loss. The increasing weights prioritize high-loss clients to promote fairness, while the upper bound prevents excessive loss-induced amplification of poisoned-client gradients. In the absence of label poisoning, we show that minimizing the FairMean objective is more conducive to solution fairness than minimizing the standard average-loss objective. Under label poisoning, we establish an average-stationarity bound whose attack-dependent term is proportional to the square of the poisoned-client fraction. Experiments show that FairMean promotes fairness by reducing accuracy variance while improving worst-client accuracy.
1. INTRODUCTION
Heterogeneous client data can leave minority clients performing poorly, while label poisoning creates large losses that complicate loss-based fairness prioritization. FairMean uses bounded loss-dependent weighting to pursue fairness while limiting poisoned-gradient amplification.
1. INTRODUCTION
- Label poisoning changes client labels while preserving features and protocol, yet can produce misleading updates and degrade the global model on clean data.Robust screening can be ineffective under heterogeneous data, where it may discard useful regular-client updates.Source: §1 · p. 1
- Mean aggregation may retain useful heterogeneous updates, but equal gradient weights do not prioritize high-loss clients and unbounded scaling can amplify poisoned gradients.The resulting question is how to prioritize high-loss regular clients without excessive scaling of poisoned-client gradients.Source: §1 · p. 1
- FairMean uses a nondecreasing, uniformly bounded marginal weight to prioritize large-loss clients while limiting extra loss-induced scaling of poisoned-client gradients.The paper also gives a clean-case global-optimality guarantee for its loss-dispersion measure.Source: §1 · p. 2
- Experiments on Fashion-MNIST and CIFAR-10 report improved fairness and worst-client accuracy under both attacks, with competitive average accuracy.The introduction also situates fairness-aware approaches in loss transformation and reweighting, as well as multi-objective optimization.Source: §1 · pp. 1–2
2. PROBLEM FORMULATION
The formulation separates regular and poisoned clients and evaluates fairness over regular clients under label poisoning. Regular-client loss dispersion serves as a training-time proxy for performance disparity.
2. PROBLEM FORMULATION
- The model has n clients partitioned into fixed but unknown regular and poisoned sets, with at least one regular client and poisoned-client fraction δ = p/n < 1.The nominal objective is defined over regular clients.Source: §2 · p. 2
- A poisoned client relabels an arbitrary subset of its data but follows the prescribed protocol rather than transmitting an arbitrary message.The resulting update changes through relabeling, which can still yield a large loss and misleading gradient.Source: §2 · p. 2
- Fairness is assessed over regular clients, with their loss dispersion used during training as a proxy for performance disparity.Larger regular-client losses can receive larger marginal weights, while poisoned clients may also have large losses.Source: §2 · p. 2
3. PROPOSED FAIRMEAN
FairMean transforms local losses and scales client gradients by a nondecreasing, bounded marginal weight. This prioritizes high-loss clients while limiting extra loss-induced scaling, and the transformed objective penalizes regular-client loss dispersion.
3. PROPOSED FAIRMEAN
- FairMean retains mean aggregation and applies a bounded, nondecreasing marginal weight to prioritize large losses while limiting poisoned-gradient amplification.The design requirements also include Lipschitz continuity to avoid abrupt weight changes.Source: §3 · p. 2
- The transformation is ϕκ,τ(z) = z + κ[z − τ log(1 + z/τ)], with κ, τ > 0.Its marginal weight multiplies the local loss gradient; it is nondecreasing, bounded by 1 + κ, and κ/τ-Lipschitz continuous.Source: §3 · pp. 2–3
- The marginal-weight cap limits extra loss-induced scaling, while larger κ or smaller τ makes prioritization of high-loss clients more aggressive.The parameter τ shifts the loss at which the marginal weight reaches its midpoint.Source: §3 · pp. 2–3
- The transformed regular-client objective separates mean loss from a nonnegative dispersion term that vanishes when all regular-client losses are equal.The Taylor-expansion result links the transformation to an explicit penalty on squared deviations of regular-client losses from their mean.Source: §3 · p. 3
- FairMean applies the transformed loss to every client because the server does not know which clients are regular or poisoned.Each client computes its loss and gradient, scales the gradient by its marginal weight, and sends the resulting vector for server averaging.Source: §3 · p. 3
4. THEORETICAL ANALYSIS
The theoretical analysis compares FairMean’s clean-case fairness with average-loss minimization and bounds its optimization under label poisoning. The guarantees depend on stated assumptions and account for the poisoned-client fraction.
Clean-case fairness
- Without poisoning, a global minimizer of the FairMean objective has no larger Dτ than a global minimizer of the original regular-client objective, supporting fairness at global optimality.The section also states that minimizing FairMean’s objective is more conducive to solution fairness in the clean case.Source: §4 · p. 3
Assumptions and cross-entropy
- For multiclass cross-entropy, the analysis establishes Assumption 2 under bounded logit Jacobians and a bound on poisoned clients’ gradient deviation from the regular-client objective.The lemma also assumes a bound on the largest fraction of relabeled samples at any poisoned client.Source: §4 · p. 3
Average-stationarity guarantee
- The attack contribution to the stationarity bound contains an explicit factor δ^2, where δ is the poisoned-client fraction.The analysis treats FairMean as inexact gradient descent with respect to its transformed regular-client objective.Source: §4 · pp. 3–4
- Under Assumptions 1–2, FairMean’s optimization term in the average-stationarity bound decays as O(1/T).Increasing κ tightens the theoretical stepsize bound, while increasing τ relaxes it; more aggressive high-loss prioritization requires a more conservative stepsize.Source: §4 · p. 4
5. EXPERIMENTS
Across four dataset–attack settings, FairMean achieves the lowest accuracy variance and highest worst-client accuracy while retaining competitive average accuracy. The experiments also vary the poisoned-client fraction on Fashion-MNIST under pairwise flips.
Experimental setup
- The experiments use 10 clients, Dirichlet partitions, and two poisoned clients in the main experiments, with random and pairwise label flips.Fashion-MNIST uses an MLP and concentration 0.5; CIFAR-10 uses a 5-layer CNN and concentration 0.1.Source: §5 · p. 4
Overall results
- FairMean achieves the lowest accuracy variance and highest worst-client accuracy in all four dataset–attack settings while retaining competitive average accuracy.On Fashion-MNIST under pairwise flip, Figure 2 shows the fairness advantage as the poisoned-client fraction varies from 0 to 0.3.Source: §5 · p. 4, Figure 2 · p. 4
Poisoned-client fraction
- At δ = 0.3, FairMean reduces accuracy variance from 91.99 to 75.97 and raises worst-client accuracy from 56.19% to 60.85% relative to q-FFL.These comparisons are for Fashion-MNIST under pairwise flip.Source: §5 · p. 4
Future work
- Future work will extend FairMean to stochastic client participation and develop mechanisms accounting for benign data heterogeneity when mitigating label poisoning.Source: §5 · p. 4
A. ADDITIONAL EXPERIMENTAL RESULTS
On CIFAR-10, FairMean is competitive without attacks and leads on fairness metrics under both random and pairwise flips.
Additional CIFAR-10 results
- Under random and pairwise flips, FairMean achieves the lowest accuracy variance and highest worst-client accuracy while maintaining competitive average accuracy on CIFAR-10.It also remains competitive without attacks.Source: A. ADDITIONAL EXPERIMENTAL RESULTS · p. 6
B. SUPPORTING PROOFS
The supporting proofs bound the effect of poisoned labels on client gradients and use that control in an average-stationarity argument. They also establish smoothness and descent properties for the transformed objective.
B. SUPPORTING PROOFS
- The label-poisoning proof bounds the gradient change from altered labels and averages the per-sample bound over client data.The derivation uses the softmax cross-entropy gradient and notes that unchanged labels contribute zero.Source: B. SUPPORTING PROOFS · p. 6
- The server-direction proof treats the no-poisoning case separately and combines regular- and poisoned-client contributions to control the update error.The proof states that the update error is zero when the poisoned-client fraction is zero.Source: B. SUPPORTING PROOFS · pp. 6–7
- The stationarity proof establishes smoothness of the transformed objective, applies descent to the update, and bounds accumulated error using the per-step error limit.The proof sums the descent inequality over iterations to obtain the stated bound.Source: B. SUPPORTING PROOFS · p. 7