Source-linked AI summary
FedPCD: Priority-Constrained Federated Aggregation
Yousef Irshaid, Dara Varam, Mohamed I. AlHajri
TL;DR
Federated learning can yield uneven client performance under heterogeneous data, while fairness-aware approaches may change the global update direction or treat client objectives symmetrically. FedPCD preserves the federated aggregate as primary and minimally corrects it to satisfy client constraints. Across evaluated benchmarks, it leads in global accuracy and usually in worst-10% client accuracy, while reaching FedAvg’s final accuracy in fewer rounds.
Problem
Under heterogeneous client data, federated aggregation can produce uneven client performance, and existing fairness methods may modify the global direction or treat client objectives symmetrically.
Source: §1 · pp. 1–2Method
FedPCD keeps FedAvg as the primary direction and applies only the minimum correction needed to satisfy client-level progress constraints.
Source: §1 · p. 2Results
FedPCD achieves the highest global accuracy in all tested cases, better worst-10% client accuracy in 17 of 18 configurations, and reaches FedAvg’s final accuracy in 10%–44% of rounds.
Source: §1 · p. 2, §6 · p. 14Takeaways & Limitations
Client-level protection can accompany preservation of the aggregate objective, with competitive client-performance spread and strong tail-client performance in the evaluated settings.
Source: §1 · p. 2Evaluation is limited to 100 clients and one local epoch, so behavior with many local steps, where pseudo-gradient directions may degrade through client drift, is not established.
Source: §6 · p. 14
Abstract
from arXiv · showhide
Federated learning (FL) seeks to optimize a shared global model from decentralized client data. While effective for average performance, this process can produce unfair performance across clients, particularly under statistically heterogeneous data distributions. Existing fairness-aware and multi-objective FL methods address this through client re-weighting, gradient correction, or common-descent formulations, but often change the global optimization direction or treat client objectives symmetrically. We propose FedPCD, a priority-constrained federated optimization framework that separates the global optimization objective from client protection. FedPCD preserves the conventional federated aggregate as the primary objective, while allowing only the minimum correction necessary to meet client performance constraints. Across heterogeneous FL benchmarks, FedPCD consistently outperforms state-of-the-art baselines, achieving the highest global accuracy across all configurations and leading worst-10% client accuracy in nearly all. FedPCD further preserves competitive trade-offs between global accuracy and client performance variability. Beyond this trade-off, FedPCD demonstrates faster empirical convergence, reaching matching or superior performance in substantially fewer communication rounds.
1 Introduction
FedPCD keeps the conventional federated aggregate as the primary objective and uses client-level constraints to require only necessary corrections. The paper evaluates its performance and convergence across heterogeneous federated-learning settings and establishes conditional convergence guarantees.
1 Introduction
- Under heterogeneous client data, aggregate updates can help some clients while providing little, no, or adverse progress for others.Performance fairness seeks more uniform client-level performance while maintaining global-model utility.Source: §1 · p. 1
- FedPCD preserves FedAvg as the primary direction and applies the minimum correction needed to satisfy client-level progress constraints.If FedAvg already satisfies the constraints, the update remains unchanged; fairness improvement comes from client protection rather than an explicit fairness surrogate.Source: §1 · p. 2
- A client-history mechanism retains recently observed client directions to increase their representation under partial participation.Source: §1 · p. 2
- FedPCD achieves the highest global accuracy across evaluated settings, superior worst-10% client accuracy, and competitive client-performance spread versus fairness-aware baselines.The evaluation covers two participation levels, three datasets, and three heterogeneity levels.Source: §1 · p. 2
- FedPCD reaches FedAvg’s final accuracy using 10%–44% of communication rounds and beats the strongest baseline in convergence speed in 16 of 18 configurations.Source: §1 · p. 2
- Under deterministic full participation, FedPCD establishes an O(1/T) rate for the round-averaged squared global-gradient norm and simultaneous client stationarity under stated geometric conditions.Source: §1 · p. 2
2 Related Work
Related work addresses client-performance fairness through reweighting, conflict-aware aggregation, and multi-objective optimization. FedPCD contrasts with these approaches by preserving an asymmetric priority for the aggregate objective and treating client progress as constraints.
2 Related Work
- Fair federated aggregation is organized into reweighting client objectives, resolving gradient conflicts, and multi-objective optimization.Source: §2 · p. 2
2.1 Fair Federated Learning
- Fairness methods reweight client losses or modify parameter contributions, while conflict-aware methods adjust incompatible updates and may use history for absent clients.These approaches include worst-case or disadvantaged-client weighting, update projection, and historical information that trades broader client coverage against gradient staleness.Source: §2.1 · pp. 2–3
2.2 Multi-Objective Federated Optimization
- Multi-objective methods seek shared compromise directions, whereas FedPCD minimally projects the standard federated update to meet hard client-progress constraints.The section also distinguishes FedPCD from FedMC+, which separates average objectives from local conflict mitigation but does not use FedPCD’s minimum-progress constraint formulation.Source: §2.2 · p. 3
3 The Federated PCD Framework
The framework builds on weighted FedAvg and priority-constrained descent, using the federated aggregate as the preferred direction while requiring progress on client objectives. For partial participation, a history window retains recent client directions as additional constraints.
3.1.1 Federated Learning Setup
- FedAvg approximates the weighted-average client objective through local training and server aggregation of participating clients’ updates.Clients perform K local optimization steps before their updates are aggregated.Source: §3.1.1 · p. 4
3.1.2 Priority-Constrained Descent
- PCD solves a strictly convex QP that projects the primary direction onto the intersection of secondary-progress half-spaces; satisfied constraints do not alter the solution.The progress threshold controls how much progress is required and can increase deviation from the primary direction.Source: §3.1.2 · pp. 4–5
3.2 Priority-Constrained Federated Aggregation
- FedPCD keeps the weighted FedAvg aggregate as its preferred direction and uses normalized client directions as progress constraints, applying no correction when the aggregate already satisfies them.The resulting direction is rescaled using the norm of the current aggregate.Source: §3.2 · pp. 5–6
3.3 Partial Participation and Client History
- With partial participation, a history window retains each recently sampled client’s latest pseudo-gradient as a secondary constraint, removing it after H rounds.The primary direction remains the current-round aggregate; cached directions broaden client representation while creating a potential staleness trade-off.Source: §3.3 · p. 6
4 Theoretical Analysis
The analysis studies FedPCD under full client participation, deterministic local steps, and assumptions on smoothness and constrained-direction alignment. Under these conditions, it establishes an O(1/T) stationarity rate and a conditional composite multi-stationarity result.
Theoretical assumptions
- The analysis assumes full client participation, a common number of deterministic full-batch local steps, H = 0, L-smooth client objectives, and a global objective bounded below.It also assumes a feasible QP and a uniform positive alignment margin, with local drift accounted for when K > 1.Source: §4 · pp. 7–8
Theorem 4.1
- FedPCD has an O(1/T) deterministic nonconvex stationarity rate with fixed admissible local and server learning rates, including when K > 1.The theorem assumes positive effective alignment and the stated conditions on the constrained direction.Source: §4 · p. 8
Corollary 4.2
- Under Theorem 4.1’s conditions, an infinite run has every client gradient converging to zero and every accumulation point composite multi-stationary.CMS requires each client objective to be individually stationary, stronger than Pareto stationarity.Source: §4 · p. 8
Scope of guarantees
- The CMS conclusion is conditional on geometric conditions because arbitrary heterogeneous client objectives need not admit a common stationary point.CMS requires every client objective to be individually stationary.Source: §4 · p. 8
5 Experiments
Across three datasets, two participation rates, and three heterogeneity levels, FedPCD achieves the best global accuracy in all 18 tested configurations and leads worst-10% accuracy in 17. The experiments also compare client-accuracy spread and communication-round efficiency.
5.1 Experimental Setup
- Experiments use CIFAR-10, CIFAR-100, and Fashion-MNIST with Dirichlet-partitioned data at α ∈ {0.01, 0.1, 1} and 5% or 10% client participation.Models train for 500 rounds on the CIFAR datasets and 300 rounds on Fashion-MNIST; reported results average five seeds.Source: §5.1 · pp. 8–9
5.2.1 Global and Tail Accuracy
- FedPCD achieves the best global accuracy in all 18 configurations and the highest worst-10% accuracy in 17.Table 1 covers CIFAR-10, CIFAR-100, and Fashion-MNIST at 5% and 10% participation and α ∈ {0.01, 0.1, 1}.Source: §5.2.1 · p. 9, §5.2 · p. 9, Table 1 · p. 11
- At α = 0.01 and 5% participation on CIFAR-10, FedPCD reaches 71.90% global accuracy against 65.59% for FedLF, and its worst-10% accuracy exceeds FedLF’s by more than 14%.On CIFAR-100 at α = 1, FedPCD reaches 35.40% and 36.47% global accuracy at 5% and 10% participation, against 29.77% and 31.29% for CRAFT; Fashion-MNIST has the sole tail-accuracy exception, where FedLF reaches 65.73% against FedPCD’s best 63.22%.Source: §5.2.1 · pp. 9–10
5.2.2 Client-Level Fairness
- FedPCD is at least as accurate and even as all 10 baselines on CIFAR-10 and Fashion-MNIST at 5% participation, and at least 9 at 10%.At 10% participation on CIFAR-10 and Fashion-MNIST, FedLF has lower spread and higher worst-10% accuracy despite lower global accuracy; on CIFAR-100, FedPCD is more accurate but has higher spread than baselines.Source: §5.2.2 · p. 10, Figure 4 · p. 10, Figure 5 · p. 12
5.2.3 Convergence
- FedPCD reaches FedAvg’s final accuracy using 10%–44% of communication rounds and reaches the strongest baseline’s final accuracy earlier in 16 of 18 configurations.The convergence figures compare global accuracy over communication rounds at α = 0.01.Source: §5.2.3 · pp. 12–13, Figure 7 · p. 13
6 Conclusion
FedPCD preserves FedAvg as its primary descent direction and applies the minimum correction needed to satisfy client constraints, using client history to represent recently observed directions under partial participation. Across the tested benchmarks it leads in global accuracy and nearly all worst-10% client-accuracy comparisons, while the evaluation and history mechanism retain stated limitations.
6 Conclusion
- FedPCD retains FedAvg as the primary descent direction and applies only the minimum correction needed to satisfy client-level constraints.For partial participation, a client-history mechanism retains recently observed directions as secondary constraints.Source: §6 · p. 14
- The convergence analysis establishes an O(1/T) rate under deterministic full participation and shows client-level progress toward simultaneous stationarity under persistent feasibility.Source: §6 · p. 14
- The evaluation reports the highest global accuracy in all tested cases, better worst-10% client accuracy in 17 of 18 configurations, and convergence in significantly fewer communication rounds than 10 fairness-aware baselines.Experiments cover CIFAR-10, CIFAR-100, and FashionMNIST, with two participation levels and three heterogeneity levels.Source: §6 · p. 14
- The history mechanism requires the server to observe each client’s pseudo-gradient individually, limiting applicability to secure aggregation systems.The evaluation also covers only 100 clients and one local epoch; behavior under many local steps is not established.Source: §6 · p. 14
A Reproducibility Details
The reproducibility details specify model architectures and training choices by dataset, control experimental comparisons through paired seeds and shared conditions, and describe baseline selection and hyperparameter tuning.
A Reproducibility Details
- CIFAR-10 uses a three-layer CNN, CIFAR-100 a ResNet-18, and FashionMNIST an MLP, with dataset-specific batch sizes and learning rates.FashionMNIST uses full-batch local training; CIFAR-10 and CIFAR-100 use local batch sizes of 128 and 64, respectively.Source: A Reproducibility Details · p. 16
- Experiments use five paired seeds and match data partitions, initial models, participating-client sequences, minibatch ordering, and client test partitions across methods.Clients are sampled uniformly without replacement, and metrics use the model from the final communication round.Source: A Reproducibility Details · p. 16
- Baselines were selected for public implementation availability, and tunable hyperparameters were compared using the search spaces in Table 2.FedFV and FedLF were both evaluated at H = 10; best-performing settings were selected per dataset after averaging five seeds.Source: A Reproducibility Details · p. 16
B.1 Assumptions and Proof Overview
The proof overview analyzes a constrained update under full participation and requires feasible client constraints with a uniform alignment margin. This geometric condition excludes some client-direction configurations, including exactly opposing nonzero directions.
B.1 Assumptions and Proof Overview
- The proof first controls individual client gradients, since a small global gradient can result from cancellation even when individual gradients are not small.Source: §B.1 · p. 16
- Table 2 presents the hyperparameter search spaces.Source: Table 2 · p. 17
- The analysis covers full participation, deterministic local gradient descent, fixed K ≥ 1 local steps, and H = 0, with fixed positive learning rates and smooth objectives bounded below.It requires neither convexity nor a uniform bound on local gradients; stochastic gradients, partial participation, and historical directions are outside the theorem’s scope.Source: §B.1 · p. 17
- At each round with nonzero pseudo-gradients, the quadratic program must be feasible and satisfy Equation 13 with the same γ ∈ (0, 1] throughout the run.The passage characterizes γ as the resulting geometric margin, not a user-selected parameter.Source: §B.1 · p. 17
- Feasibility with τ > 0 gives positive alignment at a round but not a uniform lower bound; the separate alignment condition supplies that bound.Exactly opposing nonzero client directions cannot both satisfy positive progress constraints, so the theorem does not cover arbitrary heterogeneous objectives.Source: §B.1 · p. 17
B.2 Supporting Results
The supporting analysis relates client alignment and local drift to aggregate-update magnitude, then uses those bounds to establish objective decrease and client stationarity under its stated conditions. The conclusions are conditional: client protection is directional, and the limit statement does not require convergence of the full parameter sequence.
B.2 Supporting Results
- Under the alignment condition, the aggregate pseudo-gradient magnitude is bounded relative to the weighted average client pseudo-gradient magnitudes, preventing arbitrarily small aggregation while γ remains fixed.The passage distinguishes Gt, the weighted average of client pseudo-gradient magnitudes, from the magnitude of their weighted aggregate.Source: §B.2 · p. 17
- The local-drift bound is relative to a client’s round-start gradient, so the error vanishes as that gradient vanishes, even with a fixed local learning rate.In the deterministic setting, the pseudo-gradient is the sum of K local gradients, with a local-training error bounded using smoothness.Source: §B.2 · p. 18
B.3 Proof of the Finite-Time Convergence Rate
- Summing the roundwise decrease yields a finite-time bound whose coefficient is independent of T when learning rates and the uniform alignment margin are fixed.The proof also notes that the drift restriction need not require a local learning rate that decreases with T; feasibility and the uniform margin remain assumptions.Source: §B.3 · p. 19, §B.4 · p. 20
- Client protection guarantees directional progress, not monotonic reduction of every client loss or a guarantee about client accuracy.The fixed server step guarantees global objective descent, but need not satisfy the additional condition for every client at every round.Source: §B.3 · p. 20
B.4 Client Stationarity and Composite Multi-Stationarity
- With fixed positive client weights, the analysis shows client gradients vanish; any convergent parameter subsequence is stationary for every client and for the primary global objective.Composite multi-stationarity is conditional on the alignment geometry, which does not establish that every heterogeneous objective collection has a common stationary point.Source: §B.4 · p. 20
- Bounded iterates ensure an accumulation point, but the result does not require convergence of the full parameter sequence to one point.Without boundedness, the result still shows that every client gradient vanishes but does not assert a finite limiting parameter.Source: §B.4 · p. 20
B.5 Comparative Analysis with CRAFT
FedPCD differs from nearby correction methods by preserving the FedAvg-like direction as its primary objective and imposing lower bounds on client progress. The analysis compares inequality constraints with exact alignment and contrasts the methods’ convergence bounds under their respective assumptions.
B.5 Comparative Analysis with CRAFT
- FedPCD keeps the FedAvg-like direction primary and imposes lower bounds on client progress, unlike CRAFT’s prescribed per-client alignment rates.FedMGDA+, FedFV, and FedLF instead reshape the aggregate through min-norm search, pairwise projection, or a fairness-gradient addition.Source: §B.5 · p. 21
- With identical reference directions and targets, inequality constraints require no larger correction than exact-alignment constraints, and strictly less when their projections differ.The inequality-feasible set contains the equality-feasible set.Source: §B.5 · p. 21
- Feasible inequalities need not make exact equalities feasible; moreover, inequality projection leaves an already feasible reference unchanged even when some targets are exceeded.The unchanged-reference result concerns constrained directions before rescaling in Equation 11.Source: §B.5 · p. 22
- Both methods’ published bounds give O(1/T) for the average squared global gradient, but under different assumptions; this compares upper bounds, not CRAFT’s attainable rate.CRAFT assumes bounded local gradients and updates; FedPCD uses a uniform angular margin and relative drift, and its rate does not follow from inequality constraints alone.Source: §B.5 · p. 22
C Expanded Results
The expanded results show client-accuracy distributions and convergence comparisons at α = 0.1 and α = 1, alongside fairness views of global accuracy and client-accuracy spread. They also describe Jain’s index as a complementary measure of client fairness.
C.1 Additional α values
- At α = 0.1 and α = 1, the additional results use τ = 0.02 and H = 10, matching the settings for α = 0.01.The figures cover client distributions, comparison plots, and convergence results.Source: §C.1 · p. 22
- Client-accuracy distributions at α = 0.1 and α = 1 compare FedPCD, FedAvg, and the strongest non-FedPCD baseline, with distribution means reported per panel.The cross-dataset figures order methods by global accuracy and pool client accuracies over five seeds.Source: Figure 9 · p. 22, Figure 11 · p. 23, Figure 10 · p. 23
- The convergence plots mark when FedPCD reaches FedAvg’s final accuracy and the strongest baseline’s final accuracy, across α = 0.1 and α = 1.When FedPCD does not reach the strongest baseline’s final accuracy, a hollow ring marks when that baseline reaches FedAvg’s final accuracy.Source: Figure 13 · p. 24, Figure 14 · p. 25
C.2 Fairness Study
- At α = 0.1 and α = 1, fairness plots compare global accuracy with client-accuracy spread, while Table 3 reports Jain’s index, where higher is better.Jain’s index equals 1 when clients have equal accuracy and is unchanged by scaling every client’s accuracy by the same factor.Source: Figure 15 · p. 26, Figure 16 · p. 26, §C.2 · p. 26
D.1 Component Ablation
The appendix tests FedPCD’s components, history-window length, and normalization choice. The reported results identify history as the largest component contribution, show gains with longer histories that eventually diminish, and favor instantaneous ℓ2 normalization overall.
D.1 Component Ablation
- At α = 0.01, removing history lowers CIFAR-10 global accuracy from 74.84% to 59.26%; removing τ constraints, normalization, or the primary direction also degrades performance.The history effect is particularly strong under greater heterogeneity; removing the primary direction causes a larger loss, while normalization removal causes smaller but consistent degradations.Source: §D.1 · p. 28, Table 4 · p. 28
D.2 History window sweep
- Increasing H improves global and worst-10% accuracy, with gains diminishing as histories grow; Fashion-MNIST obtains most global-accuracy improvement by approximately H = 3–5.The gains coincide with more active secondary constraints; their performance benefit eventually diminishes even as constraint counts continue to rise.Source: §D.2 · pp. 28–29, Figure 17 · p. 29
D.3 Normalization Study
- Instantaneous ℓ2 normalization achieves the best global accuracy at all three heterogeneity levels and strongest results on most other metrics, while EMA has small exceptions at α = 0.1.The α = 0.1 EMA variant slightly improves client standard deviation from 7.45% to 7.44% and worst-10% accuracy from 61.28% to 61.83%; these differences do not persist across other settings.Source: §D.3 · p. 29, Table 5 · p. 30