Source-linked AI summary
Conflict-Aware Client Selection for Multi-Server Federated Learning
Mingwei Hong, Zheng Lin, Zehang Lin, Lin Li, Miao Yang, Xia Du, Zihan Fang, Zhaolu Kang, Dianxin Luan, Shunzhi Zhu
TL;DR
Multi-server FL faces resource contention when overlapping servers select clients without coordination. RL-CRP combines categorical-HMM conflict prediction with decentralized reinforcement learning and fairness-aware rewards, and experiments show reduced conflicts and improved convergence.
Problem
Overlapping client coverage and uncoordinated selection in multi-server FL cause resource contention, bandwidth conflicts, and training failures.
Method
RL-CRP uses categorical HMMs to predict conflicts from sparse histories, decentralized reinforcement learning for client selection, and fairness-aware rewards.
Results
RL-CRP achieves the highest test accuracy among benchmarks, reaching 67.68% under IID and 61.32% under non-IID settings.
Takeaways & Limitations
RL-CRP significantly reduces conflict occurrences and improves convergence speed in multi-server federated learning.
Abstract
from arXiv · showhide
Federated learning (FL) has emerged as a promising distributed machine learning (ML) that enables collaborative model training across clients without exposing raw data, thereby preserving user privacy and reducing communication costs. Despite these benefits, traditional single-server FL suffers from high communication latency due to the aggregation of models from a large number of clients. While multi-server FL distributes workloads across edge servers, overlapping client coverage and uncoordinated selection often lead to resource contention, causing bandwidth conflicts and training failures. To address these limitations, we propose a decentralized reinforcement learning with conflict risk prediction, named RL CRP, to optimize client selection in multi-server FL systems. Specifically, each server estimates the likelihood of client selection conflicts using a categorical hidden Markov model based on its sparse historical client selection sequence. Then, a fairness-aware reward mechanism is incorporated to promote long-term client participation for minimizing training latency and resource contention. Extensive experiments demonstrate that the proposed RL-CRP framework effectively reduces inter-server conflicts and significantly improves training efficiency in terms of convergence speed and communication cost.
I. INTRODUCTION
Multi-server FL improves scalability and latency but overlapping client coverage creates resource contention and training failures. RL-CRP addresses this with conflict prediction, decentralized selection, and fairness-aware participation.
- Overlapping client coverage and uncoordinated selection can cause bandwidth and computing contention, leading to training timeouts and failures.
- RL-CRP predicts client conflict risk from sparse historical selection sequences using a categorical HMM.
- Decentralized reinforcement learning lets each server select clients from local observations while reducing inter-server contention.
- A fairness-aware reward encourages sustained participation and improves training performance.
- Extensive experiments demonstrate that RL-CRP reduces conflicts and improves convergence compared with existing benchmarks.
II. SYSTEM MODEL
The multi-server FL model distributes clients and local models across servers, but overlapping coverage can cause simultaneous client selections, resource conflicts, and longer training.
- Each server maintains a local model over its associated client subset, while the global objective is formulated as a multi-task problem across servers.The system contains M servers and N clients, with server m associated with client subset N_m.
- Each server selects a fixed number S of clients per communication round because of bandwidth constraints.
- Overlapping server coverage can cause clients to be selected simultaneously, creating resource conflicts that delay or prevent model updates.
- The proposed mitigation predicts client conflict probabilities from historical information and uses fairness-aware rewards to balance participation.
III. CONFLICT-AWARE CLIENT SELECTION IN MULTI-SERVER FEDERATED LEARNING
The framework combines conflict-risk prediction with decentralized reinforcement learning to avoid inter-server conflicts while maintaining long-term fairness in client participation.
- The proposed multi-server FL framework uses decentralized client selection to avoid inter-server conflicts while supporting long-term fairness.Its two components are conflict risk prediction and decentralized RL with a fairness guarantee.
A. Conflict Risk Prediction with Sparse Historical Sequence
The method predicts client conflict risk from sparse historical selections using a categorical HMM, then incrementally updates its parameters as new observations arrive.
- Given a client sequence with an unavailable recent history gap d > 1, the method computes forward probabilities and propagates the hidden-state posterior to time t − 1.
- A categorical HMM models client conflict risk using hidden states, observation categories, state transitions, emissions, and an initial state distribution.Binary conflict indicators are given as an example of the observation categories.
- The conflict probability at time t is obtained by combining the propagated hidden-state distribution with the transition matrix and conflict-event emission probabilities.The conflict-event emission vector is B[:, 1].
- As new observation sequences arrive, an incremental Baum–Welch procedure updates the HMM parameters using backward probabilities, transition and state posteriors, and an exponential moving average.The indicator function equals 1 when its condition is true and 0 otherwise.
B. Fairness-aware Client Selection based on RL
The framework formulates decentralized client selection as an MDP in which each server uses local latency and conflict-risk observations, while a fairness-aware reward balances communication efficiency, conflict avoidance, and equitable participation.
- Fairness-aware Reward: The reward jointly penalizes overall communication latency and conflict or timeout costs while rewarding fairer client participation.Latency is determined by the slowest communication and computation client, and the conflict term captures selection-induced contention or timeouts.
- Fairness-aware Reward: The fairness metric approximates the inverse coefficient of variation of clients’ training-round counts, with α controlling its trade-off against latency and conflict costs.The metric uses the average and standard deviation of training rounds, plus a small constant for numerical stability.
- SAC Optimization: SAC learns the policy through twin Q-networks, target networks, entropy regularization, and replay-buffer updates to maximize cumulative discounted reward.The temperature parameter is automatically adjusted toward a target entropy level.
- Client Selection with Decentralized RL: Each server independently selects a client subset from local observations and conflict predictions using a decentralized SAC policy, without inter-server communication.The policy uses server states containing upload latency and estimated client conflict probabilities.
- Bandwidth Allocation and Procedure: After selection, a water-filling procedure assigns bandwidth in descending channel-quality order until the available bandwidth cannot support another client.The summarized algorithm initializes SAC, conflict-prediction, HMM, and bandwidth parameters, then updates the policy and conflict predictor each round.
A. Experimental Setup
The evaluation uses CIFAR-10 with IID and non-IID client data, a three-layer convolutional model, and a two-server communication environment with specified bandwidth, latency, and fairness settings.
- Dataset and Model: Experiments evaluate RL-CRP on CIFAR-10 under both IID and non-IID distributions, using Dirichlet parameter η = 0.1 for the non-IID partition.The global model has three convolutional layers, and clients train with mini-batch SGD using batch size 16 for five local epochs.
- Communication Environment: The communication environment has two servers, 50 total clients with 40 inside coverage, 100 MHz bandwidth, 40-second maximum latency, and fairness weight α = 100.Each server is deployed at a different location with a 1 km coverage radius.
- Benchmarks: The study compares RL-CRP with FedAvg and an RL-CRP variant without the fairness component.FedAvg randomly selects clients, while the supplied setup passage names the fairness-ablated RL-CRP benchmark.
B. Superb performance of RL-CRP
RL-CRP achieves the strongest reported accuracy and reward behavior while reducing conflicts across server scales, though fairness introduces a conflict–accuracy trade-off.
- Test accuracy: 67.68% IID and 61.32% non-IID convergence accuracy make RL-CRP the best-performing benchmark in Fig. 2.The fairness-aware reward encourages participation from previously underrepresented clients; the fairness-ablated variant performs considerably worse.
- Reward convergence: RL-CRP converges to rewards faster than ENSAC and retains better final reward as the number of servers increases.Increasing server count raises conflict probability and lengthens convergence for all algorithms.
- Conflict performance: RL-CRP produces fewer conflicts than ENSAC and FedAvg at two servers, while its fairness-ablated variant produces the fewest conflicts.The comparison indicates a trade-off: fairness improves global accuracy but can increase client-selection conflicts.
- Conflict performance: Conflict counts increase with the number of servers, yet RL-CRP and its fairness-ablated variant remain below ENSAC across scales.Larger deployments expand overlapping coverage areas and intensify competition for shared clients.
V. CONCLUSION
The paper proposes RL-CRP for conflict-aware client selection in multi-server federated learning. It combines categorical-HMM conflict prediction with fairness-aware reinforcement learning and reports fewer conflicts and faster convergence than existing benchmarks.
- Conclusion: RL-CRP predicts client-selection conflicts from sparse historical sequences using a categorical HMM before selecting clients with reinforcement learning.Its fairness-aware reward promotes balanced participation while minimizing latency and resource competition.
- Conclusion: The framework significantly reduces conflict occurrences and improves convergence speed compared with existing benchmarks.
- Conclusion: The authors identify satellite networks, distributed learning systems, and LLM systems as potential future application areas.