Source-linked AI summary
Client Selection in Federated Learning: Principles, Challenges, and Opportunities
Lei Fu, Huanle Zhang, Ge Gao, Mi Zhang, Xin Liu
TL;DR
FL client heterogeneity makes random sampling potentially inefficient, motivating methods that select clients according to their characteristics. This paper systematically reviews client-selection research by organizing methods around prioritization criteria and examining implementation practices, challenges, and opportunities. It presents utility-based selection as a framework spanning statistical and system factors, while noting that the survey is not exhaustive.
Problem
Heterogeneity in FL clients’ data distributions and hardware configurations means random sampling may not fully exploit local updates and may degrade accuracy, convergence, and fairness.
Method
The paper conducts a systematic literature review of FL client-selection work and organizes representative methods by their criteria for prioritizing clients.
Results
The review presents client-selection approaches using statistical and system utility, along with implementation practices, challenges, and research opportunities.
Takeaways & Limitations
The survey provides guidance for selecting client-selection mechanisms and insights for designing FL systems with heterogeneous clients.
Takeaways & Limitations
The survey is not exhaustive: it emphasizes broadly applicable explicit utility and decision criteria while excluding several FL variants and implicit selection approaches.
Abstract
from arXiv · showhide
As a privacy-preserving paradigm for training Machine Learning (ML) models, Federated Learning (FL) has received tremendous attention from both industry and academia. In a typical FL scenario, clients exhibit significant heterogeneity in terms of data distribution and hardware configurations. Thus, randomly sampling clients in each training round may not fully exploit the local updates from heterogeneous clients, resulting in lower model accuracy, slower convergence rate, degraded fairness, etc. To tackle the FL client heterogeneity problem, various client selection algorithms have been developed, showing promising performance improvement. In this paper, we systematically present recent advances in the emerging field of FL client selection and its challenges and research opportunities. We hope to facilitate practitioners in choosing the most suitable client selection mechanisms for their applications, as well as inspire researchers and newcomers to better understand this exciting research topic.
I. INTRODUCTION
Federated learning clients differ substantially in data and system capabilities, so client selection is used to address heterogeneity and improve training outcomes. This paper surveys client selection systematically, discussing representative methods, implementation practices, challenges, and opportunities.
- Cross-device FL clients exhibit significant heterogeneity in data statistics and system configurations, which can degrade FL performance if not handled appropriately.
- Client selection chooses devices in each training round and can improve accuracy, fairness, robustness, and training overheads.
- The paper provides an overview organized by the criteria used to prioritize FL clients rather than simply listing existing work.
- The review covers client heterogeneity, prioritization criteria, implementation practices, research challenges, and research opportunities.
A. Research Questions
The paper asks how client behavior affects selection, how existing algorithms prioritize clients, how methods are implemented, and what challenges and opportunities remain. It answers these questions through a systematic search and analysis of standard-FL studies with explicit utility measurements and scheduling decisions.
- The research questions cover client behavior, prioritization principles, implementation practices, challenges, and opportunities for FL client selection.
- The review searches Scopus, Web of Science, and Google Scholar using terms for federated learning, selection, sampling, and scheduling.
- Included papers target standard FL and use explicit client utility measurements and scheduling decisions.
- For selected papers, the authors analyze client behavior, categorize prioritization methods, summarize implementation platforms, and identify challenges and opportunities.
D. Existing Surveys/Reviews
Existing FL surveys cover broad topics but provide limited depth on client selection. This paper motivates its focused review by linking client-selection needs to system and statistical heterogeneity and their effects on training.
- Existing FL surveys cover topics from data distribution and partitioning to real-world implementations but only touch on client selection.
- System and statistical heterogeneity can reduce model accuracy by up to 9.2% and increase convergence time by 2.64X in a study of 136k smartphones.
- System heterogeneity includes differences in computation, communication, energy, and other client capabilities.
- FL data are massively distributed, unbalanced, and non-IID across clients.
IV. PRIORITIZING FL CLIENTS
FL client selection measures client utility and schedules clients according to those measurements. Existing approaches represent utility through statistical and system factors, with statistical utility capturing the usefulness of local updates.
- Each training round measures client utility or priority and selects clients with the best measurements for training and aggregation.
- An overall client utility is often represented as the product of statistical utility and system utility.
- Statistical utility measures the usefulness of a client’s local update to the global model.
- Statistical utility methods are categorized as data sample-based or model-based, with data sample-based methods using local data to quantify utility.
1) Data Sample-Based Utility Measurement:
Data sample-based utility measures how useful a client’s local data are for the global model, using sample quantity, sample importance, or cumulative loss. These approaches trade representational simplicity against computational cost.
- 1) Data Sample-Based Utility Measurement:: Statistical utility measures the usefulness of a client’s local update and includes data sample-based and model-based categories.
- 1) Data Sample-Based Utility Measurement:: Client data size is a simple utility proxy when every sample has equal quality, such as under IID data.The proxy uses the number of data samples on client i.
- 1) Data Sample-Based Utility Measurement:: Importance sampling assigns higher scores to samples whose gradients diverge farther from the model, but evaluating all sample gradients has overwhelming overhead.
- 1) Data Sample-Based Utility Measurement:: Replacing gradient norms with losses greatly reduces computation because each sample’s loss is available during local training.
- 1) Data Sample-Based Utility Measurement:: Cumulative client loss further simplifies sample-based utility calculation and has shown promising results when adopted by prior work.
2) Model-Based Utility Measurement:
Model-based utility compares client models or gradients with the global model, or evaluates how client models move during local training. Different measures prioritize divergence, direction, or convergence trends.
- 2) Model-Based Utility Measurement:: Normalized model divergence measures the average difference between a client’s model weights and the global model.A small divergence can indicate an insignificant local update that may be ignored.
- 2) Model-Based Utility Measurement:: Same-sign weight percentages compare client and global model directions, but a lower percentage can yield better communication efficiency upon convergence.This finding contrasts with methods that prioritize more divergent local models.
- 2) Model-Based Utility Measurement:: Convergence-trend measures select clients whose weights move most away from zero by comparing local and global layer-wise movement directions.The importance score uses the norms of client and global movement vectors.
- 2) Model-Based Utility Measurement:: Another approach compares a client’s local model before and after training, assuming greater contribution when local training produces a significantly different model.
- 2) Model-Based Utility Measurement:: L2-norm-based measures treat a higher model-gradient norm as indicating a more valuable client.
- 2) Model-Based Utility Measurement:: Gradient inner products capture the relative direction between local and global gradients, and FOLB and FedPNS remove clients with negative inner products.
B. System Utility
System utility addresses hardware-driven differences in training and transmission time by prioritizing clients that meet hard or soft performance constraints. Deadline design determines how stragglers affect aggregation.
- B. System Utility: Hardware heterogeneity creates different training and transmission overheads, while stragglers prolong training rounds and can deteriorate overall performance.
- B. System Utility: Hard deadline selection removes clients whose total round time exceeds T from FL aggregation.Total round time includes local training, transmission, compression, and related costs.
- B. System Utility: Soft deadlines penalize stragglers instead of strictly removing them; the penalty is controlled by α and increases exponentially for stragglers.Non-stragglers receive no punishment.
- B. System Utility: Deadline thresholds may represent time, FLOPs, or Mbps, and manually chosen values can cause inferior performance because T strongly affects aggregation.FedBalancer automatically adjusts T and reports better performance than a fixed deadline.
C. Scheduling
Client scheduling combines statistical and system utility to rank participants, while allowing additional utility dimensions or alternative weighted formulations. In practice, utility measurement is constrained because client utility may be observable only after participation.
- C. Scheduling: Ideal scheduling measures every client’s utility each round and selects clients with the highest values, but utility often becomes available only after participation.
- C. Scheduling: The paper’s default overall utility multiplies statistical and system utility, a form also used by Oort and PyramidFL.
- C. Scheduling: Multiplication can incorporate fairness and robustness utilities or neutralize unwanted aspects by assigning their utility functions a value of 1.The paper describes this form as expressive.
- C. Scheduling: Alternative scheduling can add loss, data size, computation power, resource demand, and update age with adjustable weights.Eiffel uses this additive formulation.
V. CURRENT IMPLEMENTATION
The paper describes data partitioning approaches, FL frameworks, and unresolved challenges affecting practical client selection. These challenges include device availability, exploitation–exploration tradeoffs, and scenario-dependent heterogeneity.
- Data simulation: Synthetic partitions use conventional ML datasets to simulate different degrees of data heterogeneity across clients.Realistic partitions instead use datasets containing client IDs.
- FL frameworks: FL client selection research can use purpose-built frameworks such as TFF, FedScale, Leaf, and FedML.Their capabilities span experimentation, device-profile simulation, federated datasets, evaluation, reference implementations, and cross-platform edge training.
- Challenges: Device availability is often incorrectly assumed to be constant, although devices may participate only when idle, charged, and connected to WiFi.Scarce availability data and dominance by some active devices can create participant bias.
- Challenges: Always selecting prioritized clients can exclude underrepresented clients, creating an exploitation–exploration tradeoff.The paper notes that this dilemma is especially challenging in FL and that current methods use simple tradeoffs.
- Challenges: A general client selection algorithm remains difficult because heterogeneity differs across regions and application scenarios.For example, mobile users in the US generally have stable network conditions, unlike a common assumption of frequent transmission interruptions.
VII. RESEARCH OPPORTUNITIES
The paper identifies research opportunities in choosing client counts, developing theoretical guarantees, and addressing experiment bias. It emphasizes that client selection must account for heterogeneous scenarios and requires stronger analytical foundations.
- Client quantity: Selecting more clients can improve FL convergence rate, but the gains diminish as the number increases.The paper also states that more clients are preferred when data are more heterogeneous, while Google Gboard uses 100 clients for keyboard query suggestions.
- Theoretical performance guarantee: Experimental evidence on which clients to prioritize is contradictory, with some studies favoring divergent models and others reporting the opposite.The paper attributes these differing observations as potentially stemming from different application scenarios.
- Theoretical performance guarantee: Most client selection studies rely on experiments, making reported results susceptible to experiment bias.The paper calls for theoretical analysis frameworks because heuristic algorithms make guarantees about accuracy, convergence, robustness, and fairness challenging.
- Theoretical performance guarantee: The lack of theoretical performance guarantees can lead practitioners to adopt random client selection despite potentially sub-optimal performance.The paper identifies theoretical analysis as a research need for client selection algorithms.
C. Benchmark and Evaluation Metrics
The paper argues that FL client selection needs fairer evaluation across metrics, settings, and scales. It also limits its survey scope to broadly applicable selection criteria while identifying open infrastructure needs.
- Benchmark and evaluation metrics: Different evaluation metrics are not directly comparable because client count can improve round-to-accuracy while increasing round duration and transmission load.The paper lists final model accuracy, time-to-accuracy, round-to-accuracy, and transmission load as commonly used metrics.
- Benchmark and evaluation metrics: Different experiment settings and local data sampling choices can change FL performance, motivating well-established benchmarks and evaluation metrics.Such benchmarks are intended to support fair and objective comparisons among client selection algorithms.
- Broader FL scenarios: Hierarchical FL and cluster-based FL extend evaluation beyond the classical global-model, server-client setting.Hierarchical FL includes cloud, edge, and client spaces, while cluster-based FL groups clients by data distribution or system capability.
- Large-scale open FL testbeds: Simulation and small-scale implementations may not match actual performance when client selection requires many clients.Building separate experiment environments also creates reproducibility and fair-comparison difficulties.
- Scope: The survey is not exhaustive and focuses on general selection criteria rather than variants for particular scenarios or methods without explicit utility functions or decisions.It covers heterogeneity, utility functions, implementation suggestions, challenges, and research opportunities.