PaperlayerView exact revision on arXiv

Source-linked AI summary

Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective

Jie Xu, Heqiang Wang

arXiv:2004.04314v1cs.DCcs.LG

TL;DR

Wireless federated learning resource allocation is typically optimized round by round, despite interdependent rounds with different importance to final performance. This paper studies long-term joint client selection and bandwidth allocation, proposing OCEAN, which approaches the desired temporal selection performance while approximately satisfying client energy constraints.

  • Problem

    Existing wireless federated learning resource allocation largely overlooks that learning rounds are interdependent and have varying significance for final learning performance.

  • Method

    The paper formulates long-term joint client selection and bandwidth allocation with client energy-deficit queues and proposes the online optimization algorithm OCEAN.

  • Results

    OCEAN achieves the desired client-selection-pattern performance within a bounded gap while approximately satisfying clients’ energy constraints and improving accuracy, loss, and robustness.

  • Takeaways & Limitations

    The findings support evaluating wireless federated learning resource allocation over multiple rounds rather than treating each round independently.

Abstract

from arXiv · show

This paper studies federated learning (FL) in a classic wireless network, where learning clients share a common wireless link to a coordinating server to perform federated model training using their local data. In such wireless federated learning networks (WFLNs), optimizing the learning performance depends crucially on how clients are selected and how bandwidth is allocated among the selected clients in every learning round, as both radio and client energy resources are limited. While existing works have made some attempts to allocate the limited wireless resources to optimize FL, they focus on the problem in individual learning rounds, overlooking an inherent yet critical feature of federated learning. This paper brings a new long-term perspective to resource allocation in WFLNs, realizing that learning rounds are not only temporally interdependent but also have varying significance towards the final learning outcome. To this end, we first design data-driven experiments to show that different temporal client selection patterns lead to considerably different learning performance. With the obtained insights, we formulate a stochastic optimization problem for joint client selection and bandwidth allocation under long-term client energy constraints, and develop a new algorithm that utilizes only currently available wireless channel information but can achieve long-term performance guarantee. Further experiments show that our algorithm results in the desired temporal client selection pattern, is adaptive to changing network environments and far outperforms benchmarks that ignore the long-term effect of FL.

I. INTRODUCTION

The paper reframes wireless federated learning resource allocation as a long-term problem because client-selection decisions across rounds are interdependent and rounds may contribute differently to final performance. It shows that later-round client selection improves learning and develops OCEAN to jointly select clients and allocate bandwidth under energy constraints.

  • I. INTRODUCTION: WFLNs coordinate repeated local training and model-update aggregation over a shared wireless link, requiring per-round client selection and bandwidth allocation.Finite bandwidth limits participation, while volatile channel conditions require adaptive allocation.
  • I. INTRODUCTION: The paper formulates long-term joint client selection and bandwidth allocation across finite FL rounds under individual client energy constraints.The formulation addresses interdependent round decisions and unpredictable future wireless channel conditions.
  • I. INTRODUCTION: Later-round client selection achieves higher accuracy, lower training loss, and greater robustness than selecting more clients in early rounds.This result is shown for image classification and text generation with a fixed total number of selected clients.
  • I. INTRODUCTION: OCEAN converts the long-term problem into per-round convex optimization using virtual energy deficit queues and only currently available wireless information.The resulting online algorithm is practical and has low complexity.
  • I. INTRODUCTION: OCEAN achieves the desired client-selection pattern within a bounded performance gap while approximately satisfying client energy constraints, with an [O(1/V ), O(V )] learning-energy tradeoff.V is an algorithm parameter.
  • I. INTRODUCTION: Each round prioritizes clients by current energy-deficit queue length divided by channel state, while selected clients with lower priority receive more bandwidth.This reverses throughput-oriented allocation, which favors clients with better channel conditions.

II. RELATED WORK · III. IMPACT OF TEMPORAL CLIENT SELECTION PATTERN

Related work addresses FL communication and wireless-resource limitations through algorithmic, architectural, and scheduling approaches, but generally overlooks unequal round significance. The paper therefore studies long-term client selection and bandwidth allocation, while experiments show that temporal selection patterns substantially affect final FL performance.

  • II. RELATED WORK: FL communication-efficiency research modifies algorithms through significant-client updates, gradient quantization, and sparse or structured updates.
  • II. RELATED WORK: Hierarchical FL performs partial aggregation at multiple edge servers before further aggregation by a cloud server.
  • II. RELATED WORK: Analog over-the-air model aggregation exploits wireless transmission, but requires very stringent synchronization.
  • II. RELATED WORK: Wireless FL resource-allocation studies optimize aggregation frequency, transmission power or rate, and local-update CPU frequency, often with all clients participating each round.
  • II. RELATED WORK: Because wireless bandwidth cannot support many simultaneous uploads, client scheduling and joint bandwidth allocation have been proposed to minimize learning loss or training time.
  • II. RELATED WORK: Existing resource-allocation formulations treat rounds separately or equally, whereas this paper considers varying round significance under long-term energy constraints and uncertain wireless channels.
  • III. IMPACT OF TEMPORAL CLIENT SELECTION PATTERN: Long-term energy constraints make temporal selection consequential because selecting more clients early can deplete batteries and reduce later participation.
  • III. IMPACT OF TEMPORAL CLIENT SELECTION PATTERN: Even with the same average number of selected clients, different temporal patterns can produce considerably different final FL performance.The section introduces two experiments to demonstrate this impact.

A. Image Classification on the MNIST Dataset · B. Text Generation on the Shakespeare Dataset · IV. WIRELESS FEDERATED LEARNING NETWORK MODEL

Experiments on MNIST and Shakespeare show that ascending client selection, with more clients participating later, improves federated learning performance and robustness. The WFLN model then frames per-round client selection around wireless channels, battery status, and learning performance.

  • A. Image Classification on the MNIST Dataset: MNIST training uses a DNN classifier across 10 clients with FedAvg for 300 rounds.Three temporal selection patterns are evaluated with an average of 5 selected clients per round.
  • A. Image Classification on the MNIST Dataset: Although the average number of selected clients is identical, temporal selection patterns produce different end-of-training loss and prediction accuracy.The compared patterns are Uniform, Ascend, and Descend.
  • A. Image Classification on the MNIST Dataset: Ascend achieves the best MNIST training loss and prediction accuracy versus Uniform and Descend, while also showing much smaller standard deviation.Its later-round participation of more clients smooths abrupt changes in individual client models.
  • A. Image Classification on the MNIST Dataset: Ascend’s advantage reflects differing round difficulty: early rounds are less sensitive to client counts, whereas later rounds require more clients to improve accuracy.Selecting fewer clients early therefore minimally affects learning speed, while greater later participation supports higher accuracy.
  • B. Text Generation on the Shakespeare Dataset: On Shakespeare text generation, Ascend similarly significantly outperforms Descend and Uniform in training loss, accuracy, and robustness.The experiment fine-tunes a pre-trained recurrent neural network on Shakespeare text using federated learning.
  • B. Text Generation on the Shakespeare Dataset: The exact optimal temporal selection pattern cannot be analytically characterized and may vary across tasks, models, datasets, and algorithms.Nevertheless, the general ascending trend provides guidance for client selection across rounds.
  • IV. WIRELESS FEDERATED LEARNING NETWORK MODEL: The WFLN comprises one server and K clients, with federated learning alternating between server aggregation and client local-model updates each learning round.In each round, the network decides which clients upload updates based on wireless channel conditions and remaining battery to maximize learning performance.

A. Client Energy Consumption · B. System Learning Performance · C. Problem Formulation

The paper models wireless upload energy under OFDMA, defines a temporally weighted learning-performance metric, and formulates long-term joint client selection and bandwidth allocation under per-client energy budgets. The resulting problem must be addressed online because future channel information is unavailable and the formulation is mixed-integer nonlinear.

  • A. Client Energy Consumption: OFDMA allocates a total bandwidth B among selected clients for local model uploading.The allocation is made per client and per learning round.
  • A. Client Energy Consumption: Unselected clients receive no bandwidth, while selected clients must receive at least bmin because arbitrarily small allocations are impractical.The model assumes bmin ≤1/K to preserve feasibility.
  • B. System Learning Performance: Client selection should include as many clients as possible, but finite radio and battery resources require avoiding battery depletion that would impair later rounds.The paper motivates resource decisions by the risk of insufficient model updates in later communication rounds.
  • B. System Learning Performance: An increasing sequence of temporal weights ηt often improves FL performance by favoring more client selections in later learning rounds.The metric captures varying significance across learning rounds but does not exactly characterize FL speed or accuracy.
  • C. Problem Formulation: The optimization maximizes the weighted sum of selected clients over T learning rounds while jointly choosing clients and bandwidth allocations each round.The objective is intended to reflect long-term performance and final FL outcome.
  • C. Problem Formulation: Each client’s total energy consumption over T rounds must not exceed its budget Hk, such as battery capacity or a client-set energy limit.The formulation also imposes feasibility conditions on bandwidth allocation and client selection.
  • C. Problem Formulation: An online approach is required because solving P1 needs unavailable future channel conditions and remains difficult as mixed-integer nonlinear programming.The approach must make joint decisions without foreseeing the far future.

D. Offline Benchmark: R-Round Lookahead Algorithm · V. ONLINE CLIENT SELECTION AND BANDWIDTH ALLOCATION · A. The OCEAN Algorithm

The paper introduces an impractical R-round lookahead offline benchmark, then develops OCEAN, a purely online algorithm that uses current channel information and energy-deficit queues to guide client selection and bandwidth allocation under long-term energy constraints.

  • D. Offline Benchmark: R-Round Lookahead Algorithm: The R-round lookahead algorithms assume channel information for the next R learning rounds and serve as offline benchmarks.They divide the FL period into M ≥1 frames of R ≥1 rounds, with T = MR.
  • D. Offline Benchmark: R-Round Lookahead Algorithm: Because they require future channel information, the R-round lookahead algorithms are impractical unless R = 1.Their purpose is to benchmark the practical online algorithm introduced next.
  • V. ONLINE CLIENT SELECTION AND BANDWIDTH ALLOCATION: The paper develops OCEAN, the Online Client sElection and bAndwidth allocatioN algorithm, and characterizes its structural properties and efficiency relative to the optimal offline algorithm.The supplied passage identifies OCEAN as the section’s main algorithmic contribution and comparison target.
  • A. The OCEAN Algorithm: Long-term client energy constraints couple client selection and bandwidth allocation across learning rounds, while decisions must be made without foreseeing the future.Selecting more clients can reduce per-client bandwidth, increase energy consumption, and reduce future energy budgets.
  • A. The OCEAN Algorithm: OCEAN uses Lyapunov control and a virtual energy deficit queue q_k(t) for each client to guide decisions toward the long-term energy constraint.The queue length indicates the deviation of client k’s current energy consumption from its long-term constraint H_k.
  • A. The OCEAN Algorithm: OCEAN is purely online and requires only currently available channel state information as inputs.Positive control parameters dynamically adjust the tradeoff between maximizing selected clients and minimizing energy consumption across M frames of R rounds.
  • A. The OCEAN Algorithm: By weighting current decisions with clients’ energy deficits, OCEAN prioritizes minimizing energy deficits when q_k(t) is larger.The long-term optimization is decomposed into per-round problems, although solving each P3 remains difficult.

B. Solving the Per-Round Problem

The per-round problem P3 is difficult because it is mixed-integer, but its special structure enables OCEAN-P to find the optimum by solving at most K convex optimization problems. OCEAN-P incrementally expands the client set by selection priority, computes optimal bandwidth allocations for candidate sets, and chooses the set with maximum objective value.

  • Problem structure: P3 is a difficult mixed-integer problem, for which polynomial-time optimal algorithms generally do not exist.
  • OCEAN-P algorithm: OCEAN-P exploits P3’s special structure to return the optimal solution by solving at most K convex optimization problems.
  • OCEAN-P algorithm: OCEAN-P initializes S with clients having ρ_k = 0, then adds remaining clients one by one in ascending selection priority ρ_k.Lower selection-priority values correspond to higher priority.
  • OCEAN-P algorithm: For each candidate selection set S, the algorithm solves the bandwidth-allocation problem, then selects S* = arg max_{S∈𝒮} W*(S) and uses b* = b*(S*).The possible selection sets generated by the expansion rule number at most K.
  • OCEAN-P algorithm: A termination condition stops adding clients when the optimal bandwidth allocation makes η_t − … < 0 for the last added client, reducing optimization solves when K is large.

C. Structural Results and Performance Analysis

The section characterizes OCEAN-P’s optimal client-selection and bandwidth-allocation structure, then proves OCEAN’s performance guarantee relative to an R-round lookahead algorithm. The results establish threshold-based selection, channel- and deficit-dependent allocation behavior, bounded energy-constraint deviation, and an O(1/V)-optimality tradeoff.

  • Structural Results: Only clients with selection priority ρ_k below a threshold are selected for a federated-learning round.This thresholding property follows from Theorem 1.
  • Structural Results: Among selected clients, bandwidth allocation determines their weighted energy consumption q_k(t)E_k^t.Proposition 1 characterizes both allocated bandwidth and incurred energy consumption.
  • Structural Results: Clients with smaller energy deficits and better channel conditions are more likely to be selected, but selected clients with smaller ρ_k receive less bandwidth.Allocating more bandwidth to lower-priority clients would reduce bandwidth available to clients with larger ρ, increasing their energy consumption and deficits.
  • Performance Analysis: OCEAN approximately satisfies every client’s energy constraint with bounded deviation when compared with the R-round lookahead algorithm.Theorem 2 applies for R and M in Z+ with T = MR.
  • Performance Analysis: OCEAN is O(1/V)-optimal for federated-learning performance relative to the optimal R-round lookahead policy.The guarantee holds for fixed R and M.
  • Performance Analysis: OCEAN retains an [O(1/V), O(⋯)] tradeoff despite using only current information rather than complete future information.The cited passage contrasts OCEAN with a T-lookahead benchmark having complete future information for all T rounds.

VI. SIMULATION RESULTS · A. Benchmarks

The simulations evaluate OCEAN in a 10-client, non-i.i.d. MNIST federated-learning network over 300 rounds. OCEAN is compared with Select-All, SMO, and AMO, alongside three temporal-importance variants.

  • VI. SIMULATION RESULTS: Simulations use TFF and MNIST, with each client representing an original digit writer and exhibiting non-i.i.d. local data.The first 10 clients are used, each with 100 training samples.
  • VI. SIMULATION RESULTS: The learning model is a three-layer neural network trained with FedAvg, with model size L = 3.4 × 105 bits.Its second layer contains 10 neurons and the third performs softmax classification.
  • VI. SIMULATION RESULTS: The wireless simulation uses OFDMA with total bandwidth B = 10 MHz and independent free-space-fading client channels.Average path loss is 36dB, and the complex white Gaussian noise variance is N0 = 10−12 W.
  • VI. SIMULATION RESULTS: Each round has a target upload time ¯τ = 300 ms, minimum bandwidth bmin = 2 × 105 Hz, and per-client energy budget Hk = 0.15 J.The network runs for T = 300 rounds.
  • A. Benchmarks: OCEAN is compared with three benchmarks: Select-All, SMO, and AMO.Select-All selects all 10 clients every round while allocating bandwidth to minimize total energy under the upload deadline.
  • A. Benchmarks: SMO independently optimizes client selection and bandwidth each round using currently available information, matching existing myopic approaches.It ranks clients by required bandwidth after allocating Hk/T energy and selects until total bandwidth exceeds B.
  • A. Benchmarks: AMO modifies SMO by recycling unused energy from unselected rounds for future rounds, addressing SMO’s energy under-utilization.OCEAN uses R = T and evaluates Ascending, Descending, and Uniform temporal-importance sequences.

B. Performance Comparison · C. Adaptability to Varying Network Condition

The comparison shows that OCEAN-a balances client selection and energy use while reproducing desirable temporal patterns, whereas AMO’s comparable performance is environment-dependent. Under varying channel conditions, OCEAN adapts through soft per-round energy allocation, while AMO develops idle periods that significantly degrade learning.

  • B. Performance Comparison: Select-All chooses all 10 clients each round, while SMO selects fewer because hard per-round energy budgets and bad channels prevent uploads.Select-All provides ideal client selection for FL, whereas SMO often leaves clients unselected.
  • B. Performance Comparison: AMO’s accumulated unused energy enables later transmissions under bad channels and produces an ascending client-selection pattern.OCEAN-a selects more clients than SMO without a hard per-round energy constraint.
  • B. Performance Comparison: OCEAN-a fine-tunes temporal selection with different η sequences: OCEAN-a increases, OCEAN-d decreases, and OCEAN-u nearly maintains selected-client counts.This demonstrates control over temporal client-selection patterns through temporal weights.
  • B. Performance Comparison: Select-All exceeds client energy budgets, SMO underuses them, while AMO and OCEAN-a consume close to the 0.15 budget per client.The comparison uses actual individual-client energy consumption after 300 learning rounds.
  • B. Performance Comparison: Select-All achieves the smallest training loss, highest accuracy, and fastest convergence; SMO performs worst, while AMO is comparable to OCEAN-a in this setting.AMO’s advantage does not extend to more complex network environments.
  • C. Adaptability to Varying Network Condition: In Scenario 1, AMO cannot sustain uploads as channels degrade, causing a long idle period and significantly worse learning performance than OCEAN.AMO resumes selecting clients only near the end, when its energy budget becomes sufficient.
  • C. Adaptability to Varying Network Condition: In Scenario 2, AMO initially selects hardly any clients under bad channels and begins selecting only after improvement, whereas OCEAN adapts in both scenarios.OCEAN’s soft per-round energy allocation keeps total consumption close to the total energy budget.

D. Features of OCEAN … VII. CONCLUSION

OCEAN adapts client selection and bandwidth allocation to channel conditions, energy deficit queues, and a tunable learning–energy tradeoff. The conclusion emphasizes that FL’s temporally dependent and unevenly significant rounds require a different wireless resource-allocation perspective, while identifying open questions about optimal selection patterns and client heterogeneity.

  • 2) Learning - Energy Tradeoff:: Figure 16 evaluates how OCEAN’s parameter V affects selected-client count, learning accuracy, and per-client energy-consumption violation.The experiment studies the learning-versus-energy tradeoff as a function of V.
  • 1) Client Selection and Bandwidth Allocation Outcomes:: Better channel conditions and larger energy deficit queues produce higher client selection priority, represented by smaller ρ values.OCEAN computes selection priority from current channel and energy states.
  • 1) Client Selection and Bandwidth Allocation Outcomes:: Among selected clients, OCEAN allocates more bandwidth to lower-priority clients with larger ρ values.Selection priority and bandwidth allocation therefore follow different ordering principles.
  • 2) Learning - Energy Tradeoff:: Larger V emphasizes learning performance, yielding more selected clients and higher accuracy.V controls the balance between learning objectives and energy considerations.
  • 2) Learning - Energy Tradeoff:: Smaller V emphasizes energy consumption and produces a smaller violation, if any, of the total energy budget.The parameter can therefore reduce energy-budget violations at the expense of learning emphasis.
  • VII. CONCLUSION: The paper argues that FL’s temporal dependency and varying learning-round significance should reshape wireless resource allocation for network and learning performance.This property is identified as largely overlooked in prior literature.
  • VII. CONCLUSION: The formulation and algorithm demonstrate superior performance in real-world FL experiments, while the optimal client selection pattern remains unclear.The paper reports that an ascending selection pattern is generally desired but does not establish the optimal pattern.
  • VII. CONCLUSION: Future work can incorporate computing-power and local-data heterogeneity to improve resource-allocation understanding in more complex WFLNs.The conclusion specifically identifies client computing power and local data size/distribution as extensions.

APPENDIX A PROOF OF LEMMA 1 · APPENDIX B PROOF OF THEOREM 1 · APPENDIX C PROOF OF PROPOSITION 1

The appendices establish the analytic properties of f(x), prove a thresholding structure and termination condition for the optimal solution, and derive the bandwidth-ordering result in Proposition 1. The proofs rely on derivative signs, exchange arguments, and monotonicity under bandwidth constraints.

  • APPENDIX A PROOF OF LEMMA 1: For x > 0, f′′(x) > 0 and f′(x) < 0, so f(x) is decreasing and convex.The proof uses the increasing behavior of f′(x) and limx→∞f′(x) = 0.
  • APPENDIX A PROOF OF LEMMA 1: For x < 0, f′′(x) < 0 and f′(x) > 0, so f(x) is increasing and concave.This follows from limx→−∞f′(x) = 0 and the sign of the second derivative.
  • APPENDIX B PROOF OF THEOREM 1: Theorem 1’s optimal solution has a thresholding structure: there exists k∗ such that a∗_k = 0 for all k > k∗.The proof swaps decisions for two clients and derives a contradiction to optimality.
  • APPENDIX B PROOF OF THEOREM 1: The termination condition is correct because the first client i∗ violating the condition cannot belong to the optimal selection set.The thresholding structure then excludes clients k = i∗ + 1, ..., K as well.
  • APPENDIX C PROOF OF PROPOSITION 1: With ρ1 < ρ2, any optimal allocation must satisfy b∗_1 ≤ b∗_2.If b∗_1 > b∗_2, swapping the allocations improves the objective because f is decreasing, contradicting optimality.
  • APPENDIX C PROOF OF PROPOSITION 1: The unconstrained bandwidth proof establishes that g1(x) ≜ f(x)/f′(x) is non-increasing for x > 0.It reduces the claim to proving g2(x) ≜ (f′(x))^2 − f(x)f′′(x) ≤ 0, using y = β/x and monotonicity arguments.
  • APPENDIX C PROOF OF PROPOSITION 1: Considering b1, b2 ≥ bmin preserves the bandwidth-ordering conclusion through separate cases, including allocations modified to the boundary bmin.The proof uses f(x) decreasing to compare the modified and original objective terms.

APPENDIX D PROOF OF THEOREM 2

The proof analyzes one-round and R-round Lyapunov drift under the queue dynamics and energy-deficit resets. It then shows that OCEAN-P maximizes a lower bound on weighted learning utility minus drift, yielding the comparison with the optimal R-round lookahead decisions.

  • Drift analysis: The proof defines the one-round drift as ∆1(t) ≜ L(q(t+1)) − L(q(t)) and the R-round drift as ∆R(t) ≜ L(q(t + R)) − L(q(t)).These quantities measure Lyapunov change over individual rounds and R-round frames.
  • Online optimization: OCEAN-P maximizes a lower bound of Vm·U(at) − ∆1(t) when solving P3.The bound is used to connect the online per-round decision to the long-term utility-drift objective.
  • Comparison with lookahead: The comparison step establishes that OCEAN-P’s decision sequence satisfies an inequality relative to the optimal decision solving the R-round lookahead problems.The proof retains qk(t) generated by OCEAN-P; its first inequality follows from lower-bound maximization, while the final inequality uses the reset condition.
Loading 2004.04314v1…