PaperlayerView exact revision on arXiv

Source-linked AI summary

A Joint Learning and Communications Framework for Federated Learning over Wireless Networks

Mingzhe Chen, Zhaohui Yang, Walid Saad, Changchuan Yin, H. Vincent Poor, Shuguang Cui

arXiv:1909.07972v4cs.NIcs.LGstat.ML

TL;DR

Wireless links introduce training errors through limited resources and inherent unreliability. The paper develops a joint framework that formulates learning, user selection, and resource allocation together, derives an expected convergence rate, and reports improved performance over implementations that do not account for wireless-channel properties.

  • Problem

    Wireless training parameters face errors from limited wireless resources and unreliable links.

  • Method

    The framework jointly considers user selection and resource allocation, using a closed-form expected convergence rate to minimize FL training loss.

  • Results

    The joint federated learning and communication framework yields significant performance improvements over FL implementations that do not account for wireless-channel properties.

  • Takeaways & Limitations

    The framework enables implementation of FL algorithms over wireless networks while incorporating wireless-channel effects into training design.

Abstract

from arXiv · show

In this paper, the problem of training federated learning (FL) algorithms over a realistic wireless network is studied. In particular, in the considered model, wireless users execute an FL algorithm while training their local FL models using their own data and transmitting the trained local FL models to a base station (BS) that will generate a global FL model and send it back to the users. Since all training parameters are transmitted over wireless links, the quality of the training will be affected by wireless factors such as packet errors and the availability of wireless resources. Meanwhile, due to the limited wireless bandwidth, the BS must select an appropriate subset of users to execute the FL algorithm so as to build a global FL model accurately. This joint learning, wireless resource allocation, and user selection problem is formulated as an optimization problem whose goal is to minimize an FL loss function that captures the performance of the FL algorithm. To address this problem, a closed-form expression for the expected convergence rate of the FL algorithm is first derived to quantify the impact of wireless factors on FL. Then, based on the expected convergence rate of the FL algorithm, the optimal transmit power for each user is derived, under a given user selection and uplink resource block (RB) allocation scheme. Finally, the user selection and uplink RB allocation is optimized so as to minimize the FL loss function. Simulation results show that the proposed joint federated learning and communication framework can reduce the FL loss function value by up to 10% and 16%, respectively, compared to: 1) An optimal user selection algorithm with random resource allocation and 2) a standard FL algorithm with random user selection and resource allocation.

I. INTRODUCTION

The paper addresses federated learning over wireless networks, where unreliable links and limited resources affect training quality, convergence, and user participation. It proposes jointly optimizing learning and communication decisions, with simulations showing improved performance over several baselines.

  • Motivation: Wireless packet errors and limited bandwidth can introduce training errors and constrain the number of participating FL users.These constraints motivate selecting an appropriate user subset and accounting for wireless conditions.
  • Prior work: Prior work often treated wireless networks as readily compatible with FL or omitted packet transmission errors and joint user-selection optimization.The paper identifies these omissions as limitations of existing communication-oriented FL studies.
  • Framework: The proposed framework jointly models FL and wireless factors, including user selection, resource allocation, transmit power, packet errors, and training loss.The optimization objective minimizes training loss while meeting delay and energy requirements.
  • Optimization: The solution derives an expected convergence-rate expression, obtains transmit power under fixed selections and RB allocations, and solves FL-aware user selection and RB allocation through bipartite matching.The matching formulation is solved using the Hungarian algorithm.
  • Results: 1.4%, 3.5%, and 4.1% improvement in identification accuracy are reported against random resource allocation, standard FL with random decisions, and wireless optimization minimizing aggregate packet error rates.The comparison targets progressively different baselines and preserves the paper’s reported ordering.

II. SYSTEM MODEL AND PROBLEM FORMULATION

The system consists of wireless users and a base station that collaboratively train a shared FL model without transferring training data. Users send local models uplink, while the BS aggregates and broadcasts the global model downlink.

  • System model: A cellular network with one BS and multiple users supports cooperative FL while keeping training data on user devices.The BS cannot collect all distributed data for centralized learning.
  • Learning procedure: Users train local FL models from their collected data, and the BS integrates them into a shared global FL model.The global model is then used to improve users’ local models.
  • Communication procedure: The uplink transmits local FL model parameters to the BS, while the downlink broadcasts global FL model parameters to users.This bidirectional exchange implements the FL learning cycle.

A. Machine Learning Model

The model formalizes local and global FL objectives together with wireless transmission rates and delays. It links learning performance to channel conditions, interference, bandwidth, transmit power, and resource-block assignments.

  • Machine learning model: Each user’s local model is trained from its input and output data, while the global model is constrained to equal the users’ converged models.The loss function captures task performance, such as prediction or classification accuracy.
  • Learning procedure: FL alternates local model updates and BS aggregation, with users transmitting local models and receiving updated global parameters.The procedure can use gradient descent, stochastic gradient descent, or randomized coordinate descent for local updates.
  • Wireless impact: Wireless channel errors can affect received local models, while transmission delay affects FL convergence, requiring joint consideration of learning and wireless performance.The paper explicitly connects wireless transmission conditions to FL performance.
  • Transmission model: Uplink rates depend on RB assignments, user transmit power, channel gain, bandwidth, noise, and interference from other service areas.The model uses OFDMA, with each user occupying one RB in the stated transmission technique.
  • Transmission model: Transmission delays are obtained from uplink and downlink rates and the bit sizes of local and global model vectors.The model assumes equal local and global model transmission sizes.

C. Packet Error Rates

The packet-error model treats each local FL model as a single packet and excludes erroneous received models from BS aggregation. Packet errors depend on user selection, RB allocation, and transmit power.

  • Packet handling: Each local FL model is transmitted as a single packet, and the BS does not request retransmission when errors occur.Instead, it uses the remaining correct local models for the global update.
  • Error detection: A CRC mechanism identifies data errors in received local FL models before aggregation.Correct packets have C(w_i)=1, while erroneous packets have C(w_i)=0.
  • Packet-error model: The packet error rate for each local model is determined from its uplink transmission configuration.The formulation expresses packet errors through RB allocation and transmit power.
  • Error-aware aggregation: The global FL model explicitly incorporates packet errors and depends on the user-selection vector, RB allocation matrix, and transmit-power vector.Erroneous local models are excluded from the BS update.

D. Energy Consumption Model

The paper models user energy consumption for local FL training and model transmission, then embeds energy, delay, power, user-selection, and RB constraints into a joint optimization problem. Its convergence analysis links packet errors, user selection, and wireless allocation to the FL loss.

  • Energy model: Each user's energy consumption includes local FL model training and transmission to the BS.The BS's energy consumption is excluded because it has a continuous power supply.
  • Energy model: Local-model training energy depends on CPU frequency, required CPU cycles per bit, a device energy coefficient, and local-model size.The training component is expressed using ςω_iϑ^2Z(w_i).
  • Problem formulation: The optimization jointly chooses user selection, uplink RB allocation, and transmit power to minimize FL training loss under wireless constraints.The constraints include binary decisions, delay, energy, one-RB-per-user and one-user-per-RB restrictions, and maximum transmit power.
  • Wireless impact: Transmit power and resource allocation determine packet error rates, which affect global-model updates and the FL loss.The selected variables remain fixed during the FL training process and must satisfy delay and energy requirements at every learning step.
  • Convergence analysis: The convergence bound contains a gap between expected loss at the current global model and the ideal loss, caused by packet errors and user selection.Lower packet error rates and more participating users reduce the gap and improve convergence speed.
  • Scope: The framework can extend to models transmitted over multiple packets by replacing the single-packet error rate with the error rate for transmitting the entire model.The assumptions cover several common loss functions, including mean squared error, logistic regression, and cross entropy; future work considers non-convex losses.

IV. OPTIMIZATION OF FL TRAINING LOSS

The FL training-loss problem is simplified using the convergence analysis, focusing on the asymptotic regime where the convergence factor satisfies A < 1. The resulting formulation accounts for packet errors, user selection, and wireless constraints.

  • Optimization objective: The objective is to minimize FL training loss while accounting for wireless network constraints.The formulation includes transmit power allocation and resource allocation for users.
  • Convergence regime: The analysis focuses on asymptotic convergence behavior for sufficiently large t and restricts attention to the regime A < 1.Under this condition, the convergence term becomes zero and the FL algorithm converges.
  • Wireless terms: When users are selected, their packet error rates enter the simplified objective, while unselected users have zero packet error contribution.This yields a_iq_i(r_i,P_i)=q_i(r_i,P_i) for selected users.
  • Solution structure: The optimization first determines optimal transmit power for a given uplink RB allocation, then optimizes the RB allocation.The formulation retains the stated delay, energy, and allocation constraints.

A. Optimal Transmit Power

For a fixed uplink RB allocation, the paper derives each user's optimal transmit power in closed form. The resulting power depends on local-model size and RB interference, with larger models increasing training loss through energy consumption.

  • Power optimization: For a given uplink RB allocation vector, Proposition 2 gives the optimal transmit power for each user.The power is defined by P_i*(r_i)=min{P_max,P_i,γE}.
  • Power dependencies: The optimal transmit power depends on the local FL model size and interference in each RB.These factors affect the energy available for model transmission.
  • Energy trade-off: As local-model size increases, users spend more energy training the model, leaving less energy for data transmission.The paper states that this increases the training loss.

B. Optimal Uplink Resource Block Allocation

The paper converts uplink RB allocation into a bipartite matching problem between users and RBs. Edge weights encode feasible allocations, and the Hungarian algorithm finds the allocation used with optimized power and user selection.

  • Graph construction: The reduced optimization problem is formulated as a bipartite matching problem over users, RBs, and connecting edges.Each edge represents a possible user-RB allocation.
  • Solution method: The Hungarian algorithm solves the matching problem without requiring variable gradients or dynamically adjusted step sizes.The paper contrasts this with traditional convex optimization algorithms.
  • Matching constraints: A matching prevents two edges from sharing an RB, so each RB is allocated to at most one user.The formulation also restricts each user to occupying only one RB.
  • Feasibility: An edge is assigned zero weight when its allocation cannot satisfy delay and energy requirements.Such an RB is therefore not allocated to that user.
  • Optimization: The objective is to find a matching that minimizes the weights of its selected edges.The matching determines the optimal RB allocation.
  • End-to-end procedure: The complete procedure analyzes convergence, computes per-RB optimal powers, solves the matching problem, and runs FL with the resulting allocation and selection.The algorithm uses the optimal RB allocation matrix, user-selection vector, and transmit-power vector.

C. Implementation and Complexity

The BS computes wireless transmission and device-related quantities, then uses the Hungarian algorithm to optimize user–RB matching. Simulations evaluate this framework for linear regression and handwritten digit identification under specified network settings and baselines.

  • Implementation: The BS estimates packet error rates from SINR and FL-model size, while model size depends on the learning task.Channel estimation provides SINR values for each RB.
  • Implementation: Given packet errors, delay, and energy consumption, the BS computes the matching weight ψin for each user–RB pair.The required device information, including CPU characteristics, can be learned when users initially connect to the BS.
  • Complexity: Because the optimization objective is linear, the Hungarian algorithm finds an optimal user–RB matching set T∗.The algorithm updates edge values to obtain the matching.
  • Complexity: The Hungarian algorithm has worst-case complexity O(U^2R) and best-case complexity O(UR).Its main computational burden is calculating edge weights and updating matching edges.
  • Baselines: The comparison includes random resource allocation, random user selection and allocation, and wireless optimization based on total packet-error reduction.The implementation code is publicly available.

SYSTEM PARAMETERS

The simulations examine how data samples, users, RBs, iterations, and wireless errors affect FL performance. Across the evaluated tasks, joint learning–communication optimization improves training or identification outcomes relative to the stated baselines.

  • Linear regression: As data samples per user increase, all considered FL algorithms reduce their loss, with rapid improvement below 30 samples and little change afterward.The plateau is attributed to having enough samples to approximate the loss gradient.
  • Linear regression: As the number of users increases, the Hungarian algorithm requires more iterations to find optimal RB allocation.The increase is attributed to a larger edge-weight matrix; growth is especially pronounced when users exceed the RB count.
  • Linear regression: With more than 10 users, a 10-RB network requires more Hungarian iterations than a 15-RB network.The difference follows from the larger user–RB mismatch in the 10-RB case.
  • Wireless effects: Theorem 1 agrees with simulation results within 9%, while the convergence-gap value increases as the number of users grows.The increase is linked to a higher probability that users cannot perform FL.
  • Handwritten digit identification: Identification accuracy increases with iterations and users, but its improvement slows at higher user counts; it also rises with RBs before leveling off.Packet errors can make the accuracy increase uneven across iterations because usable local models vary.
  • Handwritten digit identification: For 9 RBs, the proposed method achieves up to 1.4%, 3.5%, and 4.1% identification-accuracy gains over baselines a), b), and c).The method optimizes RB allocation, transmit power, and user selection.

VI. CONCLUSION

The paper develops a wireless-aware FL framework that jointly optimizes learning and communication decisions. It derives convergence and power-control results, then uses Hungarian matching for user selection and RB allocation to reduce FL loss.

  • Contribution: The framework enables FL over wireless networks by jointly considering user selection and resource allocation to minimize FL training loss.Its stated purpose is to account for wireless-channel properties in FL implementation.
  • Method: The paper derives a closed-form expected convergence-rate expression that incorporates wireless-medium limitations.This expression quantifies the relationship between wireless factors and FL convergence.
  • Method: Given user selection and uplink RB allocation, the framework determines optimal transmit power and then uses the Hungarian algorithm for optimal matching.The matching minimizes the FL loss function.
  • Results: Simulation results show significant improvements from the joint federated learning and communication framework.The supplied conclusion states the improvement but does not provide a numerical value in these passages.

APPENDIX

The appendix develops the convergence proof and transmit-power proposition used by the framework. It relates successful local-model transmission and packet-error behavior to the expected optimization process and power choice.

  • Convergence proof: The proof separates users that correctly transmit local models from users outside that set.N1 contains correctly transmitting users, while N2 contains the remaining users.
  • Transmit power: The transmit-power proof shows that packet-error rates decrease as transmit power increases.It uses this monotonicity together with the energy behavior to establish the optimal-power proposition.
  • Transmit power: User energy consumption is monotonically increasing in transmit power when Pi > 0.The appendix establishes positivity of the derivative and uses contradiction in proving the proposition.
Loading 1909.07972v4…