PaperlayerView exact revision on arXiv

Source-linked AI summary

Energy Efficient Federated Learning Over Wireless Communication Networks

Zhaohui Yang, Mingzhe Chen, Walid Saad, Choong Seon Hong, Mohammad Shikh-Bahaei

arXiv:1911.02417v2cs.ITcs.LGstat.ML

TL;DR

The paper addresses energy-efficient federated learning over wireless networks, where local computation and model transmission consume limited user energy. It jointly optimizes learning and communication resources and uses completion-time minimization to initialize the energy optimization. The proposed scheme achieves up to 59.5% energy reduction compared with conventional FL.

  • Problem

    The paper studies how to minimize total computation and transmission energy for federated learning over wireless networks under a latency constraint.

  • Method

    The paper proposes an iterative joint resource-allocation algorithm with closed-form updates and a bisection-based completion-time algorithm for obtaining a feasible initial solution.

  • Results

    59.5% energy reduction is achieved by the proposed scheme compared to the conventional FL method.

  • Takeaways & Limitations

    Joint optimization of computation and transmission resources reduces total FL energy consumption, especially at small maximum average transmit power.

Abstract

from arXiv · show

In this paper, the problem of energy efficient transmission and computation resource allocation for federated learning (FL) over wireless communication networks is investigated. In the considered model, each user exploits limited local computational resources to train a local FL model with its collected data and, then, sends the trained FL model to a base station (BS) which aggregates the local FL model and broadcasts it back to all of the users. Since FL involves an exchange of a learning model between users and the BS, both computation and communication latencies are determined by the learning accuracy level. Meanwhile, due to the limited energy budget of the wireless users, both local computation energy and transmission energy must be considered during the FL process. This joint learning and communication problem is formulated as an optimization problem whose goal is to minimize the total energy consumption of the system under a latency constraint. To solve this problem, an iterative algorithm is proposed where, at every step, closed-form solutions for time allocation, bandwidth allocation, power control, computation frequency, and learning accuracy are derived. Since the iterative algorithm requires an initial feasible solution, we construct the completion time minimization problem and a bisection-based algorithm is proposed to obtain the optimal solution, which is a feasible solution to the original energy minimization problem. Numerical results show that the proposed algorithms can reduce up to 59.5% energy consumption compared to the conventional FL method.

I. INTRODUCTION

The paper develops an energy-efficient resource-allocation scheme for federated learning over wireless networks, jointly optimizing local computation and model transmission. It also constructs a completion-time problem and bisection algorithm to obtain a feasible starting solution for energy minimization.

  • Privacy constraints and limited communication resources make transmitting all device data to a centralized data center impractical.
  • Federated learning lets wireless devices collaboratively train a shared model by uploading local models rather than their complete training data.
  • The paper formulates joint computation and transmission optimization to minimize total energy, deriving closed-form updates for time, bandwidth, power, computation frequency, and learning accuracy.
  • The considered FL procedure includes local computation, local-model transmission to the BS, and aggregation with broadcast of the global model.
  • A bisection-based completion-time minimization algorithm provides an optimal feasible solution for the original energy-minimization problem.
  • 59.5% energy reduction is achieved by the proposed jointly optimized scheme compared to the conventional FL method.

3) Information Broadcast:

The FL procedure trains local models from user data, exchanges model information with the BS, and balances learning accuracy against computation, communication, and energy costs.

  • The FL procedure includes local computation, uplink transmission of local results, BS aggregation, and downlink broadcasting, with downlink time neglected.
  • Each user’s energy consumption includes local computation energy and wireless transmission energy, while the total objective aggregates energy across users and global iterations.
  • FL performance is represented through task-specific loss functions, including linear and logistic regression examples.
  • Each user trains a local FL model using its dataset, while the BS aggregates local models into a global model for subsequent distribution.
  • The local optimization problem is solved iteratively with a target local accuracy η, requiring at least v log2(1/η) local iterations.
  • Under strong-convexity and smoothness assumptions, the global convergence rate depends on local accuracy η and parameters L, γ, ξ, and ǫ0.

C. Extension to Nonconvex Loss Function

For nonconvex local losses, the analysis replaces the original objectives with regularized versions that satisfy strong convexity and smoothness properties.

  • The nonconvex setting allows Hessian eigenvalues in [−γ, L] and replaces local and global losses with regularized versions.
  • The regularized local and global objectives are γ-strongly convex and (L + 2γ)-Lipschitz, allowing the earlier convergence analysis to apply.

III. RESOURCE ALLOCATION FOR ENERGY MINIMIZATION

The resource-allocation problem minimizes total user energy subject to FL latency, computation-capacity, transmit-power, and learning-accuracy constraints.

  • The optimization goal is to minimize total energy consumption of all users under a latency constraint.
  • The local accuracy variable satisfies 0 ≤ η ≤ 1.
  • The decision variables include time, bandwidth, computation frequency, transmit power, and learning accuracy.
  • Users’ computation capacities and average transmit powers are bounded by their respective maximum limits.

B. Iterative Algorithm

The proposed solver alternates between optimizing time and learning accuracy and updating bandwidth, computation frequency, and power, with a separate procedure providing a feasible start.

  • Each iteration first optimizes (t, η) with fixed (b, f, p), then updates (b, f, p) using the resulting time and accuracy.
  • The first subproblem derives optimal time and learning accuracy, using a fractional formulation solved through the Dinkelbach method.
  • The second subproblem decouples because its objective and constraints separate, enabling optimization of computation frequency and transmission variables.
  • Algorithm 3 initializes a feasible solution, alternates the two optimization steps, and stops when the objective converges.
  • The objective is nonincreasing and lower-bounded by zero, so the iterative algorithm converges to a local optimum.

C. Complexity Analysis

The paper develops a feasible-solution procedure for the energy minimization problem and analyzes the complexity of the proposed algorithms. The completion-time problem is solved by bisection, while the main energy algorithm scales linearly with the number of users.

  • Energy Minimization: Algorithm 3 uses closed-form resource-allocation updates within an iterative procedure for solving the total energy minimization problem.The updates cover time, bandwidth, power, computation frequency, and learning accuracy.
  • Energy Minimization: The conventional SCA method has complexity O(L_scaK^3), whereas Algorithm 3 grows linearly with the number of users K.Here, L_sca denotes the total number of SCA iterations.
  • Feasible Initialization: The completion-time minimization problem provides an initial feasible solution for the energy minimization problem when its optimum T* satisfies T* ≤ T.If T* > T, the original energy minimization problem is infeasible.
  • Feasible Initialization: Bisection obtains the globally optimal completion time despite the completion-time problem remaining nonconvex.Feasibility changes from infeasible below T* to feasible above T*, enabling bisection.
  • Feasible Initialization: For fixed T, completion-time minimization uses maximum computation capacity and maximum transmission power before reducing the problem to feasibility conditions.The resulting feasibility set is characterized through lemmas and a theorem involving the learning accuracy η* and convexity of the associated function.
  • Feasible Initialization: The optimal learning accuracy η* balances local computation and transmission time because very small η increases local iterations, while very large η increases global iterations.The optimum satisfies the first-order condition Σ_k(η*) = 0 and is found using bisection.

B. Complexity Analysis

The completion-time algorithm has low complexity because its bisection-based feasibility checks scale linearly with the number of users. The paper also evaluates loss as total computations vary under IID and pathological non-IID data partitions.

  • Complexity: Algorithm 4 has total complexity O(K log2(1/ε4) log2(1/ε5)), which grows linearly with the number of users K.The two logarithmic factors correspond to inner and outer bisection accuracies ε4 and ε5.
  • Complexity: At each iteration, Algorithm 4 primarily checks the feasibility condition using an inner bisection to obtain the optimal learning accuracy.The inner feasibility check has complexity O(K log2(1/ε4)).
  • Simulation Setup: Figure 4 compares loss-function values against total computations for FL algorithms using IID and non-IID data.The non-IID setup assigns each device two shards after sorting samples by digit label, creating a pathological partition in which most devices receive two digit classes.

V. NUMERICAL RESULTS

Numerical experiments evaluate convergence, completion time, energy consumption, and resource-allocation tradeoffs across learning and wireless baselines. The proposed scheme consistently achieves favorable completion-time and energy results, including up to 59.5% energy reduction at equal completion time.

  • Simulation setup: The experiments use 50 users, a 500 m × 500 m area, 20 MHz bandwidth, 2 GHz computation capacity, and 500 samples per user.Statistical results are averaged over 1000 independent runs using the real open blog feedback dataset.
  • Convergence behavior: Non-IID FL shows similar convergence behavior to IID FL, while multiple local updates require more total computations than one local update.The non-IID partition is described as pathological because most devices receive only two digit classes.
  • Convergence behavior: The upper-bound gap for required iterations decreases as accuracy ξ0 decreases, indicating a tight bound for small ξ0.This comparison is reported for the exact iteration count and the upper bound derived in Theorem 1.
  • Completion time: The proposed FL scheme achieves the best completion-time performance among the proposed comparisons and reduces completion time versus TDMA by up to 27.3%.The comparison includes EB-FDMA, FE-FDMA, and TDMA; the proposed scheme jointly optimizes bandwidth and local accuracy η.
  • Completion time: Communication and computation times both decrease with maximum average transmit power, but computation time remains larger and communication time decreases faster.The result is reported in Fig. 8.
  • Total energy: Communication energy increases while computation energy decreases with maximum average transmit power, with FDMA–TDMA energy advantages changing across power ranges.Below 15 dB, FDMA outperforms TDMA for both energy components; above 17 dB, the advantage differs by component.
  • Total energy: At equal completion time, the proposed FL reduces energy by up to 12.8%, 53.9%, and 59.5% versus EB-FDMA, FE-FDMA, and random selection, respectively.The proposed scheme also outperforms the comparison schemes in total energy, while EXH-FDMA achieves almost the same performance.

APPENDIX A PROOF OF LEMMA 1

The appendix establishes convergence-related inequalities under the paper’s assumptions and uses them to derive the stated bound for the objective decrease.

  • Proof of Lemma 1: Under assumptions A1 and A2, the appendix establishes gradient and Hessian relationships used in the convergence analysis.The proof invokes the Lagrange median theorem, first-order optimality, and the relation 1 − x ≤ exp(−x).
  • Proof of Lemma 1: The derived inequalities imply a bound ensuring F(w(n+1))−F(w∗) ≤ ε0(F(w(0))−F(w∗)).The appendix states that condition (17) ensures this objective-gap relation.

APPENDIX C PROOF OF THEOREM 2

The appendix derives structural properties of the resource-allocation subproblems, including optimal transmission time, bandwidth, and power choices.

  • Proof of Theorem 2: Transmitting with minimal time is always energy efficient, so the optimal time allocation uses the minimum feasible transmission time.The appendix states this conclusion directly from problem (20).
  • Proof of Theorem 2: The optimal transmit power is obtained by minimizing power subject to the transmission constraint, with its derivatives used in the proof.The appendix introduces the power minimization step through problem (28a).
  • Proof of Theorem 2: The bandwidth subproblem is convex, and its bandwidth constraint holds with equality at the optimum.The result follows from the objective being decreasing in bandwidth and the KKT conditions.

APPENDIX E PROOF OF LEMMA 2

The appendix proves feasibility and optimality properties for the completion-time formulation by comparing candidate completion times and reducing the bandwidth constraints.

  • Proof of Lemma 2: A completion time smaller than the optimum would yield a feasible solution with a lower objective, contradicting optimality.The contradiction argument applies to any feasible T̄ < T∗.
  • Proof of Lemma 2: For any completion time greater than the optimum, the optimal solution at T∗ remains feasible.The appendix constructs the feasible solution using the optimal allocation associated with T∗.
  • Proof of Lemma 2: The bandwidth feasibility condition is reduced using an increasing right-hand side and a longest-time allocation, with equality at the optimum.This reduction produces problem (37) after substituting the optimal time allocation.

APPENDIX G PROOF OF LEMMA 4

Appendix G proves that v_k(η) is convex and that u_k(η), the inverse of z_k(η), is increasing and convex. It then establishes convexity of the composition u_k(v_k(η)).

  • Convexity of v_k(η): v_k(η) is shown to be convex by evaluating its second-order derivative and establishing v_k''(η) ≥ 0.The proof derives v_k(η) from φ(ϕ_k(η)) and combines the preceding derivative relations.
  • Properties of u_k(η): z_k(η) is increasing and concave, so its inverse u_k(η) is increasing.The inverse relation is u_k(z_k(η)) = η.
  • Properties of u_k(η): The proof establishes that u_k(η) is convex using the defining inequality for concavity of z_k(η) and the inverse-function relation.Applying the increasing function u_k to the concavity inequality yields the convexity inequality for u_k.
  • Convexity of the composition: Because the relevant nonnegative condition holds, the composition u_k(v_k(η)) is convex.This conclusion follows after combining the convexity of v_k(η), the properties of u_k, and condition (G.10).
Loading 1911.02417v2…