PaperlayerView exact revision on arXiv

Source-linked AI summary

Energy-Efficient Federated Edge Learning with Joint Communication and Computation Design

Xiaopeng Mo, Jie Xu

arXiv:2003.00199v1cs.ITeess.SP

TL;DR

Federated edge devices face finite battery energy and training-delay constraints. The paper jointly designs communication and computation, transforms the resulting problems into convex forms, and reports significant gains over schemes without joint optimization by balancing energy, speed, and accuracy.

  • Problem

    Finite device energy supplies and training-delay requirements motivate minimizing energy consumption at edge devices.

  • Method

    The paper jointly designs communication and computation, transforming the resulting optimization problems into convex forms with efficient optimal algorithms.

  • Results

    The proposed designs achieve significant performance gains over benchmark schemes without joint optimization.

  • Takeaways & Limitations

    Properly choosing global and local iterations balances energy consumption, training speed, and training accuracy.

Abstract

from arXiv · show

This paper studies a federated edge learning system, in which an edge server coordinates a set of edge devices to train a shared machine learning model based on their locally distributed data samples. During the distributed training, we exploit the joint communication and computation design for improving the system energy efficiency, in which both the communication resource allocation for global ML parameters aggregation and the computation resource allocation for locally updating MLparameters are jointly optimized. In particular, we consider two transmission protocols for edge devices to upload ML parameters to edge server, based on the non orthogonal multiple access and time division multiple access, respectively. Under both protocols, we minimize the total energy consumption at all edge devices over a particular finite training duration subject to a given training accuracy, by jointly optimizing the transmission power and rates at edge devices for uploading MLparameters and their central processing unit frequencies for local update. We propose efficient algorithms to optimally solve the formulated energy minimization problems by using the techniques from convex optimization. Numerical results show that as compared to other benchmark schemes, our proposed joint communication and computation design significantly improves the energy efficiency of the federated edge learning system, by properly balancing the energy tradeoff between communication and computation.

I. INTRODUCTION

Federated edge learning reduces cloud traffic and preserves device data privacy, but wireless communication, computation, and finite battery energy constrain training. This paper jointly optimizes communication and computation resources to minimize device energy while meeting training requirements.

  • Motivation and system: Federated edge learning coordinates edge devices using local data, avoiding explicit data sharing and preserving data privacy and security.The edge server broadcasts global parameters, devices perform local updates, upload parameters, and the server aggregates them.
  • Challenges: Wireless links constrain federated edge learning because parameter exchanges are frequent and channel conditions can fluctuate substantially.These constraints motivate an interdisciplinary design spanning machine learning and wireless communications.
  • Challenges: Battery-powered edge devices face limited energy supplies while large ML models impose heavy computation and communication loads.Training can also be limited by the slowest devices in communication and computation, known as the straggler’s dilemma.
  • Contributions: Jointly allocating communication and computation resources targets energy minimization for federated edge learning under training requirements.Communication allocation covers transmission power and rates, while computation allocation covers CPU frequencies for local updates.
  • Results: The proposed convex-optimization algorithms achieve significant gains over benchmark schemes without joint optimization.Numerical results also report tradeoffs among energy consumption, training speed, and training accuracy.
  • Results: Properly selecting global and local iteration counts further balances communication and computation energy to improve system energy efficiency.The paper uses simulations to examine how iteration counts affect training accuracy and the communication-computation energy tradeoff.

II. SYSTEM MODEL

The system uses distributed batch gradient descent in which an edge server coordinates local updates and parameter aggregation across edge devices. Training quality, delay, and energy depend on the numbers of global and local iterations, while the design focuses on device energy.

  • Federated training procedure: Each global iteration broadcasts server parameters, performs local device updates, uploads updated parameters, and aggregates them at the server.After M global iterations, the server uses the resulting global parameters as the solution.
  • Training tradeoffs: Larger global and local iteration counts generally improve accuracy but increase communication and computation energy consumption and training delay.The paper denotes these counts by M and N, respectively.
  • Training tradeoffs: Increasing local iterations can trade higher computation energy and delay for fewer global iterations and lower communication energy and delay.This creates a tradeoff among training speed, accuracy, and energy consumption.
  • Energy model scope: The system focuses on energy-hungry edge devices and omits server communication and computation energy and broadcast or aggregation delay.The optimization therefore models device-side costs during local updates and parameter uploads.

A. Local ML-Parameters Update at Edge Devices

Local ML-parameter updates are modeled through their FLOP workload and CPU frequency, with DVFS adapting computation to demand. The resulting CPU computation time and energy depend on device data, hardware, and operating frequency.

  • Computation workload: Each device performs N local updates, with a constant per-sample FLOP count a yielding total workload F_k = a×|D_k|.The workload is based on the number of locally distributed data samples.
  • CPU resource allocation: DVFS adjusts each device’s CPU frequency to match computation demand and reduce local-update energy consumption.The CPU frequency is bounded by the device’s maximum frequency.
  • Time and energy model: Local-update duration is determined by the FLOP workload, the number of FLOPs completed per CPU cycle, and the selected CPU frequency.The model treats CPU power as dependent on voltage and operating frequency, with voltage approximately linear in frequency.
  • Time and energy model: The device’s local-update energy is modeled through a chip-dependent coefficient that captures hardware architecture.The energy expression is derived from CPU consumption during local ML-parameter updates.

B. Local ML-Parameters Uploading from Edge Devices to Server

The paper models local-parameter uploading from edge devices to the server under NOMA, using channel, power, rate, and decoding-order choices to characterize communication delay and energy.

  • B. Local ML-Parameters Uploading from Edge Devices to Server: The uploading stage sends local ML-parameters from edge devices to the edge server for aggregation over repeated global iterations.The total training delay combines local-update and uploading delays across M global rounds and M × N local updates.
  • B. Local ML-Parameters Uploading from Edge Devices to Server: The channel model assumes quasi-static frequency-nonselective channels that remain unchanged throughout training.Signals are modeled as CSCG variables, with additive white Gaussian noise at the server.
  • 1) NOMA-Based Transmission:: Under NOMA, the server uses MMSE-SIC and a successive decoding order to decode devices’ simultaneously transmitted parameters.The decoding order determines which interference terms are canceled before each device is decoded.
  • 1) NOMA-Based Transmission:: NOMA achievable rates depend on the decoding order, transmission powers, channel gains, bandwidth, and Gaussian noise.Time-sharing across decoding orders gives the devices’ achievable rate region.
  • B. Local ML-Parameters Uploading from Edge Devices to Server: Each device must upload S bits within an optimized duration, with communication energy determined by its transmission power and uploading time.The resulting transmission constraints connect required bits, rates, durations, and per-device energy.

2) TDMA-Based Transmission:

The TDMA formulation allocates orthogonal uploading time, transmission power, rates, and CPU frequencies to minimize device energy under training-delay and local-computation constraints.

  • 2) TDMA-Based Transmission:: In TDMA, edge devices upload their updated local ML-parameters over orthogonal time intervals allocated separately to each device.Each device’s rate follows the single-user bandwidth formula, and its uploading energy depends on power and allocated duration.
  • 2) TDMA-Based Transmission:: The objective minimizes total edge-device energy subject to uploading, training-delay, and local-update computation constraints.The decision variables include communication resources and CPU frequencies, while global and local iteration counts are fixed.
  • 2) TDMA-Based Transmission:: Communication and computation exhibit an energy tradeoff: higher transmission power can shorten uploading and permit slower, less energy-intensive local computation.The design therefore jointly chooses transmission power, rates, and CPU frequencies.
  • 2) TDMA-Based Transmission:: The formulations are generally nonconvex because transmission power is coupled with uploading duration, making direct solution challenging.The paper fixes iteration counts while selecting resource allocations to balance energy and training accuracy.
  • 2) TDMA-Based Transmission:: The optimization is relevant only when devices can complete federated training within the prescribed delay T.The paper therefore checks feasibility before solving the energy-minimization problems.

A. Feasibility Checking for Problem (P1)

The paper checks feasibility by minimizing training duration under maximum CPU frequencies and transmission powers, then compares the resulting NOMA and TDMA delay capabilities.

  • A. Feasibility Checking for Problem (P1): The NOMA feasibility calculation reduces communication feasibility to maximizing the minimum common achievable rate among edge devices.The resulting rate problem is identified as optimally solved in prior work.
  • A. Feasibility Checking for Problem (P1): NOMA feasibility is determined by whether its minimum achievable training duration does not exceed the required delay T.The minimum-delay construction uses maximum CPU frequencies and transmission powers, while selecting decoding orders for uploading.
  • B. Feasibility Checking for Problem (P2): For TDMA, the minimum training duration is attained when all devices use maximum CPU frequency and maximum transmission power.Problem (P2) is feasible when this minimum duration is at most T and infeasible otherwise.
  • A. Feasibility Checking for Problem (P1): NOMA has a superior achievable rate region and therefore a lower minimum training delay than TDMA.Every TDMA-feasible resource allocation is also feasible under NOMA, although the converse need not hold.
  • A. Feasibility Checking for Problem (P1): The paper expects NOMA to consume less energy than TDMA because of its feasibility and rate-region advantages.This comparison is stated as an expectation to be validated through numerical results.

IV. OPTIMAL SOLUTION TO PROBLEM (P1) UNDER NOMA

The NOMA energy-minimization problem is transformed into an equivalent convex formulation using auxiliary transmission-energy and rate variables. The resulting convex set and objective enable optimization-based solution, although the formulation remains difficult because of its many inequality constraints.

  • A. Transformation of Problem (P1) into Convex Form: Auxiliary variables transform problem (P1) into the equivalent formulation (P1.1).The transformation introduces e_k = p_k t_k^up and s_k = r_k t_k^up.
  • A. Transformation of Problem (P1) into Convex Form: The perspective-function constraints make C(e, t^up) convex, so problem (P1.1) is a convex optimization problem.
  • A. Transformation of Problem (P1) into Convex Form: Standard interior-point methods remain impractical because constraint (28) represents 2K − 1 inequality constraints.
  • A. Transformation of Problem (P1) into Convex Form: The transformed problem can be solved when K is sufficiently large.

B. Optimal Solution to Problem (P1.1) or (P1)

The NOMA convex problem is solved through Lagrange duality, decomposition, and subgradient optimization. The procedure reconstructs optimal primal variables and uses time-sharing when the optimal rate allocation is non-unique.

  • B. Optimal Solution to Problem (P1.1) or (P1): Strong duality holds because problem (P1.1) is convex and satisfies Slater’s condition.
  • B. Optimal Solution to Problem (P1.1) or (P1): For fixed dual variables, the problem decomposes into K + 2 subproblems whose solutions are obtained using KKT conditions and convex optimization.
  • B. Optimal Solution to Problem (P1.1) or (P1): The dual variables are optimized with subgradient-based methods, such as the ellipsoid method, because the dual function is generally non-differentiable.
  • B. Optimal Solution to Problem (P1.1) or (P1): The resulting primal solution includes optimal CPU frequencies, transmission energies, upload times, rates, and decoding order.
  • B. Optimal Solution to Problem (P1.1) or (P1): Transmission power is unique, whereas rate allocation may be non-unique and may require time-sharing among decoding orders.

V. OPTIMAL SOLUTION TO PROBLEM (P2) UNDER TDMA

Under TDMA, problem (P2) is transformed into a convex formulation and solved through Lagrange duality. The method decomposes the problem, solves the resulting subproblems, and reconstructs the optimal primal variables.

  • V. OPTIMAL SOLUTION TO PROBLEM (P2) UNDER TDMA: Problem (P2) is transformed into the convex problem (P2.1) using auxiliary transmission-energy variables.
  • V. OPTIMAL SOLUTION TO PROBLEM (P2) UNDER TDMA: Lagrange duality decomposes (P2.1) into 2K + 1 subproblems whose solutions determine the dual function.
  • V. OPTIMAL SOLUTION TO PROBLEM (P2) UNDER TDMA: The subproblem for upload time is convex and can be solved from its first-order condition.
  • V. OPTIMAL SOLUTION TO PROBLEM (P2) UNDER TDMA: After optimizing the dual variables with the ellipsoid method, the algorithm reconstructs optimal CPU frequencies, upload times, and transmission energies.
  • V. OPTIMAL SOLUTION TO PROBLEM (P2) UNDER TDMA: The procedure finally solves problem (P2.1), and equivalently problem (P2).

VI. NUMERICAL RESULTS

Numerical results show that communication and computation should be jointly balanced: local iterations can replace global communication, while the preferred balance depends on channel distance and computation capacity. The proposed joint designs outperform benchmark schemes under both NOMA and TDMA, with NOMA supporting shorter training delays.

  • VI. NUMERICAL RESULTS: For 85% training accuracy, (M, N) = (50, 8), (30, 15), and (25, 20) are feasible, demonstrating a tradeoff between local computation and global communication.
  • VI. NUMERICAL RESULTS: NOMA always outperforms TDMA in energy consumption for the evaluated distance and iteration settings.When average distance is short, fewer local iterations are preferred; when distance exceeds 100m, more local iterations become preferable.

VII. CONCLUSION

The paper minimizes federated edge learning energy by jointly designing communication and computation resources under NOMA and TDMA. The resulting designs outperform schemes without joint optimization and reveal tradeoffs involving energy, training speed, accuracy, and iteration choices.

  • VII. CONCLUSION: NOMA and TDMA are the two transmission protocols considered for uploading ML parameters.
  • VII. CONCLUSION: The formulated non-convex problems are transformed into convex forms and solved optimally with efficient algorithms.
  • VII. CONCLUSION: Numerical results reveal tradeoffs among energy consumption, training speed, and training accuracy.
  • VII. CONCLUSION: Joint communication and computation designs achieve significant performance gains over benchmark schemes without joint optimization.The optimization jointly allocates communication and computation resources for federated edge learning.
  • VII. CONCLUSION: Properly choosing the numbers of global and local iterations further affects training accuracy and the communication-computation energy tradeoff.

APPENDIX A

The appendix establishes convexity and strong duality for a frequency-allocation subproblem, enabling characterization of its solution through KKT conditions.

  • APPENDIX A: For any feasible μ, subproblem (38) is convex in the CPU-frequency variables under linear frequency constraints.
  • APPENDIX A: Slater’s condition ensures strong duality between subproblem (38) and its dual problem.
  • APPENDIX A: The optimal primal and dual solutions are characterized using KKT conditions and their associated Lagrange multipliers.
Loading 2003.00199v1…