PaperlayerView exact revision on arXiv

Source-linked AI summary

Energy-Efficient Resource Management for Federated Edge Learning with CPU-GPU Heterogeneous Computing

Qunsong Zeng, Yuqing Du, Kaibin Huang, Kin K. Leung

arXiv:2007.07122v2cs.ITeess.SP

TL;DR

The paper addresses energy-efficient implementation of federated edge learning in wireless systems through joint management of computation-and-communication resources. The supplied passages highlight an energy-learning tradeoff and assume constant per-round latency in the convergence analysis.

  • Problem

    Energy-efficient implementation of federated edge learning in wireless systems requires joint management of computation-and-communication resources at devices.

  • Method

    The paper uses joint management of computation-and-communication resources at devices, alongside frequency scaling and federated-learning updates.

  • Results

    The supplied passages report that reducing total energy consumption can increase learning latency, and vice versa.

  • Takeaways & Limitations

    Energy-efficient federated learning involves an energy-learning tradeoff in which lower energy consumption may come at the cost of increased learning latency.

  • Takeaways & Limitations

    The convergence analysis assumes that per-round latency T is constant and omits it from the bound.

Abstract

from arXiv · show

Edge machine learning involves the deployment of learning algorithms at the network edge to leverage massive distributed data and computation resources to train artificial intelligence (AI) models. Among others, the framework of federated edge learning (FEEL) is popular for its data-privacy preservation. FEEL coordinates global model training at an edge server and local model training at edge devices that are connected by wireless links. This work contributes to the energy-efficient implementation of FEEL in wireless networks by designing joint computation-and-communication resource management ($\text{C}^2$RM). The design targets the state-of-the-art heterogeneous mobile architecture where parallel computing using both a CPU and a GPU, called heterogeneous computing, can significantly improve both the performance and energy efficiency. To minimize the sum energy consumption of devices, we propose a novel $\text{C}^2$RM framework featuring multi-dimensional control including bandwidth allocation, CPU-GPU workload partitioning and speed scaling at each device, and $\text{C}^2$ time division for each link. The key component of the framework is a set of equilibriums in energy rates with respect to different control variables that are proved to exist among devices or between processing units at each device. The results are applied to designing efficient algorithms for computing the optimal $\text{C}^2$RM policies faster than the standard optimization tools. Based on the equilibriums, we further design energy-efficient schemes for device scheduling and greedy spectrum sharing that scavenges "spectrum holes" resulting from heterogeneous $\text{C}^2$ time divisions among devices. Using a real dataset, experiments are conducted to demonstrate the effectiveness of $\text{C}^2$RM on improving the energy efficiency of a FEEL system.

I. INTRODUCTION

The paper addresses energy-efficient FEEL on wireless networks with CPU-GPU heterogeneous devices by jointly managing computation and communication resources. It extends prior single-processor or radio-focused approaches to coordinate workload partitioning, frequency scaling, and radio resource management across multiple devices.

  • Motivation: FEEL distributes model training across wireless edge devices, but complex learning tasks challenge energy- and resource-constrained devices.Each learning round involves global-model broadcasting, local-update computation, and local-update uploading for server aggregation.
  • Contribution: The paper proposes energy-efficient joint management of computation-and-communication resources for FEEL devices capable of heterogeneous computing.The target architecture combines CPU and GPU processing, which prior experiments associate with improved performance and energy efficiency.
  • Contribution: The framework jointly controls bandwidth allocation, workload partitioning, frequency scaling, and C2 time division across multiple devices.Unlike prior work focused on single-processor devices or CPU-only DVFS, the paper integrates CPU-GPU control with radio resource management.
  • Motivation: Communication-efficient FEEL manages finite radio resources because uploading high-dimensional local updates can overwhelm the wireless interface.Existing approaches include device scheduling, customized multi-access technologies, and uploading-frequency optimization.
  • CPU-GPU Heterogeneous Computing: Workload partitioning assigns a task across integrated CPU and GPU processors according to task requirements, processor states, speeds, and energy efficiencies.The paper studies this control jointly with DVFS in a multi-device FEEL system and integrates it with radio resource management.

C. Contributions

The paper develops energy-efficient FEEL resource management for CPU-GPU heterogeneous devices, jointly controlling computation, communication, scheduling, and spectrum use. Its contributions include equilibrium-based optimization, C2-aware scheduling, and greedy spectrum sharing.

  • The work targets energy-efficient FEEL without introducing a new learning technique, focusing instead on resource management for heterogeneous devices.
  • Equilibrium based C2RM framework: The C2RM framework jointly controls bandwidth allocation, C2 time division, CPU-GPU workload partitioning, and CPU-GPU frequency scaling.
  • Equilibrium based C2RM framework: Optimal policies are characterized by equal energy-workload, energy-time, and energy-bandwidth rates, with CPU-GPU speeds proportional to corresponding computation requirements.
  • Equilibrium based C2RM framework: The joint C2RM problem is decomposed into one master problem and two sub-problems, producing a lower-complexity algorithm than block coordinate descent.
  • C2 aware scheduling: C2-aware scheduling selects a fixed number of devices using a metric balancing channel state and computation capacity, while analyzing scheduled-device effects on convergence.
  • Greedy spectrum sharing: Greedy spectrum sharing assigns arriving spectrum holes to available devices using energy-bandwidth acceleration rates to improve energy efficiency.

B. Model of Heterogeneous Computing

The model represents local computation and gradient uploading on CPU-GPU heterogeneous devices, linking workload, processor speed, power, latency, channel, and transmission-energy variables. It assumes fixed per-round bandwidth, slow block fading, synchronized updates, and reliable transmissions.

  • Workload model: A computation task has workload W = NFLOP × D, where D is local dataset size and NFLOP is the FLOP count per sample.
  • Workload model: CPU and GPU computing speeds are determined by their clock frequencies and respective FLOPs per cycle.
  • Workload partitioning model: Input-sample partitioning assigns CPU and GPU workload portions Wk and W′k whose sum equals the total workload.
  • Power model: Processor power depends on clock frequency, and frequency scaling controls CPU-GPU power by adjusting their speeds.
  • Power model: The GPU generally plays the main role because Gk < Ck, while the CPU supplies supplementary computation resources.
  • Communication model: Gradient uploading uses FDMA with per-round bandwidth Bk, channel-dependent achievable rate rk, gradient size L, and transmission time tk.
  • Communication model: Bandwidth is fixed throughout each round, channels use slow block fading, and each device’s channel gain remains unchanged within a round.

D. Performance Metric

The performance metric is sum energy: total energy consumed by active devices in the learning process. The resource-management design minimizes this energy per round under latency and learning-performance constraints, using CPU-GPU and communication equilibriums.

  • The objective minimizes total energy consumption of all active devices over the N-round learning process, equivalently minimizing per-round total energy.
  • The framework considers computation resource management, communication resource management, and joint C2RM for establishing an energy-learning tradeoff.
  • Computation RM: Computation management controls workload partitioning and processor speed scaling to minimize sum computation energy.
  • Computation RM: Optimal computation uses equal CPU-GPU computation times, preventing the slower processor from becoming the local-gradient bottleneck.
  • Computation RM: The optimal speed scaling is workload-proportional, and the resulting policy is given in closed form.
  • Computation RM: The CPU-GPU optimum equalizes energy-workload rates, while more workload is allocated to the processor with the smaller computation coefficient.

B. Communication Resource Management

The communication resource-management problem allocates bandwidth and transmission time to minimize devices’ sum transmission energy. Energy-bandwidth rate equilibria yield optimal bandwidth policies and faster-than-conventional computation methods.

  • Problem formulation: Bandwidth allocation and transmission time are jointly controlled to minimize the sum transmission energy of devices.The resulting optimization problem has a convex structure and can be solved by standard methods such as BCD.
  • Optimal policy: The optimal bandwidth allocation equalizes the energy-bandwidth rates across devices.This equilibrium directly determines the optimal bandwidth policy.
  • Optimal policy: More bandwidth should be allocated to devices with weaker channels to reduce sum communication energy.The optimal bandwidth is non-increasing with the channel-gain term h2_k.
  • Algorithm: The proposed iterative algorithm alternates bandwidth optimization with energy-bandwidth-rate updating until convergence.The computed bandwidths are normalized when an intermediate rate value does not satisfy the bandwidth constraint.
  • Complexity and optimality: Algorithm 1 has complexity O(log 1/ε), lower than the DFO method, while Lemma 4 guarantees optimality.The DFO computation is reported as much higher complexity than Algorithm 1.

C. Joint C2 Resource Management

The joint C2RM framework minimizes sum energy by controlling workload partitioning, C2 time division, and bandwidth allocation. Its optimal policy is characterized by energy-rate equilibriums spanning communication, computation, workload, and speed controls.

  • Framework: C2RM jointly controls workload partitioning, C2 time division, and bandwidth allocation to minimize sum energy.The formulation combines computation and communication resource-management decisions under joint constraints.
  • C2 time division: The optimal C2 time division equalizes energy-computation-time and energy-communication-time rates for each device.The division satisfies the device time constraint t′_k + t_k = T.
  • Equilibrium conditions: The optimal bandwidth allocation equalizes energy-bandwidth rates across devices.This is the second equilibrium in the main C2RM result.
  • Equilibrium conditions: Optimal workload allocation and speed scaling equalize energy-workload and energy-speed rates at each device.These equilibriums extend the resource-balancing principle to CPU-GPU computation controls.
  • Interpretation: The equilibriums equalize heterogeneity in communication channels, CPU-GPU computation efficiencies, and C2 speeds through multidimensional controls.Communication resources can compensate for computation-resource shortages and vice versa through energy-rate tradeoffs.

3) Optimal Policy Computation:

The paper develops iterative algorithms for computing optimal C2RM policies by decomposing the convex problem into a master problem and two subproblems. It also analyzes how sum energy trades off against learning latency and participating-device count.

  • Optimal Policy Computation: Algorithm 2 improves efficiency by using equilibrium-based subproblem solutions and a progressively better initialization of ν.The initialization approaches the optimal solution as outer iterations proceed.
  • Optimal Policy Computation: The optimal C2RM computation is decomposed into a master problem for C2 time division and two resource-management subproblems.Iterations solve the subproblems, update the master variables by gradient descent, and continue until convergence.
  • Optimal Policy Computation: The computation includes closed-form solution of Problem P1 and numerical solution of Problem P3 using Algorithm 1.The resulting master and subproblem iterations update time division, bandwidths, and workloads.
  • Energy-Learning Tradeoff: Learning latency is defined as Ttotal = N × T, linking total training time to the number of rounds and per-round latency.The paper examines this relation under fixed participating-device count and fixed per-round latency.
  • Energy-Learning Tradeoff: Reducing sum energy can increase learning latency, and shortening learning latency requires more energy.With fixed participating devices, latency reduction raises sum energy faster than linearly; with fixed per-round latency, lower energy can reduce participating devices and increase required rounds.
  • Scope: The functions T(EΣ) and N(EΣ) have no closed form, and analyzing their properties is outside the scope of the work.The paper identifies scaling-law analysis as an open topic rather than deriving those properties.

IV. C2 AWARE SCHEDULING

C2-aware scheduling selects a low-complexity subset of devices using computation and communication energy metrics. It evaluates devices under equal bandwidth and selects those with the smallest estimated energy consumption.

  • Scheduler Design: Selecting only a subset of devices can reduce energy when many devices provide sufficient data.With M fixed and i.i.d. data distributions, scheduling does not affect the number of communication rounds required for learning.
  • Scheduler Design: The scheduling problem is NP-hard, so the proposed C2-aware scheduler targets low complexity instead of classical optimal methods.Branch-and-bound methods are described as impractical when K is large.
  • Algorithm: The resulting scheme is called C2-aware scheduling because its decisions use the C2 states of devices.Algorithm 3 initializes equal bandwidth, computes the metrics, and sets the selection indicators for the chosen devices.
  • Scheduling Metric: C2-aware scheduling uses device energy under equal bandwidth as a metric reflecting channel and computation efficiency.High energy under equal bandwidth can indicate a poor channel, low computation efficiency, or both.
  • Scheduling Metric: For each device, the scheduler numerically solves the optimal transmission time, calculates its energy metric, and selects the M devices with the smallest values.The transmission-time equation is solved by bisection because the relevant function is monotonically increasing.

B. Effect of Scheduling on Convergence

Scheduling introduces a convergence–energy trade-off: increasing the scheduled-device count reduces scheduling bias but increases energy consumption faster than linearly. Heterogeneous time divisions also create spectrum holes that greedy sharing can exploit.

  • Effect of Scheduling on Convergence: Increasing the number of scheduled devices M makes the scheduling bias vanish at rate O(1/M), improving convergence in rounds.The scheduling-dependent term is new relative to cited results without scheduling.
  • Effect of Scheduling on Convergence: Increasing M also makes sum energy consumption scale faster than linearly.The convergence benefit therefore comes with higher energy consumption as more devices participate.
  • Greedy Spectrum Sharing: Heterogeneous computation durations can create spectrum holes because devices complete local computation at different times.These unoccupied resources can be allocated to devices that are ready to transmit.
  • Greedy Spectrum Sharing: Greedy spectrum sharing allocates each arriving spectrum hole to an available device to reduce transmission energy.The scheme divides a round into time slots and adds shared bandwidth to pre-allocated bandwidth.
  • Greedy Spectrum Sharing: The sharing metric favors devices with small acceleration rates because extra bandwidth then yields greater energy reduction.The acceleration rate is defined through the energy–bandwidth relationship.

VI. EXPERIMENTAL RESULTS

Experiments evaluate C2RM and its extensions in an MNIST FEEL system with CPU-GPU heterogeneous devices. Joint resource management, optimal time division, spectrum sharing, and C2-aware scheduling each reduce energy relative to their stated baselines.

  • Energy-efficient RM: 57.3%: separate C2RM reduces sum energy relative to the policy without RM at per-round latency 1.0 s.Only computation RM and only communication RM reduce the same baseline by 13.8% and 43.5%, respectively.
  • Energy-efficient RM: 17.2%, 47.5%, 9.8%, and 43.5%: optimal time divisions reduce the corresponding uniform-division schemes at per-round latency 1.0 s.The result supports the role of optimal C2 time division in balancing heterogeneity.
  • Energy-efficient RM: 17.2%, 59.0%, 37.4%, and 64.6%: joint C2RM reduces sum energy relative to schemes 1)-4) under uniform time division.Under optimal time divisions, it reduces energy relative to schemes 2)-4) by 21.9%, 31.3%, and 37.4%.
  • Greedy spectrum sharing: 10.6%: greedy spectrum sharing reduces sum energy relative to optimal C2RM without spectrum sharing at uplink bandwidth 2.5 MHz.The experiment attributes the improvement to scavenging unused radio resources.
  • Device Scheduling: 39.8%: C2-aware scheduling reduces sum energy for M = 35 compared with random selection.The evaluation also shows that average learning accuracy increases with M while sum energy grows faster than linearly.

APPENDIX

The appendix supplies analytical derivations for heterogeneous computation, bandwidth allocation, convergence with scheduling, and virtual-update analysis. These derivations establish processor activity conditions and the scheduling convergence bound.

  • Computation Analysis: The computation-energy derivation shows that both CPU and GPU should be active at an energy-workload rate equilibrium.The conclusion follows from complementary slackness and the nonnegative Lagrange multipliers.
  • Bandwidth Analysis: The bandwidth derivation characterizes each device’s bandwidth relation through monotonicity and convexity properties.The appendix states that Bk decreases convexly with the multiplier and that the energy-bandwidth rate is decreasing and strictly convex in Bk.
  • Convergence Analysis: The proof bounds the global model error by decomposing its distance from the optimum into virtual-model and cross-term components.The derivation uses smoothness and convexity assumptions before taking expectations over rounds.
  • Convergence Analysis: The scheduling convergence proof models unscheduled devices through virtual local updates while aggregating only the selected M devices.This construction is stated to be equivalent to the learning process under device scheduling.

F. Clarification of Definition 2

This section expresses device energy and analyzes how it changes with allocated bandwidth under a time constraint. It uses an energy-time rate equilibrium and derives first- and second-order bandwidth derivatives.

  • Device k's energy consumption is formulated using workload-related terms and a timing relation.The supplied passage introduces the energy expression and gives Wk + W′k + tk = T.
  • Because Wk and W′k are irrelevant to allocated bandwidth Bk, the energy derivative with respect to bandwidth can be calculated directly.
  • The bandwidth-derivative analysis uses the energy-time rate equilibrium stated in Lemma 5.
  • The section derives the second derivative of energy with respect to bandwidth after establishing the first derivative.
Loading 2007.07122v2…