PaperlayerView exact revision on arXiv

Source-linked AI summary

To Talk or to Work: Flexible Communication Compression for Energy Efficient Federated Learning over Heterogeneous Mobile Edge Devices

Liang Li, Dian Shi, Ronghui Hou, Hui Li, Miao Pan, Zhu Han

arXiv:2012.11804v1cs.LGcs.AI

TL;DR

FL over mobile edge devices must limit the communication and computation energy imposed by synchronization and local training, especially under heterogeneous device conditions. The paper develops a convergence-guaranteed, flexibly compressed FL algorithm and uses its convergence bound to control compression and balance computing with communication energy. The authors report that the resulting scheme supports heterogeneous edge devices without sacrificing model accuracy and shows energy-saving potential in extensive simulations.

  • Problem

    FL imposes substantial communication and computation burdens on battery-constrained devices, motivating energy-efficient training across heterogeneous mobile edge environments.

  • Method

    The paper combines flexible gradient compression with local computation, sparsification, error compensation, and batch-size increments, then selects compression parameters using a convergence-bound-guided control scheme.

  • Results

    The proposed flexibly compressed FL scheme accommodates heterogeneous edge devices without sacrificing model accuracy, with extensive simulations verifying the analysis and demonstrating energy-saving potential.

  • Takeaways & Limitations

    Compression should be controlled jointly with local computation from a long-term learning perspective to balance wireless communication and device computing energy.

  • Takeaways & Limitations

    The convergence analysis relies on smoothness and bounded-variance assumptions for the participant loss functions and training process.

Abstract

from arXiv · show

Recent advances in machine learning, wireless communication, and mobile hardware technologies promisingly enable federated learning (FL) over massive mobile edge devices, which opens new horizons for numerous intelligent mobile applications. Despite the potential benefits, FL imposes huge communication and computation burdens on participating devices due to periodical global synchronization and continuous local training, raising great challenges to battery constrained mobile devices. In this work, we target at improving the energy efficiency of FL over mobile edge networks to accommodate heterogeneous participating devices without sacrificing the learning performance. To this end, we develop a convergence-guaranteed FL algorithm enabling flexible communication compression. Guided by the derived convergence bound, we design a compression control scheme to balance the energy consumption of local computing (i.e., "working") and wireless communication (i.e., "talking") from the long-term learning perspective. In particular, the compression parameters are elaborately chosen for FL participants adapting to their computing and communication environments. Extensive simulations are conducted using various datasets to validate our theoretical analysis, and the results also demonstrate the efficacy of the proposed scheme in energy saving.

I. INTRODUCTION

FL over wireless edge devices is promising but energy-intensive because local computation and gradient transmission strain limited batteries. The paper addresses this by combining flexible compression with convergence-guided control tailored to heterogeneous devices.

  • Resource-hungry gradient transmission and local optimization consume substantial energy, while limited battery lifetime constrains continuous learning and complex-model training.
  • Communication compression can reduce traffic with little impact on learning outcomes, but existing resource-management studies largely overlook algorithm-level resource reduction.
  • 0.16J is consumed to transmit a roughly 100MB ResNet-50 over a 1Gbps wireless link, comparable to 0.2J for one local-training step on an NVIDIA Tesla V100 GPU.
  • Severe gradient compression may reduce energy per communication round yet cause extra communications or convergence failure, impairing overall energy efficiency.
  • The proposed FL algorithm combines local computation, gradient sparsification, error compensation, and batch-size increment, while its control scheme minimizes computing and communication energy across heterogeneous devices.
  • Participants may use different compression levels before uploading, and the control algorithm uses Benders decomposition and inner convex approximation to select participant-specific parameters.
  • The study evaluates the control scheme through extensive simulations spanning various data sources, learning architectures, and system configurations.

B. Convergence Analysis and Discussion

Under smoothness and bounded-variance assumptions, the paper establishes a convergence rate for flexibly compressed FL and derives its communication complexity. The analysis shows that participant sparsity and synchronization frequency jointly affect convergence, communication rounds, and learning efficiency.

  • The analysis assumes smooth local loss functions with bounded variances and bounded second momentum.
  • With participant-specific sparsity values {δm}∀m and a suitably chosen constant learning rate, Algorithm 1 has a theoretically characterized convergence rate.The theorem is stated for the sparsity series {δm}∀m and ηt = η = θ/T under specified conditions.
  • The convergence analysis uses the average expected squared gradient norm as the convergence measure in the non-convex setting.
  • The analysis derives the maximum number of global communication rounds required to reach ε-global model convergence.Communications occur once every H iterations, so the total number of rounds is K = T/H.
  • Higher gradient sparsity δ increases the communication-round bound, while excessively enlarging H can impair learning efficiency.

IV. ENERGY-EFFICIENT FEDERATED LEARNING ON GPUS: PROBLEM FORMULATION AND CONTROL ALGORITHM

The control design recognizes that sparsity and global update frequency affect both communication payload and local processing workload. These parameters therefore determine participating devices’ energy consumption.

  • Gradient sparsity and global update frequency affect device energy consumption by changing transmission payload and processing workload.

1) Communication model:

The communication model estimates the bits and energy required for sparsified gradient transmission while pairing this with a GPU-based computation-energy model. It accounts for transmission overhead, fading channels, and hardware-dependent execution costs.

  • Communication model: Each participant communicates sparsified gradients by encoding non-zero values, signs, and positions, with total bits denoted by S_m.The sparsity is δ_m=d/k_m, and FPP specifies floating-point precision.
  • Communication model: Transmission energy depends on channel fading, noise, bandwidth, transmit power, and the number of communicated bits.The channel model averages over fading h_m and uses N_0, W_m, and P_m to characterize wireless transmission.
  • Computational model: GPU computation energy is modeled using core voltage, core frequency, memory frequency, and hardware- and application-dependent coefficients.The coefficients are obtained from platform-based measurements of single-step SGD energy consumption.
  • Computational model: The model assumes per-iteration computation energy remains unchanged during training despite incremental batch sizes.This assumes GPU parallelism keeps execution time nearly constant below a batch-size threshold.

3) Problem Formulation:

The formulation models total training energy across participating devices and jointly chooses gradient sparsity and global update frequency. It captures the long-term trade-off between per-round savings and convergence-related synchronization costs.

  • 3) Problem Formulation: Total energy over the training process combines communication and computation costs across heterogeneous participants.The model captures differences in communication conditions and GPU capacities.
  • 3) Problem Formulation: The gradient-sparsification workload is omitted from energy accounting because it is considered negligible relative to local SGD.This is stated as a modeling assumption for the computational cost.
  • 3) Problem Formulation: The optimization jointly determines each participant’s gradient sparsity δ_m and the global update frequency H to minimize overall energy.The objective includes the energy associated with transmitting compressed updates and performing local computation.
  • 3) Problem Formulation: More severe compression can reduce single-round energy but increase gradient distortion, global synchronizations, and total energy through slower convergence.The formulation therefore evaluates compression over the full learning process rather than one global iteration.

B. Compression Control Algorithm

The compression-control method approximately solves a mixed-integer nonlinear energy-minimization problem whose continuous sparsity variables are coupled with the integer synchronization frequency. It combines generalized Benders decomposition with inner convex approximation.

  • B. Compression Control Algorithm: The control problem jointly selects participant sparsities and synchronization frequency under compression and convergence constraints.The resulting formulation is mixed-integer and nonlinear because H is coupled with the continuous variables {δ_m}.
  • B. Compression Control Algorithm: The sparsified-gradient bit estimate uses Stirling’s formula and the assumption δ_m=d/k_m≫1, with convexity guaranteed when δ_m≥e^3/2.The algorithm assumes the lower sparsity bound is at least e^3/2 because empirical evidence indicates this condition holds in practice.
  • B. Compression Control Algorithm: Generalized Benders decomposition separates the problem into a continuous primal problem for {δ_m} and an integer master problem for H.The two subproblems are solved in alternating outer-loop iterations.
  • B. Compression Control Algorithm: The primal problem is feasible for every candidate H because the continuous variables are independent of H in the relevant constraint.This removes the need for a conventional feasibility check on the current H.
  • B. Compression Control Algorithm: The master problem is a small-scale mixed-integer program that can be solved with classical methods such as Branch-and-Bound.Its stated computational complexity is O(2^|H|).
  • B. Compression Control Algorithm: Inner convex approximation solves the non-convex primal problem by repeatedly optimizing strongly convex approximations while maintaining feasibility.The approximated primal problem can be solved optimally at each inner iteration.

V. PERFORMANCE EVALUATION

Experiments compare FlexibleSpar with synchronous, greedy, and unified sparsification schemes across system conditions, architectures, and datasets. FlexibleSpar generally provides the strongest energy efficiency while preserving convergence behavior close to UnifiedSpar.

  • Energy efficiency: Average energy consumption decreases as the number of participating devices grows, although the curves become nearly flat at larger system scales.The reported explanation is that additional participants can accelerate convergence, with diminishing speed-ups once the dataset is adequately represented.
  • Energy efficiency: FlexibleSpar outperforms the comparison schemes in average energy consumption by using foresight and participant-specific flexibility in compression decisions.SynSgdSpars consumes the most energy because it substantially increases communication complexity.
  • Energy efficiency: Higher communication heterogeneity impairs system energy efficiency, while FlexibleSpar is more resilient than the other schemes under heterogeneous wireless conditions.The heterogeneity level is varied through L, with larger L indicating greater heterogeneity.
  • System parameters: When communication energy intensity is small, the controller favors larger temporal sparsity H to reduce the number of global rounds.As communication energy intensity grows, participants tend to increase gradient sparsity instead.
  • Learning performance: The evaluation covers ResNet20 on CIFAR-10 and LeNet5-Caffe on MNIST, reporting convergence in epochs and consumed energy.These experiments use ordinary distributed SGD as the baseline.
  • Learning performance: FlexibleSpar shows convergence and final accuracy similar to UnifiedSpar, while both slightly underperform SynSgdSpar.The results also indicate that temporal sparsity has a stronger convergence impact than gradient sparsity in this setting.

VI. CONCLUSION

The paper presents a flexible communication-compression solution for reducing federated learning energy consumption on heterogeneous edge devices while preserving model accuracy. It combines convergence analysis with energy-oriented compression control and validates the approach through simulations.

  • The proposed FL algorithm enables flexible communication compression with convergence analysis and targets energy reduction without sacrificing model accuracy.The approach is designed for heterogeneous participating edge devices.
  • The compression control scheme uses the derived convergence bound to account for heterogeneous computing and communication conditions across devices.This links compression choices to device-specific energy-efficiency considerations.
  • Extensive simulations verify the theoretical analysis and evaluate the algorithm’s performance.
  • The flexibly compressed FL scheme shows potential for accommodating heterogeneous mobile edge devices and improving their energy efficiency.

A. Proof of Theorem 1

The proof establishes bounded error behavior for local and accumulated sequences in periodically synchronized workers, then combines these bounds to complete Theorem 1. It uses contraction, expectation, convexity, and geometric-series arguments under stated assumptions.

  • A. Proof of Theorem 1: The analysis defines auxiliary sequences and uses memory and local-sequence deviation lemmas to characterize errors in the perturbed-iterate framework.The bounded-memory lemma applies to workers synchronizing with the server every H local iterations.
  • A. Proof of Theorem 1: For workers synchronizing every H iterations, the proof tracks aggregation indices and keeps the worker and global parameter vectors unchanged between aggregations.The proof assumes T is an integer multiple of H.
  • A. Proof of Theorem 1: The bounded-memory argument uses the Topk contraction property, unchanged states between synchronization points, and iterative inequalities over the aggregation intervals.The proof also applies Jensen’s inequality and expectations over sampled mini-batches.
  • A. Proof of Theorem 1: The proof combines the memory and bounded-memory lemmas with the local-sequence deviation bound to derive the theorem’s final inequality.The resulting simplification uses a partial sum of a geometric series.
Loading 2012.11804v1…