PaperlayerView exact revision on arXiv

Source-linked AI summary

HFEL: Joint Edge Association and Resource Allocation for Cost-Efficient Hierarchical Federated Edge Learning

Siqi Luo, Xu Chen, Qiong Wu, Zhi Zhou, Shuai Yu

arXiv:2002.11343v2cs.DC

TL;DR

The paper addresses the communication, energy, and latency costs of conventional device-cloud FL. It proposes HFEL with partial edge aggregation and a joint scheduling algorithm for resource allocation and edge association, reporting stable cost-efficient operation and improved performance over benchmarks and conventional FL.

  • Problem

    Conventional FL can incur substantial WAN communication latency, energy overhead, and backbone congestion during repeated large-model transmissions.

    Source: §I · p. 2

  • Method

    HFEL partially migrates model aggregation to edge servers and decomposes joint cost minimization into resource allocation and edge association subproblems.

    Source: §I · p. 3, §VII · p. 28

  • Results

    The scheduling process converges to a stable system point and HFEL achieves cost reduction over benchmarks with higher accuracies and lower training loss than conventional FL.

    Source: §VII · p. 28

  • Takeaways & Limitations

    Hierarchical edge aggregation provides a cost-efficient federated learning framework with improved reported training performance compared with conventional device-cloud FL.

    Source: §VII · p. 28

Abstract

from arXiv · show

Federated Learning (FL) has been proposed as an appealing approach to handle data privacy issue of mobile devices compared to conventional machine learning at the remote cloud with raw user data uploading. By leveraging edge servers as intermediaries to perform partial model aggregation in proximity and relieve core network transmission overhead, it enables great potentials in low-latency and energy-efficient FL. Hence we introduce a novel Hierarchical Federated Edge Learning (HFEL) framework in which model aggregation is partially migrated to edge servers from the cloud. We further formulate a joint computation and communication resource allocation and edge association problem for device users under HFEL framework to achieve global cost minimization. To solve the problem, we propose an efficient resource scheduling algorithm in the HFEL framework. It can be decomposed into two subproblems: \emph{resource allocation} given a scheduled set of devices for each edge server and \emph{edge association} of device users across all the edge servers. With the optimal policy of the convex resource allocation subproblem for a set of devices under a single edge server, an efficient edge association strategy can be achieved through iterative global cost reduction adjustment process, which is shown to converge to a stable system point. Extensive performance evaluations demonstrate that our HFEL framework outperforms the proposed benchmarks in global cost saving and achieves better training performance compared to conventional federated learning.

I. INTRODUCTION

The paper proposes HFEL, which moves partial model aggregation to nearby edge servers to reduce WAN overhead while coordinating resource allocation and edge association for global cost minimization.

Motivation

  • FL keeps device data local but repeatedly sends model parameters to the cloud for global aggregation.This privacy-preserving workflow still involves repeated device-cloud communication.Source: §I · p. 2
  • FL incurs communication latency, energy overhead, and backbone congestion because many devices transmit large models over WAN links.These constraints can degrade learning performance under a training-time budget and challenge battery-constrained devices.Source: §I · p. 2

HFEL framework

  • HFEL inserts nearby edge servers between devices and the cloud, enabling local model aggregation before updated models reach the cloud.The framework partially migrates aggregation from the cloud to edge servers.Source: §I · p. 2

Problem formulation

  • The paper formulates joint computation, communication resource allocation, and edge association to minimize global learning cost.The optimization addresses limited edge-server and device resources and the trade-off between serving more devices and allocating each device less communication resource.Source: §I · p. 3

Solution

  • The proposed scheduling algorithm decomposes the hard optimization into resource allocation and edge association, using an optimal convex allocation policy and cost-reducing association iterations.The association process is reported to converge to a stable point where edge servers retain stable training groups.Source: §I · p. 3

Evaluation

  • Experiments report that HFEL achieves global cost savings over benchmarks and better training performance than conventional device-cloud FL.The reported outcomes include lower global energy ratio, learning delay ratio, and global cost benchmark, alongside higher global and test accuracies and lower training loss.Source: §I · pp. 4–5

II. SYSTEM MODEL

HFEL performs hierarchical training through local device computation, edge aggregation, and cloud aggregation. The system models computation, communication, energy, and delay while preserving device data privacy.

II. SYSTEM MODEL

  • HFEL partitions aggregation between edge servers and the cloud, with devices sharing models through both layers.Each global iteration includes edge aggregation and cloud aggregation.Source: A. Learning process in HFEL · p. 6
  • Devices locally compute model updates, transmit them to associated edge servers, and receive edge-aggregated models for subsequent iterations.The edge procedure repeats until all servers reach a common edge accuracy ε.Source: 1) Edge Aggregation · pp. 6–8
  • Local computation overhead depends on data size, CPU cycles per sample, allocated CPU frequency, and the number of local iterations needed for accuracy θ.The local iteration count is modeled as L(θ) = µ log(1/θ).Source: 1) Edge Aggregation · pp. 6–7
  • Wireless model transmission uses OFDMA bandwidth allocation and accounts for transmission rate, communication time, and transmission energy.Each edge server allocates bandwidth through device-specific ratios βi:n.Source: 1) Edge Aggregation · p. 7
  • Edge servers average updated models from their connected devices and broadcast the aggregate, while their aggregation and broadcast costs are omitted from optimization.The omission relies on strong edge computing capability, stable energy supply, and small receiving cost relative to uploading.Source: 1) Edge Aggregation · pp. 7–8
  • The system preserves personal data privacy because edge servers do not access devices’ local datasets.The framework’s convergence analysis also covers convex tasks and can extend to non-convex tasks when accuracy-iteration relations are known.Source: 1) Edge Aggregation · p. 8

2) Cloud Aggregation:

After repeated edge aggregation, each edge server uploads its model to the remote cloud, where the models are globally aggregated and redistributed through the edge layer.

2) Cloud Aggregation:

  • Each edge server uploads its aggregated model to the cloud after I(ε, θ) edge-aggregation rounds.The cloud-upload stage uses each server’s transmission rate, power, and model size to derive delay and energy.Source: 2) Cloud Aggregation · pp. 8–9
  • The cloud receives updated models from all edge servers and aggregates them into a global model.Cloud aggregation completes the global iteration after edge-side processing.Source: 2) Cloud Aggregation · p. 9
  • Cloud aggregation time is neglected because it is much smaller than aggregation time on mobile devices.System-wide energy and delay are therefore obtained while omitting cloud aggregation time.Source: 2) Cloud Aggregation · p. 9
  • The global aggregation procedure repeats by pushing the global model to devices through edge servers until an accuracy or total-training-time stopping condition is satisfied.Algorithm 1 describes one global aggregation iteration.Source: 2) Cloud Aggregation · p. 9

B. Problem Formulation

The problem formulation minimizes system-wide energy and delay within one global iteration. It introduces weighting factors for these objectives and presents an algorithmic procedure for edge aggregation.

B. Problem Formulation

  • The optimization targets system-wide energy and delay minimization within one global iteration.This defines the central performance objective of the HFEL system model.Source: B. Problem Formulation · p. 9
  • The objective weights λe and λt represent the relative importance of energy and delay in the training objectives.Both weighting indicators lie in the interval [0, 1].Source: B. Problem Formulation · p. 9
  • The HEFL optimization problem is formulated from the system model and its energy-delay objective.The passage introduces the formal optimization problem after defining the weighting indicators.Source: B. Problem Formulation · p. 9
  • Algorithm 1 specifies the HFEL procedure under one global iteration.The algorithm organizes the edge aggregation process before global continuation.Source: B. Problem Formulation · p. 10

2: Edge aggregation:

  • During edge aggregation, devices perform local model computation in parallel.The device loop covers n = 1, ..., N.Source: §2 · p. 10
  • Each device transmits its local model to edge server i as part of the edge aggregation procedure.The transmission step follows local model computation.Source: §2 · p. 10
  • Edge servers process their connected devices in parallel during the aggregation stage.The edge-server loop covers i = 1, ..., K.Source: §2 · p. 10

17: Cloud aggregation:

The HFEL scheduling problem couples edge association with computation and communication allocation, making direct optimization difficult. The paper decomposes it into tractable resource allocation and iterative association procedures.

17: Cloud aggregation:

  • The original problem jointly optimizes computation, communication, and edge association under system-wide cost minimization.The association constraints create a large combinatorial search space and couple with resource allocation in the objective.Source: §17 · p. 11
  • The scheduling design separates resource allocation within one edge server from edge association across multiple edge servers.This divide-and-conquer formulation is used because the original optimization problem is hard to solve directly.Source: C. Overview of HFEL Scheduling Scheme · p. 11
  • The procedure initializes edge associations, solves per-server resource allocation, and then iteratively tests transferring or exchanging devices.Adjustments are permitted when they improve system-wide performance without damaging an edge server’s utility.Source: C. Overview of HFEL Scheduling Scheme · p. 12
  • The iterative process stops when no permitted association adjustment remains, yielding a stable point within a limited number of iterations.The resulting resource scheduling algorithm is described as fast and suitable for practical implementation.Source: C. Overview of HFEL Scheduling Scheme · p. 12

III. OPTIMAL RESOURCE ALLOCATION WITHIN SINGLE EDGE SERVER

For a fixed training group at one edge server, HFEL minimizes overhead through a convex joint resource allocation problem. The paper derives optimal allocation structure and reduces computation by transforming the problem to fewer decision variables.

III. OPTIMAL RESOURCE ALLOCATION WITHIN SINGLE EDGE SERVER

  • The single-edge-server subproblem minimizes overhead through joint computation and communication resource allocation for a scheduled device group.The formulation is considered under a fixed training group and edge-server setting.Source: §III · pp. 12–13
  • Theorem 1 establishes that the resource allocation subproblem is convex.The proof uses convex component subformulas and affine constraints.Source: §III · p. 13
  • Theorem 2 characterizes the optimal bandwidth and computation-capacity allocations for each device under an edge server.The structural result is obtained by exploiting the Karush-Kuhn-Tucker conditions.Source: §III · p. 13
  • Algorithm 2 replaces bandwidth variables using the structural result, solves the resulting convex problem for computation capacities, and recovers bandwidth allocations.Convex optimization solvers such as CVX and IPOPT can solve the transformed problem.Source: §III · p. 15
  • The variable-reduction transformation greatly reduces the size of the original decision space and can significantly reduce practical solution computing time.The reduced formulation preserves the optimal resource allocation policy.Source: §III · p. 15

IV. EDGE ASSOCIATION FOR MULTIPLE EDGE SERVERS

HFEL performs edge association across multiple servers by comparing utilities after optimal local resource allocation. Permitted device transfers and exchanges iteratively reduce system overhead until no server benefits from deviating.

IV. EDGE ASSOCIATION FOR MULTIPLE EDGE SERVERS

  • Each edge association strategy assigns devices to local training groups, with its system utility defined from the minimum local resource-allocation costs.A strategy is preferred when it has higher utility, corresponding to lower overhead.Source: §IV · p. 16
  • The algorithm permits device transfers or pairwise exchanges only when the resulting association strategy improves system-wide utility.These adjustments change local training-group composition while preserving the improvement criterion.Source: §IV · p. 16
  • Each edge server computes its utility and communicates with other servers to coordinate association adjustments.The process assumes knowledge of local channel gains and computation capacities obtained through feedback.Source: §IV · p. 17
  • Every permitted adjustment decreases systematic overhead by Δ = v(DS2) − v(DS1), and the process terminates at a stable strategy.At the stable point, no edge server deviates from its local training group to achieve lower global FL overhead.Source: §IV · p. 17
  • The decentralized algorithm initializes groups and local resource allocations, then repeats transfer and exchange adjustments until none is permitted.Algorithm 3 outputs the stable system point DS∗ and its associated optimal allocations.Source: §IV · p. 17
  • Performance evaluation reports that the edge association algorithm converges quickly with an almost linear convergence speed.The resulting scheduling approach is presented as converging fast enough for practical use.Source: §IV · p. 19

V. PERFORMANCE EVALUATION

The evaluation examines global cost saving and HFEL training behavior using randomized device and edge-server deployments. It measures test accuracy, training accuracy, and training loss under the stated simulation settings.

V. PERFORMANCE EVALUATION

  • The simulations evaluate global cost saving for the proposed resource scheduling algorithm.Cost saving is one of the two stated evaluation objectives.Source: §V · p. 19
  • HFEL performance is assessed using test accuracy, training accuracy, and training loss.These metrics form the stated training-performance evaluation.Source: §V · p. 19
  • Devices and edge servers are randomly distributed within a 500M × 500M area in the simulation setting.The deployment randomness represents the stated availability perspective of devices and edge servers.Source: §V · p. 19
  • Table II reports the simulation settings used for the performance evaluation.The supplied passage identifies the table but does not provide its individual parameter values.Source: Table II · p. 19

A. Performance gain in cost reduction

HFEL achieves lower learning delay, energy, and global cost ratios than the evaluated schemes, with gains varying across objectives and configurations.

B. Performance gain in training loss and accuracy

HFEL improves training accuracy and loss relative to FedAvg on MNIST and FEMNIST, while local and edge iteration choices affect convergence and communication overhead.

Training results

Iteration effects

Communication overhead

VI. RELATED WORK

Prior work addresses communication, resource allocation, and convergence in federated learning, but often omits heterogeneous devices or multiple-edge-server coordination.

  • Existing FL studies reduce communication cost through compression, selective updates, or increased local computation.Source: §VI · p. 26
  • Earlier hierarchical federated learning in MEC reports faster convergence but does not consider mobile-device heterogeneity.Source: §VI · p. 27
  • Resource-allocation research considers participation, batch size, bandwidth, energy, convergence time, and device computing capacities.Source: §VI · p. 27
  • HFEL jointly optimizes computation and bandwidth allocation for devices and edge association across multiple edge servers.Source: §VI · p. 27
  • Unlike cited studies that consider one edge server, HFEL addresses scenarios where devices can communicate with multiple edge servers.Source: §VI · p. 27

VII. CONCLUSION

The paper introduces HFEL to migrate partial aggregation to edge servers, jointly optimize training resources and association, and iteratively reduce global cost.

  • HFEL partially migrates model aggregation from the cloud to edge servers to support low-latency and energy-efficient federated learning.Source: §VII · p. 28
  • The framework formulates joint computation and communication resource allocation for global cost minimization.Source: §VII · p. 28
  • The scheduling algorithm separates resource allocation for scheduled device sets from edge association across edge servers.Source: §VII · p. 28
  • Cost-reducing iterations terminate at a stable system point and provide substantial cost-reduction gains over benchmarks.Source: §VII · p. 28
  • Compared with conventional federated learning without edge intermediaries, HFEL achieves higher global and test accuracies and lower training loss.Source: §VII · p. 28

Related papers

Loading 2002.11343v2…