PaperlayerView exact revision on arXiv

Source-linked AI summary

HFEL: Joint Edge Association and Resource Allocation for Cost-Efficient Hierarchical Federated Edge Learning

Siqi Luo, Xu Chen, Qiong Wu, Zhi Zhou, Shuai Yu

arXiv:2002.11343v2cs.DC

TL;DR

The paper addresses the communication, energy, and latency costs of conventional device-cloud FL. It proposes HFEL with partial edge aggregation and a joint scheduling algorithm for resource allocation and edge association, reporting stable cost-efficient operation and improved performance over benchmarks and conventional FL.

  • Problem

    Conventional FL can incur substantial WAN communication latency, energy overhead, and backbone congestion during repeated large-model transmissions.

  • Method

    HFEL partially migrates model aggregation to edge servers and decomposes joint cost minimization into resource allocation and edge association subproblems.

  • Results

    The scheduling process converges to a stable system point and HFEL achieves cost reduction over benchmarks with higher accuracies and lower training loss than conventional FL.

  • Takeaways & Limitations

    Hierarchical edge aggregation provides a cost-efficient federated learning framework with improved reported training performance compared with conventional device-cloud FL.

Abstract

from arXiv · show

Federated Learning (FL) has been proposed as an appealing approach to handle data privacy issue of mobile devices compared to conventional machine learning at the remote cloud with raw user data uploading. By leveraging edge servers as intermediaries to perform partial model aggregation in proximity and relieve core network transmission overhead, it enables great potentials in low-latency and energy-efficient FL. Hence we introduce a novel Hierarchical Federated Edge Learning (HFEL) framework in which model aggregation is partially migrated to edge servers from the cloud. We further formulate a joint computation and communication resource allocation and edge association problem for device users under HFEL framework to achieve global cost minimization. To solve the problem, we propose an efficient resource scheduling algorithm in the HFEL framework. It can be decomposed into two subproblems: \emph{resource allocation} given a scheduled set of devices for each edge server and \emph{edge association} of device users across all the edge servers. With the optimal policy of the convex resource allocation subproblem for a set of devices under a single edge server, an efficient edge association strategy can be achieved through iterative global cost reduction adjustment process, which is shown to converge to a stable system point. Extensive performance evaluations demonstrate that our HFEL framework outperforms the proposed benchmarks in global cost saving and achieves better training performance compared to conventional federated learning.

I. INTRODUCTION

The paper proposes HFEL, which moves partial model aggregation to nearby edge servers to reduce WAN overhead while coordinating resource allocation and edge association for global cost minimization.

  • Motivation: FL keeps device data local but repeatedly sends model parameters to the cloud for global aggregation.This privacy-preserving workflow still involves repeated device-cloud communication.
  • Motivation: FL incurs communication latency, energy overhead, and backbone congestion because many devices transmit large models over WAN links.These constraints can degrade learning performance under a training-time budget and challenge battery-constrained devices.
  • HFEL framework: HFEL inserts nearby edge servers between devices and the cloud, enabling local model aggregation before updated models reach the cloud.The framework partially migrates aggregation from the cloud to edge servers.
  • Problem formulation: The paper formulates joint computation, communication resource allocation, and edge association to minimize global learning cost.The optimization addresses limited edge-server and device resources and the trade-off between serving more devices and allocating each device less communication resource.
  • Solution: The proposed scheduling algorithm decomposes the hard optimization into resource allocation and edge association, using an optimal convex allocation policy and cost-reducing association iterations.The association process is reported to converge to a stable point where edge servers retain stable training groups.
  • Evaluation: Experiments report that HFEL achieves global cost savings over benchmarks and better training performance than conventional device-cloud FL.The reported outcomes include lower global energy ratio, learning delay ratio, and global cost benchmark, alongside higher global and test accuracies and lower training loss.

II. SYSTEM MODEL

HFEL performs hierarchical training through local device computation, edge aggregation, and cloud aggregation. The system models computation, communication, energy, and delay while preserving device data privacy.

  • II. SYSTEM MODEL: HFEL partitions aggregation between edge servers and the cloud, with devices sharing models through both layers.Each global iteration includes edge aggregation and cloud aggregation.
  • II. SYSTEM MODEL: Devices locally compute model updates, transmit them to associated edge servers, and receive edge-aggregated models for subsequent iterations.The edge procedure repeats until all servers reach a common edge accuracy ε.
  • II. SYSTEM MODEL: Local computation overhead depends on data size, CPU cycles per sample, allocated CPU frequency, and the number of local iterations needed for accuracy θ.The local iteration count is modeled as L(θ) = µ log(1/θ).
  • II. SYSTEM MODEL: Wireless model transmission uses OFDMA bandwidth allocation and accounts for transmission rate, communication time, and transmission energy.Each edge server allocates bandwidth through device-specific ratios βi:n.
  • II. SYSTEM MODEL: Edge servers average updated models from their connected devices and broadcast the aggregate, while their aggregation and broadcast costs are omitted from optimization.The omission relies on strong edge computing capability, stable energy supply, and small receiving cost relative to uploading.
  • II. SYSTEM MODEL: The system preserves personal data privacy because edge servers do not access devices’ local datasets.The framework’s convergence analysis also covers convex tasks and can extend to non-convex tasks when accuracy-iteration relations are known.

2) Cloud Aggregation:

After repeated edge aggregation, each edge server uploads its model to the remote cloud, where the models are globally aggregated and redistributed through the edge layer.

  • 2) Cloud Aggregation:: Each edge server uploads its aggregated model to the cloud after I(ε, θ) edge-aggregation rounds.The cloud-upload stage uses each server’s transmission rate, power, and model size to derive delay and energy.
  • 2) Cloud Aggregation:: The cloud receives updated models from all edge servers and aggregates them into a global model.Cloud aggregation completes the global iteration after edge-side processing.
  • 2) Cloud Aggregation:: Cloud aggregation time is neglected because it is much smaller than aggregation time on mobile devices.System-wide energy and delay are therefore obtained while omitting cloud aggregation time.
  • 2) Cloud Aggregation:: The global aggregation procedure repeats by pushing the global model to devices through edge servers until an accuracy or total-training-time stopping condition is satisfied.Algorithm 1 describes one global aggregation iteration.

B. Problem Formulation

The problem formulation minimizes system-wide energy and delay within one global iteration. It introduces weighting factors for these objectives and presents an algorithmic procedure for edge aggregation.

  • B. Problem Formulation: The optimization targets system-wide energy and delay minimization within one global iteration.This defines the central performance objective of the HFEL system model.
  • B. Problem Formulation: The objective weights λe and λt represent the relative importance of energy and delay in the training objectives.Both weighting indicators lie in the interval [0, 1].
  • B. Problem Formulation: The HEFL optimization problem is formulated from the system model and its energy-delay objective.The passage introduces the formal optimization problem after defining the weighting indicators.
  • B. Problem Formulation: Algorithm 1 specifies the HFEL procedure under one global iteration.The algorithm organizes the edge aggregation process before global continuation.
  • 2: Edge aggregation:: During edge aggregation, devices perform local model computation in parallel.The device loop covers n = 1, ..., N.
  • 2: Edge aggregation:: Each device transmits its local model to edge server i as part of the edge aggregation procedure.The transmission step follows local model computation.
  • 2: Edge aggregation:: Edge servers process their connected devices in parallel during the aggregation stage.The edge-server loop covers i = 1, ..., K.

17: Cloud aggregation:

The HFEL scheduling problem couples edge association with computation and communication allocation, making direct optimization difficult. The paper decomposes it into tractable resource allocation and iterative association procedures.

  • 17: Cloud aggregation:: The original problem jointly optimizes computation, communication, and edge association under system-wide cost minimization.The association constraints create a large combinatorial search space and couple with resource allocation in the objective.
  • 17: Cloud aggregation:: The scheduling design separates resource allocation within one edge server from edge association across multiple edge servers.This divide-and-conquer formulation is used because the original optimization problem is hard to solve directly.
  • 17: Cloud aggregation:: The procedure initializes edge associations, solves per-server resource allocation, and then iteratively tests transferring or exchanging devices.Adjustments are permitted when they improve system-wide performance without damaging an edge server’s utility.
  • 17: Cloud aggregation:: The iterative process stops when no permitted association adjustment remains, yielding a stable point within a limited number of iterations.The resulting resource scheduling algorithm is described as fast and suitable for practical implementation.

III. OPTIMAL RESOURCE ALLOCATION WITHIN SINGLE EDGE SERVER

For a fixed training group at one edge server, HFEL minimizes overhead through a convex joint resource allocation problem. The paper derives optimal allocation structure and reduces computation by transforming the problem to fewer decision variables.

  • III. OPTIMAL RESOURCE ALLOCATION WITHIN SINGLE EDGE SERVER: The single-edge-server subproblem minimizes overhead through joint computation and communication resource allocation for a scheduled device group.The formulation is considered under a fixed training group and edge-server setting.
  • III. OPTIMAL RESOURCE ALLOCATION WITHIN SINGLE EDGE SERVER: Theorem 1 establishes that the resource allocation subproblem is convex.The proof uses convex component subformulas and affine constraints.
  • III. OPTIMAL RESOURCE ALLOCATION WITHIN SINGLE EDGE SERVER: Theorem 2 characterizes the optimal bandwidth and computation-capacity allocations for each device under an edge server.The structural result is obtained by exploiting the Karush-Kuhn-Tucker conditions.
  • III. OPTIMAL RESOURCE ALLOCATION WITHIN SINGLE EDGE SERVER: Algorithm 2 replaces bandwidth variables using the structural result, solves the resulting convex problem for computation capacities, and recovers bandwidth allocations.Convex optimization solvers such as CVX and IPOPT can solve the transformed problem.
  • III. OPTIMAL RESOURCE ALLOCATION WITHIN SINGLE EDGE SERVER: The variable-reduction transformation greatly reduces the size of the original decision space and can significantly reduce practical solution computing time.The reduced formulation preserves the optimal resource allocation policy.

IV. EDGE ASSOCIATION FOR MULTIPLE EDGE SERVERS

HFEL performs edge association across multiple servers by comparing utilities after optimal local resource allocation. Permitted device transfers and exchanges iteratively reduce system overhead until no server benefits from deviating.

  • IV. EDGE ASSOCIATION FOR MULTIPLE EDGE SERVERS: Each edge association strategy assigns devices to local training groups, with its system utility defined from the minimum local resource-allocation costs.A strategy is preferred when it has higher utility, corresponding to lower overhead.
  • IV. EDGE ASSOCIATION FOR MULTIPLE EDGE SERVERS: The algorithm permits device transfers or pairwise exchanges only when the resulting association strategy improves system-wide utility.These adjustments change local training-group composition while preserving the improvement criterion.
  • IV. EDGE ASSOCIATION FOR MULTIPLE EDGE SERVERS: Each edge server computes its utility and communicates with other servers to coordinate association adjustments.The process assumes knowledge of local channel gains and computation capacities obtained through feedback.
  • IV. EDGE ASSOCIATION FOR MULTIPLE EDGE SERVERS: Every permitted adjustment decreases systematic overhead by Δ = v(DS2) − v(DS1), and the process terminates at a stable strategy.At the stable point, no edge server deviates from its local training group to achieve lower global FL overhead.
  • IV. EDGE ASSOCIATION FOR MULTIPLE EDGE SERVERS: The decentralized algorithm initializes groups and local resource allocations, then repeats transfer and exchange adjustments until none is permitted.Algorithm 3 outputs the stable system point DS∗ and its associated optimal allocations.
  • IV. EDGE ASSOCIATION FOR MULTIPLE EDGE SERVERS: Performance evaluation reports that the edge association algorithm converges quickly with an almost linear convergence speed.The resulting scheduling approach is presented as converging fast enough for practical use.

V. PERFORMANCE EVALUATION

The evaluation examines global cost saving and HFEL training behavior using randomized device and edge-server deployments. It measures test accuracy, training accuracy, and training loss under the stated simulation settings.

  • V. PERFORMANCE EVALUATION: The simulations evaluate global cost saving for the proposed resource scheduling algorithm.Cost saving is one of the two stated evaluation objectives.
  • V. PERFORMANCE EVALUATION: HFEL performance is assessed using test accuracy, training accuracy, and training loss.These metrics form the stated training-performance evaluation.
  • V. PERFORMANCE EVALUATION: Devices and edge servers are randomly distributed within a 500M × 500M area in the simulation setting.The deployment randomness represents the stated availability perspective of devices and edge servers.
  • V. PERFORMANCE EVALUATION: Table II reports the simulation settings used for the performance evaluation.The supplied passage identifies the table but does not provide its individual parameter values.

A. Performance gain in cost reduction

HFEL achieves lower learning delay, energy, and global cost ratios than the evaluated schemes, with gains varying across objectives and configurations.

  • HFEL achieves the lowest global energy ratio, learning delay ratio, and global cost ratio among the proposed schemes.
  • 63.3%, 46.2%, 43.3%, 56.0% and 44.4% learning delay ratios are reported against uniform resource allocation as device numbers grow.
  • 30% global energy cost ratio is achieved at most against uniform resource allocation under energy-only weighting.
  • 5.0%, 25.0%, 24.0%, 28.0% and 40.3% global cost reduction ratios are obtained over five comparison schemes.
  • Resource allocation optimization contributes more to global cost reduction than the edge association solution.
  • The edge association strategy converges quickly, with iteration growth almost linear in mobile-device and edge-server numbers.

B. Performance gain in training loss and accuracy

HFEL improves training accuracy and loss relative to FedAvg on MNIST and FEMNIST, while local and edge iteration choices affect convergence and communication overhead.

  • Training results: Around 5% higher test and training accuracy and around 3% lower training loss are achieved than FedAvg on MNIST.
  • Training results: Up to 4.4% and 4.0% increases in test and training accuracy, respectively, and around 4.1% lower training loss occur on FEMNIST.
  • Iteration effects: Increasing local iterations from 5 to 50 accelerates convergence on both MNIST and FEMNIST with five edge iterations.
  • Iteration effects: Fewer local iterations and more edge iterations reduce cloud communication rounds needed to reach accuracy 0.9 on MNIST and 0.55 on FEMNIST.
  • Communication overhead: HFEL saves WAN transmission overhead through edge aggregation, but frequent device-edge communication can increase wireless overhead.
  • Communication overhead: A balance between local and edge iteration numbers is needed when minimizing device training overhead.

VI. RELATED WORK

Prior work addresses communication, resource allocation, and convergence in federated learning, but often omits heterogeneous devices or multiple-edge-server coordination.

  • Existing FL studies reduce communication cost through compression, selective updates, or increased local computation.
  • Earlier hierarchical federated learning in MEC reports faster convergence but does not consider mobile-device heterogeneity.
  • Resource-allocation research considers participation, batch size, bandwidth, energy, convergence time, and device computing capacities.
  • HFEL jointly optimizes computation and bandwidth allocation for devices and edge association across multiple edge servers.
  • Unlike cited studies that consider one edge server, HFEL addresses scenarios where devices can communicate with multiple edge servers.

VII. CONCLUSION

The paper introduces HFEL to migrate partial aggregation to edge servers, jointly optimize training resources and association, and iteratively reduce global cost.

  • HFEL partially migrates model aggregation from the cloud to edge servers to support low-latency and energy-efficient federated learning.
  • The framework formulates joint computation and communication resource allocation for global cost minimization.
  • The scheduling algorithm separates resource allocation for scheduled device sets from edge association across edge servers.
  • Cost-reducing iterations terminate at a stable system point and provide substantial cost-reduction gains over benchmarks.
  • Compared with conventional federated learning without edge intermediaries, HFEL achieves higher global and test accuracies and lower training loss.
Loading 2002.11343v2…