PaperlayerView exact revision on arXiv

Source-linked AI summary

Federated Learning in Mobile Edge Networks: A Comprehensive Survey

Wei Yang Bryan Lim, Nguyen Cong Luong, Dinh Thai Hoang, Yutao Jiao, Ying-Chang Liang, Qiang Yang, Dusit Niyato, Chunyan Miao

arXiv:1909.11875v2cs.NIeess.SP

TL;DR

Mobile edge networks need collaborative learning without centralizing personal data, while heterogeneous devices create communication and resource challenges. This survey introduces FL fundamentals, reviews implementation solutions, and examines privacy-preserving mobile edge optimization, concluding with open challenges and future research directions.

  • Problem

    Mobile edge learning requires addressing privacy concerns from sharing personal data and implementation challenges caused by heterogeneous devices, communication costs, and resource constraints.

  • Method

    The paper provides a tutorial and comprehensive survey covering FL fundamentals, system design, implementation challenges, existing solutions, and mobile edge network optimization applications.

  • Results

    The survey reviews approaches for communication cost, resource allocation, data privacy, and data security, and discusses FL implementation for privacy-preserving mobile edge network optimization.

  • Takeaways & Limitations

    FL can reduce data communication by transmitting model parameters instead of raw data, support lower-latency edge decisions, and enhance privacy under non-malicious participants and servers.

  • Takeaways & Limitations

    Existing incentive-mechanism studies generally assume a monopoly federation with multiple participants collaborating with a sole FL server.

Abstract

from arXiv · show

In recent years, mobile devices are equipped with increasingly advanced sensing and computing capabilities. Coupled with advancements in Deep Learning (DL), this opens up countless possibilities for meaningful applications. Traditional cloudbased Machine Learning (ML) approaches require the data to be centralized in a cloud server or data center. However, this results in critical issues related to unacceptable latency and communication inefficiency. To this end, Mobile Edge Computing (MEC) has been proposed to bring intelligence closer to the edge, where data is produced. However, conventional enabling technologies for ML at mobile edge networks still require personal data to be shared with external parties, e.g., edge servers. Recently, in light of increasingly stringent data privacy legislations and growing privacy concerns, the concept of Federated Learning (FL) has been introduced. In FL, end devices use their local data to train an ML model required by the server. The end devices then send the model updates rather than raw data to the server for aggregation. FL can serve as an enabling technology in mobile edge networks since it enables the collaborative training of an ML model and also enables DL for mobile edge network optimization. However, in a large-scale and complex mobile edge network, heterogeneous devices with varying constraints are involved. This raises challenges of communication costs, resource allocation, and privacy and security in the implementation of FL at scale. In this survey, we begin with an introduction to the background and fundamentals of FL. Then, we highlight the aforementioned challenges of FL implementation and review existing solutions. Furthermore, we present the applications of FL for mobile edge network optimization. Finally, we discuss the important challenges and future research directions in FL

I. INTRODUCTION

The survey introduces FL as a privacy-preserving approach for collaborative learning at mobile edge networks, where devices train locally and share model updates. It reviews implementation challenges, existing solutions, and applications of FL to mobile edge network optimization.

  • Motivation: Mobile devices’ growing sensing, computing, and communication capabilities, combined with DL, create opportunities for crowdsensing and other applications.Examples include medical applications and air-quality monitoring.
  • Motivation: Cloud-centric learning centralizes measurements, photos, videos, and location data, creating privacy, latency, and communication-efficiency concerns.The survey motivates moving intelligence closer to where data is produced.
  • MEC and FL: MEC brings computation and storage closer to data sources, but conventional edge training still transmits personal data and can incur significant communication costs.Persistent-training applications are especially unsuitable under this arrangement.
  • MEC and FL: FL keeps training data on end devices, sends model weights to an FL server for aggregation, and repeats the process until desirable accuracy is achieved.This supports collaborative ML model training without sending raw data to the server.
  • Benefits: FL can reduce communication burden, enhance privacy under non-malicious-participant assumptions, and lower latency by enabling local model updates and decisions.The survey identifies these as advantages over conventional cloud-centric training.
  • Challenges and scope: At scale, FL faces communication-cost, resource-allocation, privacy, and security challenges caused by heterogeneous devices and large model updates.The survey reviews solutions and applications for mobile edge network optimization.

II. BACKGROUND AND FUNDAMENTALS OF FEDERATED LEARNING

This section provides background on DNNs and the fundamentals needed to understand federated training, including representation learning, forward computation, and gradient-based optimization.

  • Federated Learning Fundamentals: FL consists of local model training on end devices followed by global aggregation of updated parameters at the FL server.The survey uses this process as the basis for discussing federated DNN training.
  • Deep Learning: Traditional ML relies on hand-engineered features, whereas DNNs automatically learn representations from raw data and often perform better when data is abundant.This difference motivates the survey’s focus on DNN model training.
  • Deep Learning: A DNN contains input, hidden, and output layers, mapping inputs through weighted nonlinear transformations to produce outputs such as classification scores.The highest image-classification score indicates the predicted class.
  • DNN Training: DNN training optimizes model weights using stochastic gradient descent, whose updates combine a learning rate with the loss derivative for each weight.The training dataset supplies inputs for weight optimization.
  • DNN Training: Minibatch gradient descent averages gradients over random subsets of training samples, avoiding the slow training and memorization risks associated with full-batch gradient descent.Backpropagation derives gradient matrices from the input gradient e.

B. Federated Learning

Federated Learning (FL) enables participants to collaboratively train a shared model while keeping private datasets on their devices. The process alternates local training and server aggregation until convergence or a target accuracy is reached.

  • FL architecture: FL separates data owners from the model owner: participants train local models on private datasets and send parameters for global aggregation.Unlike centralized training, raw datasets are not first combined at the server.
  • Training process: The server initializes the task and global model, then broadcasts them with training requirements and hyperparameters to selected participants.The initialized global model is denoted w0_G.
  • Training process: Each participant uses local data and device resources to update its local model, after which the updated parameters are sent to the server.Local optimization is performed on minibatches from the participant’s dataset.
  • Training process: The server aggregates participant models into an updated global model and repeats the process until global loss converges or desirable accuracy is achieved.The section describes averaging aggregation and repeated training rounds.
  • Model training: The FL process applies to machine-learning models trained using stochastic gradient descent, including SVMs, neural networks, and linear regression.The paper introduces common loss functions for these model classes.

C. Statistical Challenges of FL

FL faces statistical challenges because participants’ private datasets may be non-IID and globally imbalanced, while local data cannot be centrally repartitioned. Existing solutions address personalization, rebalancing, and convergence.

  • Data heterogeneity: FL participants may hold non-IID datasets, and FedAvg performance can deteriorate substantially relative to centralized training.For CIFAR-10, one cited study reports 51% lower accuracy for a FedAvg-trained CNN than a centrally trained CNN.
  • Data heterogeneity: Global class imbalance can reduce model accuracy, motivating Astraea’s participant-side data-distribution reporting and minority-class augmentation before training.The rebalancing step is performed during initialization.
  • Personalization: Multi-task approaches learn separate but structurally related participant models to account for heterogeneous data distributions.MOCHA modifies the loss to model relationships among tasks, but cannot be applied to non-convex deep-learning models.
  • Personalization: FEDPER shares FedAvg-trained base layers while participants independently train personalization layers, making it suitable for recommender systems with diverse preferences.The approach separates shared representation learning from participant-specific training.
  • Convergence: FedProx adds a tunable restriction on local updates to address convergence concerns in federated training.Faster convergence can reduce communication rounds, participant dropouts, time, and resource use.

D. FL protocols and frameworks

FL protocols organize decentralized training through participant selection, configuration, and reporting, while frameworks support implementation. Practical deployments must address connectivity, communication security, heterogeneous devices, and untrusted participants.

  • Protocols: The proposed FL protocol improves scalability by addressing unstable device connectivity and communication security at the system level.It is designed for practical federated deployments involving connected mobile devices.
  • Protocols: Each training round has three phases: selecting connected devices, configuring aggregation and schedules, and reporting updates for aggregation.Selection criteria can be calibrated to goals such as training efficiency, while configuration may use simple or secure aggregation.
  • Security: Communication security requires mechanisms such as secure aggregation and differential privacy to reduce the risk of linking updates to participant identities.Secret sharing with authenticated encryption is described for transmitting local updates.
  • Frameworks: TensorFlow Federated provides high-level FL and Federated Core layers, while PySyft preserves the native PyTorch interface and simulates participants as virtual workers.These frameworks support decentralized or simulated federated computation.
  • Unique characteristics: FL differs from data-center distributed learning through slow or unstable communication, heterogeneous device resources, and privacy and security concerns.Participants may vary in bandwidth, CPU state, battery level, willingness to participate, and connection stability.

III. COMMUNICATION COST

Federated Learning communication can become a training bottleneck because repeated exchanges of high-dimensional updates are costly, especially under unreliable or asymmetric networks. Reviewed solutions increase local or edge computation, compress or selectively transmit updates, and use hierarchical aggregation, with effectiveness depending on data distribution and system conditions.

  • Communication challenges: Repeated communication rounds and high-dimensional model updates can create a training bottleneck, worsened by unreliable networks and asymmetric connection speeds.The survey identifies communication efficiency as a central requirement for FL training.
  • Communication-reduction approaches: Communication costs can be reduced by performing more computation before aggregation, compressing updates, or selectively transmitting important updates.Compression may use sparsification, quantization, or subsampling, while importance-based updating omits less relevant updates.
  • End-device computation: FedAvg reduced communication rounds by more than 30 times for IID MNIST CNN data, 2.8 times for non-IID data, and 95.3 times for non-IID LSTM simulations.The gains came from increasing computation per participant before communication; increased parallelism provided little improvement beyond a threshold.
  • Edge and end computation: The reviewed computation-increasing approaches include more local passes, two-stream training using the global model as a reference, and intermediate edge-server aggregation.These approaches are summarized in Fig. 6 as alternatives for increasing computation at edge and end devices.
  • Hierarchical aggregation: HierFAVG can reduce communication overhead relative to FedAvg through intermediate edge aggregations, but may fail to reach 90% accuracy for some non-IID settings.Failures are reported when edge-cloud divergence is large or many edge servers are involved, motivating calibration of local and edge aggregation intervals.

B. Model Compression

Model compression reduces FL communication costs through structured or sketched updates, lossy compression, and federated dropout, but accuracy, convergence, and computation tradeoffs remain task- and participation-dependent.

  • Update compression: Structured updates constrain participant updates to low-rank or random-mask forms, while sketched updates encode updates through subsampling, quantization, or rotation before transmission.These methods reduce the information communicated to the server, which decodes or reconstructs updates before aggregation.
  • Update compression: On CIFAR-10, random-mask structured updates outperform low-rank updates and sketching approaches in accuracy.Sketching removes training information, whereas combining subsampling, quantization, and rotation provides higher compression and faster convergence with some accuracy loss.
  • Update compression: 256 times fewer bits and 85% accuracy are achieved using 2-bit quantization while sketching out all but 6.25% of update data.More participants per round can support more aggressive subsampling while retaining higher training accuracy.
  • Model-to-participant compression: Kashin quantization matches the uncompressed baseline with nearly 8 times lower communication cost at 4 bits, while 25% federated dropout reduces communicated model size by around 43%.More aggressive dropout can slow convergence, so compression strength must be controlled.
  • Tradeoffs and scalability: The reviewed methods reduce communication in both directions, but compression can reduce accuracy, increase computation, or delay convergence.The survey calls for formalizing these tradeoffs because they vary across tasks and numbers of FL participants.
  • Tradeoffs and scalability: Communication cost remains a key obstacle to scaling FL because complex models contain millions of parameters and mobile devices have slow upload speeds.The survey also notes that empirical tuning of local iterations and unresolved convergence issues in heterogeneous edge settings limit scalability.

IV. RESOURCE ALLOCATION

Resource allocation in FL must address heterogeneous device resources, data distributions, training delays, fairness, and participation incentives. Surveyed solutions include adaptive participant selection, data-aware aggregation, deep reinforcement learning, and fairness-oriented objectives.

  • Resource-allocation challenges: Heterogeneous datasets, computing capabilities, energy states, and willingness to participate make resource allocation central to FL efficiency.The survey identifies participant selection, joint radio and computation management, adaptive aggregation, and incentive mechanisms as key issues.
  • Participant Selection: FedCS gathers channel and computing information, then greedily selects the maximum number of participants able to finish before a preset aggregation deadline.The protocol iteratively selects participants with the shortest model upload and update times.
  • Participant Selection: Hybrid-FL selects participants whose uploaded data forms an approximately IID server dataset, improving non-IID classification accuracy with data from 1% of participants.The approach merges a model trained on the collected IID data with the global participant-trained model, but can create privacy, security, and transmission-cost concerns.
  • Dynamic resource allocation: Deep Q-Learning and deep reinforcement learning optimize resource allocation under dynamic wireless, energy, mobility, channel, and coverage conditions.The surveyed methods include channel selection and device energy-consumption optimization without prior knowledge of the mobile network.
  • Fairness: Efficiency-focused allocation can exclude stragglers and overrepresent data from devices with stronger computing resources, motivating fairness-aware objectives.The q-FFL formulation can reduce testing-accuracy variance and converge faster than AFL, although some calibrations may slow convergence because of stragglers.

B. Joint Radio and Computation Resource Management

The survey reviews multi-access BAA as an alternative to OFDMA for reducing FL communication latency through over-the-air aggregation. BAA reuses bandwidth and can preserve test accuracy while its scheduling choice affects performance under different mobility conditions.

  • Over-the-air aggregation: BAA integrates communication and computation through signal superposition, reusing the whole bandwidth unlike OFDMA’s orthogonal allocation.In OFDMA, communication latency increases with participant count, whereas multi-access latency is independent of that count.
  • Performance comparison: BAA achieves similar test accuracy to OFDMA while reducing latency by 10 times to 1000 times.The reported comparison concerns the proposed BAA scheme and the OFDMA scheme.
  • Scheduling: Cell-interior scheduling outperforms all-inclusive scheduling in test accuracy under high mobility, while alternating scheduling outperforms cell-interior scheduling under low mobility.The three schemes are cell-interior, all-inclusive, and alternating scheduling.
  • Over-the-air aggregation: Subsequent designs add error accumulation and gradient sparsification to over-the-air computation, while another approach selects more devices under signal-quality constraints.These extensions address dropped gradients and aggregation-error effects associated with power limits and signal distortion.

C. Adaptive Aggregation

Adaptive and asynchronous aggregation address stragglers, heterogeneous participation, and changing resource constraints, while incentive mechanisms target participant quality and utility. The surveyed approaches expose tradeoffs involving convergence, resource efficiency, fairness, and federation structure.

  • Adaptive aggregation: Synchronous FedAvg is limited by the straggler effect because each round waits for the slowest device before aggregation.Asynchronous FL instead updates the global model whenever a local update arrives and accommodates participants joining mid-round.
  • Adaptive aggregation: Adaptive global aggregation varies aggregation frequency to balance desirable model performance with efficient use of resources such as energy.The MEC model separates local updates, edge aggregation, and global aggregation.
  • Incentive mechanism: Stackelberg pricing models participants as training-service providers and the model owner as buyer, with valuable model updates receiving higher equilibrium prices.Cooperative relaying is proposed to support model-update transfer and trading.
  • Incentive mechanism: Contract-theoretic incentives use self-revealing contracts satisfying individual-rationality and incentive-compatibility constraints to match participants with their types.Participants achieve maximum utility when selecting contracts corresponding to their types, and the approach outperforms Stackelberg incentives in model-owner profit.
  • Participant selection: Reputation-based selection excludes participants below a threshold and can improve model accuracy by detecting unreliable workers.Reputation combines direct interaction opinions with indirect opinions from other task publishers and is stored in a reputation blockchain.
  • Lessons learned: Resource-allocation methods involve tradeoffs such as communication-cost reduction versus computation cost or inference accuracy, motivating customizable scalable designs.Asynchronous FL improves tolerance to stragglers and mid-round participation, but convergence can be delayed under non-IID, unbalanced data; incentive studies often assume a monopoly federation and privacy assumptions may fail with malicious parties.

V. PRIVACY AND SECURITY ISSUES

FL reduces the need to exchange raw data, but shared models can still leak sensitive information and enable attacks. The survey reviews privacy inference threats, model-extraction risks, and differential-privacy-based protection.

  • Privacy threats: Malicious participants can infer sensitive attributes or whether a person’s data contributed to training by inspecting shared models.Reported examples include inferring gender, occupation, location, and dataset membership.
  • Privacy threats: Trained models can leak correlations from their training samples, allowing adversaries to infer user ethnicity or gender.Model-inversion methods can reconstruct a victim’s face image from its label with very high accuracy.
  • Privacy threats: Prediction-query attacks can extract information from a trained model when a malicious participant can query it.The reviewed work reports extraction of model information from a data owner through prediction queries.
  • Privacy defenses: Differentially private stochastic gradient descent adds randomized noise to trained parameters before they are sent to the server.The technique is presented as a privacy-preserving mechanism for deep-learning algorithms.

3) Collaborative training solutions:

Collaborative-training defenses seek to protect participants from malicious servers and inference attacks while retaining model-training functionality. The survey also covers poisoning attacks and their difficult-to-prevent consequences in large federations.

  • Collaborative training: Collaborative deep-learning frameworks can let participants learn a global model without uploading explicit training models to the server.This addresses privacy threats posed by malicious servers.
  • Collaborative training: Selective parameter sharing and differential privacy make information exploitation harder, but GAN-based attacks can infer sensitive information from only part of a victim’s shared parameters.A secret-sharing and extreme-boosting defense removes this inference route for other participants but relies on a trusted third party for signature keys.
  • Encryption-based solutions: Homomorphic encryption protects parameters from honest-but-curious servers while achieving the same accuracy as centralized deep learning.The reviewed approach encrypts trained parameters before transmission; related secret-sharing mechanisms provide similar protection.
  • Encryption-based solutions: Hybrid additive homomorphic encryption and differential privacy can address both curious-server exploitation and collusion with malicious participants.The reviewed encryption and secret-sharing techniques require multi-round communication, while the hybrid approach is not compared against a stated accuracy baseline.
  • Poisoning attacks: Data poisoning uses dirty-label training data to falsify global learning, while sybil attacks amplify poisoning by creating multiple malicious participants.With two malicious participants, the cited MNIST experiment reports an attack success rate of up to 96.2%.
  • Poisoning attacks: Model poisoning directly modifies submitted models and is more effective than data poisoning, especially in large-scale federations.Model-poisoning attacks are difficult to prevent because evaluating every participant’s contribution is intractable when millions participate.
  • Poisoning attacks: A semantic backdoor model-poisoning attack can achieve 100% accuracy on the attacker’s task within a single learning round.The attack embeds a backdoor function in the poisoned model shared for aggregation.

3) Free-Riding Attacks:

Free-riding lets participants benefit from the global model while contributing little training effort, shifting resource burdens onto honest participants. Blockchain-based designs aim to deter this behavior while adding implementation costs, and FL applications extend across edge-network optimization tasks.

  • Free-riding attacks: Free-riders can pretend to have few samples or train on only a small dataset to save resources while benefiting from the global model.This increases the resources required from honest participants.
  • Free-riding attacks: A blockchain-based framework can prevent free-riding and incentivize participants to contribute to learning.Related blockchain designs also provide data confidentiality and computation auditability.
  • Free-riding attacks: Blockchain-based defenses incur significant costs for implementing and maintaining miners.The cost is identified as a limitation of using blockchain technology in FL.
  • Applications: FL applications in mobile edge networks include cyberattack detection, edge caching and computation offloading, base-station association, and vehicular-network prediction tasks.These applications use collaborative learning while addressing privacy-sensitive user or device data.

A. Cyberattack Detection

Federated learning supports privacy-preserving cyberattack detection in mobile edge networks, while related optimization applications address caching and computation offloading under resource constraints.

  • Cyberattack Detection: Deep learning can detect diverse cyberattacks accurately, but sensitive training data motivates FL-based detection models for mobile edge networks.The cited discussion identifies KDDcup 1999, NSL-KDD, and UNSW-NB15 as evaluation datasets for conventional detection comparisons.
  • Cyberattack Detection: 95.6% of attacks were detected in approximately 257 ms without false alarms in a real-world smart home deployment using FL.
  • Cyberattack Detection: Blockchain-based management can identify malicious participants by recording incremental anomaly-detection model updates in a ledger.
  • Edge Caching and Computation Offloading: FL-based caching and computation offloading learn decisions collaboratively while keeping user information or request histories from the FL server.The caching scheme trains a stacked autoencoder with FedAvg, and its reported cache efficiency exceeds Thompson sampling methods.
  • Edge Caching and Computation Offloading: A resource-constrained service-placement optimization maximizes edge-served demand and outperforms placement based only on the most popular services.

C. Base Station Association

The survey describes learning-based approaches for associating users and managing mobile-edge resources, then identifies scalability, heterogeneity, communication, and participation challenges for future FL systems.

  • Base Station Association: Cell association is formulated as a mean-field game with imitation and reduced to a single-user Markov decision process solved by neural Q-learning.
  • Vehicular Networks: FL can support collaborative learning from privacy-sensitive vehicular images despite differences in vehicle computing capabilities.A multidimensional contract assigns bundles with varying data quality, compute resources, and contractual payoffs to address information asymmetry.
  • Mobile Edge Network Optimization: FL-based optimization spans dynamic edge-network applications, including caching, computation offloading, and other resource-management decisions.The survey states that DL and DRL model dynamic environments while FL preserves user privacy during training.
  • Future Research Directions: Synchronous FL is limited by the slowest device, motivating asynchronous approaches that also allow participants to join during an ongoing round.
  • Future Research Directions: Non-IID data and unequal sample counts make statistical heterogeneity important to quantify for improving FL convergence.
  • Future Research Directions: Combining communication-reduction techniques could reduce both model-update sizes and the number of communications with the FL server.

VIII. CONCLUSION

The paper surveys federated learning fundamentals, implementation challenges, and mobile-edge optimization applications. It also reviews solutions and identifies future research directions for deploying FL at scale.

  • The survey connects federated learning with mobile-edge collaborative training and reviews implementation challenges, solution approaches, and network-optimization applications.
Loading 1909.11875v2…