PaperlayerView exact revision on arXiv

Source-linked AI summary

Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challenges

Latif U. Khan, Walid Saad, Zhu Han, Ekram Hossain, Choong Seon Hong

arXiv:2009.13012v2cs.NI

TL;DR

The paper addresses how federated learning can support IoT machine learning despite massively distributed, private, and heterogeneous data. It surveys advances, develops a taxonomy, proposes dispersed federated-learning use cases, and identifies open challenges and possible solutions, concluding that dispersed federated learning is promising for future IoT applications.

  • Problem

    Massively distributed, private, and heterogeneous IoT data make classical centralized learning challenging while federated learning still has privacy concerns.

  • Method

    The paper surveys federated-learning advances for IoT, evaluates them across multiple metrics, develops a taxonomy, proposes dispersed federated-learning use cases, and outlines open challenges with possible solutions.

  • Results

    The paper identifies robustness, privacy, and high communication-resource consumption as important issues, and presents dispersed federated learning as a key scheme for future IoT applications.

  • Takeaways & Limitations

    Dispersed federated learning can improve privacy by using homomorphic encryption against malicious sub-global aggregators and sub-global aggregation against malicious global aggregators.

  • Takeaways & Limitations

    Blockchain-based aggregation can be undesirable for federated learning because consensus algorithms introduce high latency across communication rounds.

Abstract

from arXiv · show

The Internet of Things (IoT) will be ripe for the deployment of novel machine learning algorithms for both network and application management. However, given the presence of massively distributed and private datasets, it is challenging to use classical centralized learning algorithms in the IoT. To overcome this challenge, federated learning can be a promising solution that enables on-device machine learning without the need to migrate the private end-user data to a central cloud. In federated learning, only learning model updates are transferred between end-devices and the aggregation server. Although federated learning can offer better privacy preservation than centralized machine learning, it has still privacy concerns. In this paper, first, we present the recent advances of federated learning towards enabling federated learning-powered IoT applications. A set of metrics such as sparsification, robustness, quantization, scalability, security, and privacy, is delineated in order to rigorously evaluate the recent advances. Second, we devise a taxonomy for federated learning over IoT networks. Third, we propose two IoT use cases of dispersed federated learning that can offer better privacy preservation than federated learning. Finally, we present several open research challenges with their possible solutions.

I. INTRODUCTION

Federated learning addresses IoT’s distributed, private-data setting by training models on end-devices and exchanging model updates rather than raw data. Its deployment still faces privacy, heterogeneity, resource, and security challenges.

  • Motivation: IoT applications generate massive, geographically distributed datasets, making centralized learning difficult because data migration can leak user privacy.The paper cites projections of 125 billion IoT devices by 2030 and 79.4 zettabytes of generated data.
  • Federated learning for IoT: Federated learning preserves data locality while addressing system and data heterogeneity through on-device training and aggregation of local model updates.End-devices compute local models, send updates to an aggregation server, and receive global model parameters.
  • Challenges: Despite avoiding raw-data migration, federated learning does not completely guarantee privacy and also faces robustness, security, and communication-resource challenges.The paper identifies differential privacy-aware federated learning and device-to-device protocols as possible responses.
  • Challenges: Federated learning for IoT requires substantial computational power, backup energy, and sufficient local data, especially for sensors generating few samples.Resource limitations can make local model training challenging, while autonomous driving cars are described as more suitable from a resource-usage perspective.
  • Federated learning schemes: Global aggregation may be performed at the edge, cloud, or jointly across edge and cloud servers.Edge-based learning suits regional models, whereas cloud-based learning is more suitable for generalized models across geographically distributed users.

A. Market Statistics and Research Trends

The paper situates federated learning and IoT within expanding markets, reviews prior surveys, and organizes its tutorial around advances, evaluation metrics, taxonomy, and open challenges.

  • Market statistics: The IoT market is projected to reach 1463.19 billion US Dollar in 2027 from 250.72 billion US Dollar in 2019, at a 24.9% CAGR.The Asia Pacific region generated the highest revenue in 2018, with China holding the highest share within that region.
  • Research trends: Existing surveys primarily address resource optimization, incentive mechanisms, and federated learning algorithm design, while this paper focuses on IoT applications and develops a taxonomy.The authors state that their open research challenges also differ significantly from those in prior surveys.
  • Our tutorial: The tutorial critically evaluates recent federated learning literature relevant to IoT applications.Its evaluation uses scalability, quantization, robustness, sparsification, security, and privacy as key metrics.
  • Our tutorial: The taxonomy covers optimization, incentives, security and privacy, aggregation modes, device design, local models, resources, miners, edge collaboration, and cloud-server design.These parameters span both learning and system-level design choices for federated learning over IoT networks.
  • Our tutorial: The paper presents several open research challenges together with possible solutions.The tutorial’s organization places these challenges after the review of recent IoT-related advances and taxonomy.

II. FEDERATED LEARNING FOR IOT: FUNDAMENTAL CONCEPTS

Federated learning for IoT combines device-side model computation with aggregation at edge, cloud, or blockchain-based infrastructure. The architecture also incorporates sparsification, quantization, encryption, and channel coding to address resource, privacy, and communication constraints.

  • High-level architecture: Federated learning-enabled IoT architectures select participating end-devices according to resource and communication constraints before local model computation.Candidate criteria include clean datasets, computational resources, and backup power.
  • Security and privacy: Local learning updates can be protected with additive homomorphic encryption, which enables aggregation without decrypting individual models but increases computation and communication costs.Encryption and decryption require multiple modular exponentiations.
  • Quantization and encryption: BatchCrypt combines batch encoding, quantization, and analytical quantization modeling to improve computation and communication efficiency while preserving aggregated model quality similar to cross-silo homomorphic-encryption federated learning.The passage identifies aggregated model quality and efficiency as BatchCrypt’s main advantage.
  • Channel coding: Short block-length codes and other channel codes can reduce packet errors, but code complexity must match IoT devices’ computation constraints.BCH codes are reported as having the highest reliability under optimal decoding among the compared codes.
  • Aggregation: Local models may be aggregated at edge servers, cloud servers, or blockchain miners before global updates are returned to end-devices.FedAvg uses averaging for edge- or cloud-server aggregation, whereas blockchain aggregation distributes model exchanges among miners.

B. Federated Learning Schemes

Federated learning schemes address heterogeneous devices and data by defining a global optimization objective, local update procedures, and aggregation strategies. FedAvg improves on single-step federated SGD through multiple local iterations, while FedProx adds a proximal mechanism for heterogeneity.

  • Heterogeneity: Federated learning must account for system heterogeneity in device resources and wireless conditions and data heterogeneity in unbalanced, non-IID local datasets.These heterogeneities motivate the design of federated optimization schemes.
  • Optimization objective: The global objective minimizes a loss function over device-local datasets, with the loss depending on the IoT task, such as prediction or classification.Each device contributes a local dataset, and the global model weights are optimized across participating data.
  • FedAvg: FedAvg performs multiple local iterations before sending model updates to the aggregation server, improving on federated SGD’s single local-update pattern.Algorithm 1 selects a client subset and runs device updates in parallel.
  • Federated SGD: Federated SGD computes local gradients on every participating device and aggregates them without migrating local datasets to the centralized server.The approach uses one local gradient computation per communication round before aggregation.
  • FedAvg limitations: FedAvg is difficult to prove convergent in heterogeneous scenarios because averaging dissimilar local objectives can steer devices toward local rather than global optima.Higher local iteration counts can also prevent some devices from completing within the communication round.
  • FedProx: FedProx adds a proximal term that restricts local updates toward the global model, reducing the impact of heterogeneous data on local models.The cited discussion presents this as a response to FedAvg’s limitations.

C. Local Learning Models

IoT federated learning uses application-dependent local models, optimized weight updates, and implementation frameworks. Model architecture and training settings must balance learning performance with end-device energy and resource limits, while frameworks still need stronger wireless support.

  • Local model selection: CNN, FNN, LSTM, and SVM are among the local learning models usable for different IoT applications.The paper states that local-model choice strictly depends on the application considered.
  • Architecture design: Neural architecture search can identify an efficient local-network architecture from available candidates while accounting for IoT end-device energy limitations.The cited passage motivates NAS as a way to choose neural-network size.
  • Weight optimization: SGD updates local model weights using loss derivatives multiplied by a learning rate, and mini-batch size affects global federated-model performance.The learning rate represents the gradient-descent step size.
  • Training settings: For FedAvg on MNIST, client fraction C = 0.1 and batch size 10 generally performed better than full-batch training.The passage also emphasizes that client fraction, batch size, and learning rate require careful selection.
  • Frameworks: PySyft, FedML, TensorFlow Federated, LEAF, and Paddle FL provide frameworks for implementing federated learning across secure computation, topology, datasets, APIs, and large-scale applications.Their capabilities span encrypted computation, network-topology management, distributed datasets, experimental scenarios, and distributed clusters.
  • Framework requirements: Existing frameworks do not effectively address wireless-channel uncertainty and limited communication resources, motivating IoT frameworks with channel-coding support.The paper identifies channel coding as a possible way to improve federated-learning performance over IoT networks.

III. STATE-OF-THE-ART

The paper evaluates federated-learning advances for IoT using metrics spanning participation, communication efficiency, failure tolerance, scalability, and privacy. It organizes the literature by centralized, distributed, and hierarchical aggregation while highlighting unresolved security and robustness issues.

  • Evaluation approach: Recent federated-learning works for IoT are rigorously evaluated using metrics derived from the literature and summarized in comparative tables.The evaluation focuses on advances enabling IoT-based smart applications.
  • Security and privacy: Security and privacy remain concerns because servers or malicious users can infer sensitive device information from exchanged model updates or local models.Federated learning preserves privacy more than centralized data transfer but does not eliminate leakage risks.
  • Scalability: Scalability measures the ability to incorporate more devices under limited communication resources, using resource optimization, device selection, or model-parameter compression.Allowing more devices to participate can offer better performance according to the cited discussion.
  • Quantization: Quantization reduces local-update size, increasing throughput and subsequently reducing federated-learning convergence time.The metric concerns schemes that compress model updates before transmission.
  • Robustness: Robustness concerns continued federated-learning operation during edge or cloud aggregation-server failure, which can otherwise interrupt the process.The cited definition focuses on centralized aggregation failure.
  • Taxonomy: The literature is categorized by centralized, distributed, and hierarchical aggregation architectures.Hierarchical schemes can perform intermediate aggregation before a central global aggregation.

A. Advances Based on Centralized Aggregation

Centralized aggregation remains a common federated-learning design for IoT, with end-devices sending local model updates to an edge or cloud server. Recent work addresses wireless resource allocation, client selection, malicious devices, personalization, and application-specific privacy concerns.

  • End-devices compute local models and send updates to a centralized edge or cloud server for global aggregation.
  • Joint wireless resource, power, and user-selection optimization can reduce federated-learning loss while supporting scalability and sparsification.The framework derives a closed-form convergence expression accounting for wireless factors.
  • Centralized aggregation still faces security and robustness limitations, including vulnerability to attacks and possible server failure from physical damage.Encryption can further improve security, while centralized-server failure creates robustness concerns.
  • CoLearn targets malicious devices and asynchronous participation by resource-constrained IoT devices in edge-network federated learning.The architecture uses PySyft and Manufacturer Usage Description (MUD).
  • FedCS selects clients according to available resources to improve the efficiency of federated-learning participation.Multi-base-station scenarios additionally require resource-efficient devices and base-station association to minimize transmission latency.
  • Application studies include personalized healthcare, privacy-preserving object detection, vehicular image classification, and federated-learning-enabled augmented reality.FedHealth uses transfer learning for personalization; the object-detection study considers IID, non-IID, and extreme non-IID data; augmented-reality validation uses CIFAR-10.

B. Advances Based on Distributed Aggregation

Distributed aggregation replaces a single centralized aggregator with multiple servers that exchange local models before global aggregation. Blockchain-enabled aggregation is presented as a robustness-oriented design, while dense networks still require efficient resource and power management.

  • Distributed aggregation sends end-device updates to multiple aggregation servers that share associated local models before global aggregation.Unlike centralized aggregation, this design does not require a single centralized aggregation server.
  • FL-Block uses decentralized, blockchain-enabled aggregation and cross-verification to offer robustness against poisoning attacks and centralized-server malfunction.Local models are transmitted to fog servers, followed by cross-verification, mining, and block propagation.
  • Dense device-to-device networks can improve scalability by incorporating more nodes, but require efficient resource allocation and power control.Suggested resource-allocation approaches include optimization theory, heuristics, and game theory; power control targets packet-error rates and model accuracy.
  • Hierarchical aggregation performs local-model aggregation at edge servers before a subsequent central aggregation at the cloud.The approach is presented as combining aspects of centralized and distributed aggregation.
  • Hierarchical federated learning depends on handling data heterogeneity and selecting suitable local-iteration and edge-aggregation counts.The passage links these design choices to performance improvement.

D. Lessons Learned and Recommendations

The paper identifies communication, robustness, heterogeneity, trust, security, and privacy as central design challenges for federated learning over IoT networks. It organizes these concerns in a layered taxonomy and recommends compression, robust aggregation, adaptive protocols, secure sharing, and privacy mechanisms.

  • Communication compression is needed because IoT networks have limited communication resources and many participating devices.Gradient compression reduces transmitted local-model size, while model-broadcast compression reduces update representation size.
  • Robust federated learning must address attacks and physical damage that can disrupt a single centralized aggregation server.The paper identifies distributed designs as one direction for addressing this robustness issue.
  • Heterogeneity- and noise-aware protocols are needed because devices differ in computation, dataset size, data distribution, and local-data noise.FedProx adds a scaled proximal term to the FedAvg local loss, although its scaling constant may be difficult to adjust across applications.
  • Distributed aggregation should share model updates securely and trustfully, while blockchain introduces high latency through consensus algorithms.This creates a security–latency trade-off for blockchain-based federated learning.
  • The proposed taxonomy places global aggregation, resources, local models, incentives, optimization, security, and privacy across end-device, edge, and cloud layers.It also includes miners, cloud-server design, and edge collaboration as federated-learning parameters.
  • Federated learning does not completely eliminate privacy leakage because malicious devices or aggregation servers may infer private information from model updates.Differential privacy adds noise to local model parameters before transmission to the global aggregation server.

B. Resources

Federated learning over IoT networks requires joint management of computation, communication, energy, accuracy, and iteration resources. These resources create trade-offs that depend on device heterogeneity and aggregation design.

  • Computational resources: Local learning computation depends on device performance, dataset size, operating frequency, and local iterations.The framework considers both end-device and aggregation-server computational resources.
  • Energy-computation trade-off: Increasing device operating frequency decreases local-model computation time but increases energy consumption proportionally to its square.This creates a trade-off between computation delay and energy consumption.
  • Communication resources: Federated learning can use centralized edge/cloud aggregation or distributed aggregation through miner-enabled base stations.The aggregation architecture determines how local updates are exchanged and combined.
  • Iteration trade-offs: For fixed global accuracy, computational cost is represented by local iterations, whereas communication cost is represented by global communication rounds.These costs have contradictory relations under the same target accuracy.
  • Iteration trade-offs: Lower relative local accuracy requires more local iterations and fewer global iterations, whereas higher relative local accuracy has the opposite requirement.Figure 11 describes this relationship for fixed global accuracy.
  • Open resource challenges: Federated learning over wireless networks requires joint optimization of energy and latency, including for multitask applications.The section identifies multitask energy-latency optimization as a needed extension.

C. Local Learning Models

IoT federated learning requires participation mechanisms that account for heterogeneous devices, learning models, and implementation settings. Incentive design is needed to motivate device participation across centralized and blockchain-based architectures.

  • Local learning models: Local learning-model selection depends on the IoT task and the end-device’s computational power and backup energy.For a CNN, architectural choices such as hidden layers and activation functions affect performance.
  • Incentive mechanisms: Large-scale federated learning adoption requires incentive mechanisms for end-device participation.Different federated-learning implementations may require different incentive designs.
  • Centralized architectures: Centralized federated learning models interaction between devices and an edge/cloud server to train the global model.The literature summarizes several approaches for designing this interaction.
  • Blockchain-based architectures: Blockchain-based federated learning uses devices and miners, with miners verifying and exchanging local model parameters before consensus.A block containing verified local models is added after consensus completes.

E. Miners Classification for Federated Learning based on Blockchain

Blockchain-based federated learning uses miners to support secure and trusted model-parameter exchange, but miner mobility, wireless communication, consensus, and resource management affect convergence and scalability.

  • Miner categories: Blockchain-based federated learning uses static, mobile, or flying miners to support secure and trusted exchange of model parameters.The section identifies base stations, autonomous cars, and UAVs as miner types.
  • Blockchain operation: Every miner stores associated local updates in a block and participates in repeated hash computation to generate the distributed ledger.The block body stores updates, while the header contains control information.
  • Miner comparison: Base-station miners have lower association complexity and faster backhaul links than mobile and flying miners.Fast backhaul can reduce the forking effect, whereas UAV propagation delays make forking more prominent.
  • Miner comparison: Mobile and flying miners require additional design because mobility can interrupt communication ranges and increase coordination difficulty.Radio-access resources must also be optimized to reduce blockchain-based federated-learning convergence time.
  • Hierarchical aggregation: Hierarchical edge-cloud aggregation reduces communication-resource consumption through two levels of aggregation.Edge aggregation requires fewer communication resources than direct aggregation at a remote cloud.
  • Hierarchical aggregation: Hybrid edge-cloud collaboration can produce weight divergence between devices and edge servers and between edge servers and the cloud.The divergence depends on the federated-learning scheme and data heterogeneity, and may vary with local and edge aggregation counts.
  • Cloud resources: Cloud-based aggregation must manage hundreds to millions of devices and updates ranging from kilobytes to tens of megabytes.Containerization is presented as preferable to virtual machines because of its lower overhead.
  • Wireless reliability: Wireless federated optimization must account for packet errors and balance channel-coding performance, overhead, and complexity.The section proposes excluding devices with higher packet-error rates from global-model computation.

I. Operation Modes Based on Global Aggregation Fashion

The paper classifies federated learning by global-aggregation fashion and describes centralized, distributed, hierarchical, collaborative, and dispersed modes. Dispersed aggregation addresses communication and privacy concerns through group-level sub-global models.

  • Aggregation taxonomy: Federated learning is categorized into centralized, distributed, collaborative, hierarchical, and dispersed aggregation modes.The categories differ in where and how local models are aggregated.
  • Aggregation taxonomy: Distributed aggregation uses multiple base-station servers, while hierarchical aggregation performs edge aggregation before cloud aggregation.These modes avoid relying exclusively on one centralized aggregation operation.
  • Collaborative federated learning: Collaborative federated learning enables devices unable to reach a centralized base station to participate through device-to-device model-update sharing.Its operation depends on successful communication among participating devices.
  • Robustness and privacy: Centralized, hierarchical, and collaborative schemes face robustness challenges and potential privacy leakage from sensitive-information inference.Differential privacy can add noise to model parameters but increases complexity.
  • Dispersed federated learning: Dispersed federated learning computes sub-global models within groups, transfers them between groups, and aggregates them iteratively into a global model.Communication resources can be reused within groups for more efficient operation.
  • Dispersed federated learning: Dispersed aggregation can improve privacy preservation because a global aggregator cannot easily infer end-device private information from sub-global models.The approach also requires efficient resource allocation for further performance enhancement.
  • IoT use case: Intelligent transportation motivates federated learning because autonomous vehicles generate frequent data updates and experience mobility-related connectivity changes.Transmitting all vehicle data would create high communication overhead.

J. End-Device Design for Federated Learning

End-device design for federated learning must balance computation, energy, programmability, communication, heterogeneity, security, and mobility. The section identifies hardware, software, and networking directions for making IoT federated learning practical.

  • Higher CPU operating frequencies reduce local learning time but increase device power consumption.
  • Programmable hardware supports models with different application complexities but increases data movement, computation, and weight-storage energy costs.
  • Neural architecture search identifies suitable architectures from a search space for a particular dataset and application.
  • Federated learning design must account for heterogeneous datasets, energy, computation, and communication resources that affect algorithms such as FedAvg and FedProx.
  • Hardware-software co-design is recommended for end-devices, alongside authentication, communication allocation, dispersed aggregation, and mobility management.

V. OPEN RESEARCH CHALLENGES

The paper frames IoT federated learning as an open research agenda involving scalable resource use, data heterogeneity, mobility, security, privacy, and communication constraints. It pairs these challenges with proposed directions such as sparsification, clustering, and homomorphic encryption.

  • The paper presents open challenges and possible solutions that differ from those in existing federated-learning surveys and tutorials.
  • Sparsification can reduce wireless-resource use by selecting only a small set of devices to transmit local model parameters.
  • Data and system heterogeneity challenge FedAvg’s synchronous-work assumption, motivating clustering and improved protocols for statistically heterogeneous devices.
  • Device mobility can cause coverage loss during training, motivating federated-learning protocol modifications for seamless communication.
  • Secure federated learning must address model-update tampering and sensitive-information inference, with homomorphic encryption as one proposed approach.

E. Secure and Trustful Aggregation-Enabled Federated Learning

Secure and trustful aggregation addresses malicious updates, interference, communication limits, and heterogeneous device resources in IoT federated learning. The paper identifies low-complexity security, resource allocation, quantization, and adaptive training as directions, and concludes that dispersed federated learning is promising for massive IoT deployments.

  • Malicious local updates can slow convergence or prevent global-model convergence, motivating low-complexity protocols for secure and trustful verification.
  • Joint transmit-power allocation and device association can reduce federated-learning cost while protecting cellular users during spectrum reuse.
  • Quantization can reduce communication consumption when sparsification alone is insufficient, but signal distortion introduces global-model errors.
  • Heterogeneous computation, energy, and communication resources significantly affect local training time and federated-learning convergence.
  • The paper concludes that dispersed federated learning is a key prospective scheme for IoT applications with massive numbers of end-devices.
  • In dispersed federated learning, homomorphic encryption protects against malicious sub-global servers while sub-global aggregation protects against malicious global servers.
Loading 2009.13012v2…