PaperlayerView exact revision on arXiv

Source-linked AI summary

Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge based Framework

Qiong Wu, Kaiwen He, Xu Chen

arXiv:2002.10671v3cs.NIcs.DCcs.LG

TL;DR

Traditional federated learning must exploit distributed IoT data without compromising privacy, but device, statistical, and model heterogeneity complicate direct deployment. The paper proposes the cloud-edge PerFit framework with personalized federated learning methods, and a human activity recognition case study reports improved and more consistent client performance, with communication varying substantially across methods.

  • Problem

    Device, statistical, and model heterogeneities in complex IoT environments make traditional federated learning unsuitable for direct deployment despite its privacy-preserving use of distributed data.

  • Method

    PerFit combines a cloud-edge architecture with personalized federated learning methods that exchange model information flexibly while keeping sensitive data on devices.

  • Results

    In human activity recognition, FD-CNN improves accuracy by 5.69% compared with FL-CNN, while personalized methods narrow performance differences across clients.

  • Takeaways & Limitations

    PerFit can integrate multiple personalized federated learning methods to address IoT heterogeneity and support intelligent IoT applications.

Abstract

from arXiv · show

Internet of Things (IoT) have widely penetrated in different aspects of modern life and many intelligent IoT services and applications are emerging. Recently, federated learning is proposed to train a globally shared model by exploiting a massive amount of user-generated data samples on IoT devices while preventing data leakage. However, the device, statistical and model heterogeneities inherent in the complex IoT environments pose great challenges to traditional federated learning, making it unsuitable to be directly deployed. In this article we advocate a personalized federated learning framework in a cloud-edge architecture for intelligent IoT applications. To cope with the heterogeneity issues in IoT environments, we investigate emerging personalized federated learning methods which are able to mitigate the negative effects caused by heterogeneity in different aspects. With the power of edge computing, the requirements for fast-processing capacity and low latency in intelligent IoT applications can also be achieved. We finally provide a case study of IoT based human activity recognition to demonstrate the effectiveness of personalized federated learning for intelligent IoT applications.

I. INTRODUCTION

IoT creates substantial opportunities for data-driven intelligent services, but traditional federated learning is difficult to deploy because devices, data, and models are heterogeneous. The paper therefore advocates personalized federated learning in a cloud-edge architecture to address these challenges while supporting privacy, processing efficiency, and low latency.

  • Motivation: IoT devices generate massive user data that can support task-specific machine-learning models and intelligent services.Centralized collection can be ineffective because data transmission and model deployment create practical difficulties.
  • Motivation: Federated learning collaboratively trains a shared model by aggregating local device updates without directly accessing users’ training data.Its key benefit is decoupling model training from direct data access, thereby preserving user data privacy.
  • Challenges: Traditional federated learning faces device, statistical, and model heterogeneity in complex IoT environments, making direct deployment unsuitable.These challenges include diverse device resources, non-IID data, and devices’ desire to customize models for their environments.
  • Approach: The paper investigates personalized federated learning approaches that personalize at device, data, and model levels to mitigate IoT heterogeneity.The approaches are grouped into federated transfer learning, federated meta learning, federated multi-task learning, and federated distillation.
  • Approach: PerFit combines personalized federated learning with cloud-edge computing to address heterogeneity and provide on-demand computation near IoT devices.Devices can offload intensive learning tasks to the edge, while local data can remain nearby; the framework targets fast processing and low latency.

II. MAIN CHALLENGES OF FEDERATED LEARNING IN IOT ENVIRONMENTS

Federated learning in IoT is challenged by wide variation in device resources and connectivity. These differences increase communication costs, create stragglers, and can cause device dropouts that disrupt synchronous training.

  • Device Heterogeneity: IoT devices differ in hardware, network conditions, and battery power, producing diverse computing, storage, and communication capacities.These variations create device heterogeneity within federated learning deployments.
  • Device Heterogeneity: Device heterogeneity causes high communication cost, stragglers, and fault-tolerance challenges in federated learning.Communication costs are identified as a principal concern in federated settings.
  • Device Heterogeneity: Limited computing capacity can make devices stragglers because they take longer to report model updates during synchronous rounds.Stragglers and faults are prevalent in complex IoT environments because of device heterogeneity.
  • Device Heterogeneity: Poor connectivity and energy constraints may cause participating devices to drop out of training, negatively affecting federated learning.The passage frames dropout as a practical consequence of device heterogeneity.

B. Statistical Heterogeneity

IoT devices can have different data distributions, model architectures, resource constraints, and application requirements, limiting naive federated aggregation. The proposed cloud-edge framework uses personalization and flexible information exchange to support device-specific learning.

  • B. Statistical Heterogeneity: User data distributions can be non-IID because devices reflect different usage environments and patterns.The passage identifies feature-distribution skew, label-distribution skew, and concept shift as forms of statistical heterogeneity.
  • C. Model Heterogeneity: Different IoT devices may require customized model architectures because of distinct application environments and resource constraints.Varying local architectures make naive aggregation of model weights impossible in traditional federated learning.
  • Cloud-Edge Framework: PerFit uses a cloud-edge architecture that lets devices offload intensive computing tasks to nearby gateways or servers.The framework is designed to provide on-demand edge computing power while fulfilling high-processing-efficiency and low-latency requirements.
  • Cloud-Edge Framework: Federated learning operates among end devices, edge servers, and the remote cloud while sensitive data remains on device.Personalization then fine-tunes the learning model for each individual device.
  • Cloud-Edge Framework: PerFit’s collaborative process comprises offloading, learning, and personalization stages.The stages respectively support device-edge computation, cloud aggregation of local model information, and device-specific model training.
  • Personalization: Personalized models can be lightweight for resource-limited devices and tailored to local data, application requirements, and deployment environments.Model pruning or transfer learning can help mitigate communication and computation heterogeneity.
  • Cloud-Edge Framework: The framework can exchange different kinds of model information, including selected parameters or local output class probabilities.This flexibility allows integration of federated transfer learning and federated distillation when devices use different models.

IV. PERSONALIZED FEDERATED LEARNING MECHANISMS

The paper reviews personalized federated learning mechanisms that can be integrated with PerFit for intelligent IoT applications. It organizes these mechanisms into four categories addressing different personalization needs.

  • Mechanism Categories: Federated transfer learning is one personalized federated learning mechanism integrated with PerFit.The framework reviews this category alongside three other mechanisms.
  • Mechanism Categories: Federated meta learning is one of the four categories of personalized federated learning mechanisms considered.The categories are presented as mechanisms suitable for integration with the PerFit framework.
  • Mechanism Categories: Federated multi-task learning is included among the personalized federated learning mechanisms reviewed for intelligent IoT applications.The category appears in the paper’s classification of mechanisms integrated with PerFit.
  • Mechanism Categories: Federated distillation completes the paper’s four-category classification of personalized federated learning mechanisms.These mechanisms are reviewed as options for integration with the proposed framework.

A. Federated Transfer Learning

Federated transfer learning personalizes IoT models by transferring shared knowledge while retaining device-specific components. Approaches differ in which parameters are transferred or fine-tuned, balancing personalization against communication and computation costs.

  • Federated transfer learning transfers a globally shared model to IoT devices for local personalization, mitigating statistical heterogeneity from non-IID data.
  • One approach trains a global model first, then fine-tunes specified layers on each device instead of retraining the entire model.
  • FedPer separates deep models into collaboratively trained base layers and locally owned personalization layers for device-specific inference.
  • By uploading and aggregating only part of the model, FedPer reduces computation and communication overhead in IoT environments.
  • Model pruning and compression can further support lightweight deployment after personalized models are obtained.

B. Federated Meta Learning

Federated meta learning addresses non-IID and unbalanced IoT data by training models that can rapidly adapt to individual devices. Its greater implementation complexity is balanced by robustness for devices with few data samples.

  • Federated meta learning combines collaborative training with meta learning to personalize models for devices facing non-IID and unbalanced data.
  • Meta learning trains across many similar tasks so a model can quickly adapt to a new related task using limited data.
  • Personalized FedAvg adds a MAML-based fine-tuning stage so federated global models capture device-specific information.
  • Federated meta learning has higher implementation complexity than federated transfer learning but can produce robust models for devices with few samples.

C. Federated Multi-Task Learning

Federated multi-task learning trains distinct but related models for different IoT devices while preserving personalization. It uses model relationships to share information, though full client participation can be impractical.

  • Federated multi-task learning simultaneously learns distinct tasks for different devices and captures relationships among their models without privacy risk.
  • Alternating optimization of cloud-side model relationships and task-specific parameters helps devices collaboratively train personalized local models.
  • MOCHA supports additional local computation, approximate updates for resource-limited devices, asynchronous updates, and periodic client dropouts.
  • Because the framework produces one model per task, it requires all clients to participate in every iteration, which is impractical for IoT applications.

D. Federated Distillation

Federated distillation addresses model heterogeneity by exchanging standardized knowledge rather than requiring shared model architectures. Federated augmentation separately targets skewed, non-IID client data through privacy-preserving data synthesis.

  • Model heterogeneity arises when participants want unique architectures and may not share model details because of privacy or intellectual-property concerns.
  • FedMD lets clients independently design models by translating learned knowledge into a standard format through knowledge distillation.
  • The cloud server aggregates and averages client-produced class probabilities for each data sample, then distributes the consensus back to clients.
  • Federated augmentation responds to highly skewed and non-IID user data, which can degrade model performance.
  • FAug has clients identify lacking target labels and upload a few seed samples so a collectively trained generative model can augment local data toward an IID dataset.

V. CASE STUDY

The case study evaluates personalized federated learning for human activity recognition using the MobiAct dataset and simulated heterogeneous clients. Experiments assess accuracy and communication size across personalized approaches.

  • Dataset and evaluation: The experiments evaluate personalized federated learning approaches for human activity recognition using the publicly accessible MobiAct dataset.The setup uses smartphone accelerometer and gyroscope signals from 30 volunteers treated as clients.
  • Dataset and evaluation: The evaluation measures accuracy and communication size under different personalized federated learning approaches and heterogeneity settings.The study compares multiple approaches in a cloud-edge federated setting.
  • Dataset and evaluation: Each client has 480 training samples with potentially non-IID activity distributions and 160 balanced test samples.The data are generated from predefined activities recorded through smartphone sensors.
  • Client participation: The study varies the number of participating clients per communication round to examine its effects on learning behavior and training cost.Figure 5 reports test accuracy and time cost for different client-participation levels.

1) Comparing Methods:

Personalized federated methods improve client-level activity-recognition performance while addressing non-IID variation and communication constraints. The experiments also identify a practical participation setting balancing stability and efficiency.

  • Client participation: 3.26 times longer training time was required for K = 30 than for K = 3, although converged test accuracies were similar.Smaller K values produce more erratic learning curves, so K = 5 is selected as a stability-efficiency trade-off.
  • Performance evaluation: 95.37% accuracy was achieved by FTL-3NN, 11.12% higher than FL-3NN in human activity recognition.Personalized methods capture fine-grained personal information and produce participant-specific models.
  • Performance evaluation: 85.22% average accuracy was achieved by FL-CNN, similar to centralized cCNN despite federated statistical heterogeneity.Centralized deep-learning methods outperform traditional machine-learning baselines in the reported comparison.
  • Client-level variation: Personalization narrows accuracy differences across clients, reducing the client-level degradation observed with a single federated global model.Some global-model client accuracies fall below 70%, whereas others exceed 95%.
  • Client-level variation: 5.69% accuracy improvement was reported for FD-CNN compared with FL-CNN, while client performance differences were also narrowed.The result indicates benefits for many participating clients in the reported human activity-recognition experiment.
  • Communication efficiency: All four personalized methods exceeded 90% accuracy, while communication sizes varied substantially between federated transfer learning and federated distillation.FTL communication depends on model parameters, whereas FD communication depends on output dimension and transmitted class scores.

VI. CONCLUSION

The paper concludes that PerFit combines cloud-edge computing with personalized federated learning for privacy-preserving intelligent IoT applications. Its human activity-recognition case study supports the framework’s effectiveness across IoT heterogeneities.

  • Conclusion: PerFit is a cloud-edge personalized federated learning framework designed to address device, statistical, and model heterogeneity in IoT applications.It aggregates local updates while leveraging edge computing and preserving data privacy.
  • Conclusion: The framework integrates varied personalized federated learning methods to obtain enhanced, device-specific performance in intelligent IoT applications.The paper demonstrates this approach through a human activity-recognition case study.
Loading 2002.10671v3…