PaperlayerView exact revision on arXiv

Source-linked AI summary

A Federated Learning Aggregation Algorithm for Pervasive Computing: Evaluation and Comparison

Sannara Ek, François Portet, Philippe Lalanda, German Vega

arXiv:2110.10223v1cs.LGcs.AIcs.DC

TL;DR

Pervasive applications need federated learning that handles heterogeneous, non-IID client data while balancing generalization and personalization. The paper proposes FedDist, which matches neuron dissimilarities and can modify model architecture, and evaluates it on smartphone Human Activity Recognition. FedDist outperforms the compared federated algorithms across global, personalization, and generalization measures in the reported experiments.

  • Problem

    Federated learning lacks realistic, structured evaluation and suitable aggregation for pervasive settings with heterogeneous, non-IID data and competing generalization and personalization goals.

  • Method

    FedDist uses neuron-distance matching to adapt the federated model architecture, alongside an evaluation method measuring generalization and personalization on smartphone Human Activity Recognition.

  • Results

    FedDist outperforms the other evaluated federated algorithms on global, personalization, and generalization accuracy, including 74.23% generalization accuracy versus 72.99% for FedAvg.

  • Takeaways & Limitations

    FedDist provides the best reported overall performance while adapting the CNN architecture across communication rounds for heterogeneous pervasive data.

Abstract

from arXiv · show

Pervasive computing promotes the installation of connected devices in our living spaces in order to provide services. Two major developments have gained significant momentum recently: an advanced use of edge resources and the integration of machine learning techniques for engineering applications. This evolution raises major challenges, in particular related to the appropriate distribution of computing elements along an edge-to-cloud continuum. About this, Federated Learning has been recently proposed for distributed model training in the edge. The principle of this approach is to aggregate models learned on distributed clients in order to obtain a new, more general model. The resulting model is then redistributed to clients for further training. To date, the most popular federated learning algorithm uses coordinate-wise averaging of the model parameters for aggregation. However, it has been shown that this method is not adapted in heterogeneous environments where data is not identically and independently distributed (non-iid). This corresponds directly to some pervasive computing scenarios where heterogeneity of devices and users challenges machine learning with the double objective of generalization and personalization. In this paper, we propose a novel aggregation algorithm, termed FedDist, which is able to modify its model architecture (here, deep neural network) by identifying dissimilarities between specific neurons amongst the clients. This permits to account for clients' specificity without impairing generalization. Furthermore, we define a complete method to evaluate federated learning in a realistic way taking generalization and personalization into account. Using this method, FedDist is extensively tested and compared with three state-of-the-art federated learning algorithms on the pervasive domain of Human Activity Recognition with smartphones.

I. INTRODUCTION

Pervasive computing increasingly combines connected devices, edge resources, and machine learning, but cloud-centered architectures create security, latency, and cost limitations. Federated learning addresses these challenges by training models locally and aggregating them centrally, while FedDist is proposed and evaluated for heterogeneous smartphone activity-recognition data.

  • Machine learning supports pervasive services by identifying patterns for prediction or classification when explicit modeling is difficult.
  • Cloud-based model deployment limits pervasive applications through data-security, communication-latency, and communication-cost concerns.
  • Federated learning trains models on edge devices, aggregates them at a server, and redistributes the resulting model for further local training.
  • FedDist is proposed as a new aggregation algorithm and evaluated against three representative algorithms on smartphone Human Activity Recognition.

A. Federated Learning principles

Federated learning repeatedly specializes client models and aggregates them into a shared model, making server-side aggregation central to the balance between generalization and personalization. FedAvg and FedMA emphasize whole-model aggregation, whereas FedPer preserves client-specific upper layers.

  • A. Federated Learning principles: Federated learning repeats local specialization, server aggregation, and redistribution across communication rounds.
  • A. Federated Learning principles: Aggregation strategies differ between whole-model generalization and client specialization through retained upper layers.
  • B. Federated Averaging (FedAvg): FedAvg aggregates client models by averaging corresponding parameters before sending the resulting model back to clients.
  • B. Federated Averaging (FedAvg): Coordinate-wise FedAvg averaging can combine functionally different neurons under non-IID data, producing sub-optimal results and requiring long recovery phases.
  • C. Federated Learning with Personalization Layers (FedPer): FedPer shares base layers for representation learning while clients retain upper layers associated with client-specific decision making.

D. Federated Matched Averaging (FedMA)

FedMA performs layer-wise aggregation by matching and merging similar neurons while allowing layer sizes to change. Although it can outperform FedAvg on computer-vision datasets, its permutation-matrix computation increases aggregation complexity and slows the process.

  • D. Federated Matched Averaging (FedMA): FedMA matches similar neurons layer by layer, fuses them into global neurons, and can add new neurons rather than fixing layer widths in advance.
  • D. Federated Matched Averaging (FedMA): FedMA uses neuron permutation invariance and non-parametric clustering to identify neurons that should be merged across client models.
  • D. Federated Matched Averaging (FedMA): FedMA outperforms FedAvg on computer-vision datasets but reduces communication at the cost of greater complexity and slower aggregation.
  • E. Synthesis: Federated-learning evaluations remain insufficiently structured for heterogeneous pervasive settings, especially regarding generalization, specialization, networks, and client behavior.

III. FEDDIST, A NEW FEDERATED LEARNING ALGORITHM

FedDist addresses heterogeneous, non-IID clients by identifying divergent neurons and adding client-specific neurons to the aggregated model. Its layer-wise process uses distance thresholds and retraining to preserve specialization while supporting broader generalization.

  • III. FEDDIST, A NEW FEDERATED LEARNING ALGORITHM: FedDist identifies client-specific neurons with euclidean distances and adds divergent neurons to the aggregated model instead of erasing them through coordinate averaging.The approach targets sparse or client-specific features and can produce larger models able to generalize better.
  • III. FEDDIST, A NEW FEDERATED LEARNING ALGORITHM: The algorithm computes pairwise neuron distances between client and server models, forming a cost-distance matrix before calculating each neuron's mean and standard deviation.These statistics characterize the direction taken by most clients and support threshold-based outlier detection.
  • III. FEDDIST, A NEW FEDERATED LEARNING ALGORITHM: Each full communication round distributes the server model, performs local client updates and weighted averaging, then evaluates neuron distances to modify the aggregated architecture.Figure 4 summarizes outlier identification and insertion into the model returned to clients.
  • III. FEDDIST, A NEW FEDERATED LEARNING ALGORITHM: A threshold penalty increases during training, limiting continual neuron addition; neurons whose client distance exceeds the threshold are added to the server model layer by layer.The process updates and freezes treated layers while subsequent layers adapt to newly added weights.
  • III. FEDDIST, A NEW FEDERATED LEARNING ALGORITHM: FedDist introduces intermediate layer-wise communication rounds, but once client models saturate without new neurons, learning becomes equivalent to FedAvg.The reported communication overhead is lower in practice when layers require no new units.

IV. EVALUATION METHOD

The evaluation method defines the datasets, baseline HAR models, evaluation strategy, and metrics needed for reproducible federated-learning comparisons.

  • IV. EVALUATION METHOD: The evaluation framework specifies datasets, baseline HAR models, evaluation strategy, and evaluation metrics for reproducible comparisons of federated-learning algorithms.

A. HAR Task and Datasets

The study selects a realistic smartphone HAR dataset to evaluate federated learning under heterogeneous, imbalanced, and device-diverse conditions. REALWORLD provides outdoor recordings across subjects, devices, body positions, sensors, and activities.

  • A. HAR Task and Datasets: The dataset-selection criteria prioritize freely accessible, real-life data with multiple participants and devices, high class imbalance, careful annotation, and sufficient duration for repeated local learning phases.
  • A. HAR Task and Datasets: HAR is suited to realistic federated-learning evaluation because smartphone datasets vary in sensors, sampling frequencies, protocols, time windows, and activity balance.Activity distributions can be highly imbalanced, such as standing at 14% and jumping at 2% in REALWORLD.
  • A. HAR Task and Datasets: REALWORLD was selected over the small, laboratory-acquired UCI dataset because it contains 125 hours of outdoor recordings from 15 subjects and seven device or body-position configurations.The data use accelerometer and gyroscope readings sampled at 50 Hz and cover eight labeled activities.

B. Baseline HAR models

The baseline-model study favors compact CNN architectures for smartphone HAR because they learn from raw data while reducing communication and on-device computing costs.

  • B. Baseline HAR models: Table I summarizes state-of-the-art HAR model performance and accuracy on the UCI test set, which serves as a commonly used benchmark despite limited size and variability.
  • B. Baseline HAR models: CNNs are widely used for HAR because they model features from raw data and are smaller than more complex architectures that only slightly outperform basic CNNs.The paper notes that comparisons are not fully uniform because some models use handcrafted features or different learning sets.
  • B. Baseline HAR models: The study uses standard CNN models tuned specifically for the HAR task as its selected baseline model family.

C. Evaluation strategy

The evaluation measures federated learning’s generalization, personalization, and model divergence under heterogeneous client data. It combines client-specific partitioning with classification and pair-wise dissimilarity metrics.

  • C. Evaluation strategy: The authors also note that the experiments were conducted in simulation, although FedDist was implemented on Google Pixel 2 devices for feasibility checking.The device implementation was not reported because of space limitations.
  • C. Evaluation strategy: Global, personalization, and generalization accuracies assess the aggregated model and client models across local and combined test data.The metrics distinguish overall performance from local specialization and cross-client generalization.
  • C. Evaluation strategy: Each client uses one participant’s records, with separate local training and test sets; concatenated client tests form the global test set.This design avoids equally distributing one dataset across clients and preserves realistic personalization.
  • D. Metrics: Macro F1 averages per-class F1 scores so performance on minority activities penalizes models that perform well only on majority classes.This addresses class imbalance in Human Activity Recognition.
  • D. Metrics: Pair-wise dissimilarity uses Euclidean distance between neuron-weight vectors to quantify differences between server and client models.Large distances indicate stronger dissimilarity and can identify clients that diverge from the server model.

A. Settings

The experiments use realistic smartphone activity data, multiple federated evaluation settings, and several implementation baselines. Centralized model selection identifies a CNN that substantially exceeds the prior REALWORLD benchmark, while local-only training exposes a generalization gap.

  • A. Settings: Experiments simulate 15 clients from individual REALWORLD participants, using 80% local training data and 20% local test data.Local test sets are aggregated into a global test set for evaluating the genericity-personalization trade-off.
  • A. Settings: The study evaluates global, personalization, and generalization accuracy, recall, precision, and F-score across the federated models.FedPer lacks a global model, so it cannot participate in the global-accuracy evaluation.
  • A. Settings: The smartphone inputs combine three-axis accelerometer and gyroscope data sampled at 50 Hz, with channel-wise normalization and windowed preprocessing.No handcrafted feature extraction was applied.
  • B. HAR model with Traditional Learning: The selected CNN reaches 92.48% F-Score on REALWORLD, exceeding the previous 81% state-of-the-art result.The model uses 196 convolutional filters, a 4x1 max-pooling layer, 1024 dense units, and an 8-unit softmax layer.
  • B. HAR model with Traditional Learning: Local-only models achieve a mean 96.04% F-Score on local tests but only 51.94% on the global test set, indicating strong personalization with poor generalization.The authors describe this pattern as overfitting.

C. Federated Learning

On the REALWORLD smartphone HAR experiments, FedDist achieved the strongest overall balance of generalization and personalization among the federated algorithms. FedAvg remained competitive, while FedPer and FedMA showed weaker generalization or personalization.

  • FedDist outperformed all other federated algorithms on global, personalization, and generalization accuracy, with 74.23% generalization accuracy.FedAvg reached 72.99%, compared with 60.09% for FedMA and 53.01% for FedPer on generalization accuracy.
  • FedAvg achieved 83.44% global-test F-Score and 72.99% generalization accuracy, remaining difficult to beat despite slower convergence.Its local-test F-Score was 95.82%, while centralized learning reached 92.48% global accuracy.
  • FedPer kept its dense personalized layer local, but reached only 53.01% generalization accuracy without significantly improving personalization.The authors associate this result with an overly strong personalization layer that does not react appropriately to new data.
  • FedMA reached 78.67% global accuracy and 60.09% generalization accuracy while requiring 25 local epochs per communication round.Its personalization accuracy was 93.65%, the lowest personalization result among the compared approaches.
  • FedDist’s two-step learning behavior reflects adding neurons or filters early and stabilizing after architectural growth stops.Its final model added 26 convolutional filters and 1,222 dense-layer units; a same-size FedAvg run underperformed the original FedDist implementation.
  • On the uniform UCI partition, FedAvg and FedDist both achieved a 96.96% global F-score because FedDist added no neurons and behaved like FedAvg.This result demonstrates that FedDist can adapt its behavior to a more uniform dataset.

VI. CONCLUSION AND FURTHER WORK

The paper concludes that FedDist combines FedAvg’s efficiency with architectural flexibility and outperformed the other federated algorithms across generalization and personalization measures. Further validation is needed across datasets, tasks, deployment conditions, and longer-term learning settings.

  • FedDist combines FedAvg’s efficiency with the ability to evolve the model architecture across communication rounds.The method automatically determines how many specialized neurons to add as training proceeds.
  • FedDist clearly outperformed the other federated algorithms on the three measures of generalization and personalization.The evaluation used a realistic smartphone Human Activity Recognition setting.
  • FedAvg also maintained generalization and personalization, whereas FedMA and FedPer did not show superior personalization while losing generalization.FedDist was more computationally intensive than FedAvg but less complex than FedMA and performed better on HAR.
  • The study should be replicated with more datasets and tasks and tested under asynchronous learning, changing client data, communication issues, and heterogeneous devices.The authors also identify server–client data mismatch as an open robustness condition.
  • Long-term studies remain necessary to optimize communication schedules and examine lifelong-learning effects such as catastrophic forgetting.The authors also call for benchmarks supporting comparison and replication.
Loading 2110.10223v1…