PaperlayerView exact revision on arXiv

Source-linked AI summary

Mobility-Aware Cluster Federated Learning in Hierarchical Wireless Networks

Chenyuan Feng, Howard H. Yang, Deshun Hu, Zhiwei Zhao, Tony Q. S. Quek, Geyong Min

arXiv:2108.09103v1cs.LGcs.NIcs.SYeess.SY

TL;DR

The paper examines how user mobility affects learning in hierarchical federated learning, an aspect that prior work has seldom considered. It develops a mobility-aware clustered approach and analyzes HFL convergence under mobility. The analysis links increased mobility to parameter dropout and reports that more frequent communication can accelerate convergence.

  • Problem

    Few works have considered how user mobility affects learning performance in hierarchical federated learning.

    Source: §I · p. 3

  • Method

    The paper derives HFL convergence expressions accounting for data and network heterogeneity and mobility, then proposes a mobility-aware scheme for aggregating mobile-user and edge-AP parameters.

    Source: §VI · p. 22

  • Results

    The analysis finds that increased communication frequency can accelerate convergence, whereas increased user mobility leads to dropout among mobile-user and edge-AP parameters.

    Source: §VI · p. 22

  • Takeaways & Limitations

    The analysis identifies user mobility and communication frequency as factors associated with HFL convergence behavior.

    Source: §VI · p. 22

Abstract

from arXiv · show

Implementing federated learning (FL) algorithms in wireless networks has garnered a wide range of attention. However, few works have considered the impact of user mobility on the learning performance. To fill this research gap, firstly, we develop a theoretical model to characterize the hierarchical federated learning (HFL) algorithm in wireless networks where the mobile users may roam across multiple edge access points, leading to incompletion of inconsistent FL training. Secondly, we provide the convergence analysis of HFL with user mobility. Our analysis proves that the learning performance of HFL deteriorates drastically with highly-mobile users. And this decline in the learning performance will be exacerbated with small number of participants and large data distribution divergences among local data of users. To circumvent these issues, we propose a mobility-aware cluster federated learning (MACFL) algorithm by redesigning the access mechanism, local update rule and model aggregation scheme. Finally, we provide experiments to evaluate the learning performance of HFL and our MACFL. The results show that our MACFL can enhance the learning performance, especially for three different cases, namely, the case of users with non-independent and identical distribution data, the case of users with high mobility, and the cases with a small number of users.

I. INTRODUCTION

Prior work often treats hierarchical wireless networks as static, while mobile users can leave an access point during local training, disrupting participation. The paper analyzes HFL under mobility and proposes MACFL, reporting improved convergence and accuracy.

I. INTRODUCTION

  • The introduction positions cluster FL as suitable for hierarchical networks and notes that data heterogeneity can make conventional FL suboptimal.Cluster FL divides clients into groups that learn separate tasks and shared models; prior work [13] reports faster convergence than single-cell FL.Source: §I · p. 2
  • Mobile users may leave an access point during local training, reducing participant counts and potentially causing missing valid training data and an under-fitting global model.The introduction notes that prior mobility-aware FL works and do not select users with high mobility who may leave coverage.Source: §I · p. 3
  • The introduction identifies three design challenges: analyzing mobility's effects on HFL, clustering non-IID users under resource constraints, and improving equal-weight aggregation.The stated aggregation bottleneck arises from data-distribution divergence and user mobility.Source: §I · p. 3
  • The paper proposes MACFL to address HFL with mobile users by redesigning access, local updates, and aggregation.The design also introduces personalized local updates and attentive averaging.Source: §I · p. 3
  • The paper develops a theoretical framework deriving HFL convergence rates while accounting for user mobility and heterogeneity in both data and network architecture.The introduction identifies the lack of convergence analysis for HFL with mobile users as a research gap.Source: §I · p. 3
  • The experiments report improved convergence and accuracy for HFL with mobile users under the proposed schemes.The paper also studies network-parameter effects on learning performance.Source: §I · p. 3

II. SYSTEM MODEL

The system is a client-edge-cloud learning network in which users connect to nearby access points and models are aggregated at edge and cloud levels. User movement between neighboring access points is represented with a shared Markov transition model.

A. Network Setup

  • The network has mobile users, edge access points, and a cloud server, with users connecting to the nearest access point based on real-time locations.The paper assumes many more users than access points and uses physical accessibility for clustering.Source: A. Network Setup · p. 4
  • Users retain local datasets, so the global loss cannot be directly evaluated by servers and is minimized through federated computing.Training proceeds within each access point's cluster, followed by aggregation of edge models at the cloud; user-edge and cloud-edge communications occur at separate aggregation intervals.Source: A. Network Setup · p. 5

B. User Mobility Model

  • A shared Markov transition-probability matrix models users moving among neighboring access points during local computation.The matrix entry for a pair of access points gives the probability of moving between them; the model assumes users share the same matrix and begins from a uniform distribution.Source: B. User Mobility Model · pp. 6–7
  • For tractability, the mobility model considers a steady-state condition in which each cluster's incoming and departing users are balanced.The cluster size remains constant at equilibrium, although the users in each cluster can change.Source: B. User Mobility Model · p. 7

III. HFL WITH MOBILE USERS

Section III extends conventional HFL to mobile users and outlines its local-training, edge-aggregation, and cloud-aggregation procedure. Only users who remain in the same coverage area can upload to their original edge AP.

III. HFL WITH MOBILE USERS

  • Mobile-user HFL coordinates local updates with edge and cloud model aggregation.The section extends conventional HFL to mobile users and presents its general scheme in Algorithm 1.Source: §III · p. 7, A. Algorithm Description · p. 7

A. Algorithm Description

  • Users download a cluster model from their closest AP, update locally, and return the result for edge aggregation.The algorithm also includes cloud updates based on edge models.Source: A. Algorithm Description · pp. 7–8
  • Only users still in the same coverage area can upload successfully to their original edge AP; the cloud aggregates cluster models after every κ2 edge aggregations.The algorithm returns the global model after the cloud update.Source: A. Algorithm Description · p. 8

1) Local Update:

The procedure separates local SGD updates, edge-level cluster aggregation, and periodic cloud-level global aggregation. User mobility can change cluster membership between communication rounds, limiting which updates an edge AP can collect.

1) Local Update:

  • At each κ1-step interval, users download the latest cluster model from their tagged AP and perform local SGD updates.The local update uses an independently and identically sampled mini-batch from the user's local dataset.Source: 1) Local Update · p. 8

2) Edge update:

  • Because users may change clusters between rounds, an edge AP collects updates only from users connected to that cluster at the next aggregation.The upload succeeds only when the user's connection state matches the state required by the aggregation procedure.Source: 2) Edge update · p. 9

3) Cloud Update:

  • After every κ2 edge aggregations, edge APs upload cluster models to the cloud, which updates the global model and returns it to connected users.The assumption is that user connection states change during local training but remain stable during communication and aggregation.Source: 3) Cloud Update · p. 9, 2) Edge update · p. 9

B. Convergence Analysis

The convergence analysis bounds HFL's suboptimality under a learning-rate condition and identifies mobility, data divergence, and aggregation intervals as relevant factors. In particular, the bound is unbounded when no mobile user participates in aggregation.

B. Convergence Analysis

  • When ps = 0, the bound is unbounded and the training algorithm does not converge because no mobile user participates in model aggregation.The analysis says this case calls for a new aggregation scheme for highly mobile users.Source: B. Convergence Analysis · pp. 10, 12
  • Under the stated learning-rate condition, Theorem 1 bounds HFL's θ-suboptimality after T global aggregation rounds.The theorem specifies η < 1/(12Lκ1κ2).Source: B. Convergence Analysis · p. 11
  • The convergence error floor depends on initial states, loss-function divergences, SGD variance, and the rate of loss descent.The analysis identifies divergences among local, cluster, and global loss functions as contributors.Source: B. Convergence Analysis · p. 11
  • More frequent user–AP information exchange lowers θHFL; with κ1κ2 fixed, reducing κ1 helps more than reducing global-model exchange intervals.The analysis reports that θHFL decreases as κ1 or κ2 decreases, and that decreasing κ1 is preferable when their product is fixed.Source: B. Convergence Analysis · pp. 11–12
  • θHFL is not monotonic in ps; its dependence varies with the training data and local loss functions.The analysis separately notes that IID data yield smaller local and cluster loss divergences than non-IID data.Source: B. Convergence Analysis · p. 12

IV. MOBILITY-AWARE CLUSTER FEDERATED LEARNING

MACFL redesigns access, local updates, and aggregation for hierarchical wireless networks with mobile users. It combines personalized local and cluster learning with attentive collaboration to address mobility and data heterogeneity.

IV. MOBILITY-AWARE CLUSTER FEDERATED LEARNING

  • MACFL redesigns the access mechanism, local update rule, and weighted model aggregation scheme.The access mechanism lets a user download a cluster model at one AP and upload its updated parameters at another, increasing successful participation in aggregation.Source: §IV · p. 12

A. Learning Task

  • The learning task jointly learns shared cluster and global models alongside personalized local models for mobile users.The personalized local model is learned from the shared model with an additional gradient-descent step.Source: A. Learning Task · p. 13
  • MACFL introduces attentive collaboration and personalized cluster learning to address mobility and data heterogeneity without leaking private data.The paper states these features can boost collaboration effectiveness and mitigate the impact of mobility and data heterogeneity.Source: A. Learning Task · p. 13

B. Algorithm Description

MACFL combines user-side local updates with attention-weighted edge and cloud aggregation. Its updates account for users moving between access points and clusters.

1) Local Update:

  • Users download the latest personalized model from their nearest edge AP, then perform κ1 local SGD steps using mini-batch gradients.The gradient estimate uses a first-order approximation to avoid the high computation cost of a Hessian term, with a slight loss of accuracy.Source: 1) Local Update · pp. 13–14

2) Edge Update:

  • Because mobile cluster membership changes and equal or data-size weighting may not reflect users’ mixture distributions, MACFL redesigns edge aggregation.It uses learnable attention weights based on model similarity, with one scalar hyperparameter; more complex attention methods may add computation overhead.Source: 2) Edge Update · p. 14
  • The edge attention coefficient weights each user’s model in AP aggregation, using cosine similarity between that model and the cluster model.After the edge update, the edge server distributes the cluster model to its connecting users.Source: 2) Edge Update · p. 14

3) Cloud Update:

  • Every κ2 edge communication rounds, the cloud collects cluster models, updates the global model with attention coefficients, and sends it back to edge servers.The edge servers distribute the shared model to their connected users for a new round.Source: 3) Cloud Update · p. 15

C. Convergence Analysis

The paper analyzes HFL convergence with mobile users and evaluates how mobility and system settings affect test accuracy. Experiments show that more frequent aggregation benefits HFL, particularly with non-IID data.

C. Convergence Analysis

  • Theorem 2 gives an ε_MACFL-suboptimality result for Algorithm 2 after T global aggregations, subject to a learning-rate condition.The supplied theorem passage does not expose the full bound or condition.Source: C. Convergence Analysis · p. 15

A. Experimental Settings

  • Experiments evaluate cluster FL on heterogeneous model and dataset configurations, using 50 users, five clusters, and MNIST with IID and pathological non-IID data.In the non-IID setting, each user has at most two digit categories.Source: §V · p. 15, A. Experimental Settings · pp. 15–16

B. Impact of User Mobility on Conventional HFL

  • User mobility can reduce participants and omit training data, while also shuffling data in ways that may reduce divergence among clusters.The resulting samples may be more representative at each global-model SGD step.Source: B. Impact of User Mobility on… · p. 16

1) Impact of the Frequency of Aggregation:

  • Reducing κ1 or κ2 improves HFL convergence in IID and non-IID cases; at a fixed product κ1κ2, decreasing κ1 accelerates convergence.κ1 and κ2 represent edge and global aggregation intervals, respectively.Source: 1) Impact of the Frequency of… · p. 17
  • Higher frequency of local averaging, instead of a large number of local update iterations, benefits HFL learning performance; IID data performs better than non-IID data, with smaller local and cluster loss divergences and SGD variance.Source: 1) Impact of the Frequency of… · p. 17

2) Impact of Staying Probability:

The experiments compare conventional HFL with MACFL under varying staying probability and user counts. MACFL performs better in the reported settings, with particularly large gains under high mobility and non-IID data.

2) Impact of Staying Probability:

  • For HFL, increasing ps improves non-IID learning, while IID performance changes mildly as long as ps is positive.When ps = 0, edge servers do not collect updated results; increasing ps retains more effective participants in each cluster.Source: 2) Impact of Staying Probability · pp. 17–18

3) Impact of the Number of Users:

  • Increasing M improves HFL convergence for both IID and non-IID data and reduces convergence-curve fluctuations, especially when global aggregations are few.For IID data, the paper links more users to reduced SGD variance per iteration.Source: 3) Impact of the Number of Users · p. 18

C. Comparison between HFL and Our Proposed MACFL Algorithm

VI. CONCLUSION

The conclusion analyzes hierarchical FL with mobile users and proposes MACFL by exploiting correlations between users’ and edge APs’ parameters. It reports that communication frequency and mobility affect convergence, while MACFL’s experimental gains are particularly pronounced for non-IID data.

VI. CONCLUSION

  • The convergence analysis accounts for dataset and network-architecture heterogeneity as well as user mobility.It derives analytical expressions for FL convergence rates in the conventional setup.Source: §VI · p. 22
  • Increasing communication frequency can accelerate convergence, whereas greater user mobility causes participant dropout and decreases the convergence rate.The analysis also notes that communication frequency among users and their connected APs matters.Source: §VI · p. 22
  • MACFL exploits correlations between mobile users’ and edge APs’ parameters in its proposed mobility-aware scheme.The approach aggregates users’ parameters.Source: §VI · p. 22
  • MACFL’s experimental gain over conventional training is particularly pronounced when mobile users have non-IID datasets.The proposed approach’s efficacy is corroborated by extensive experiments.Source: §VI · p. 22

APPENDIX

The appendix proofs use averaged virtual models and assumptions about gradients, mobility, and loss functions to derive bounds. The supplied passages cover the proofs of Lemmas 1 and 2.

A. Proof of Lemma 1

  • Lemma 1’s proof uses averaged global model parameters as a virtual analysis quantity, although they are not observed at each iteration.The derivation assumes unbiased mini-batch gradients with bounded variance and independence between mobility and mini-batch sampling.Source: A. Proof of Lemma 1 · pp. 22–23
  • The Lemma 1 proof identifies a descent term that is negative when the learning rate is sufficiently small.The stated condition includes ηL(ps + αum(1 − ps)) ≤ 1.Source: A. Proof of Lemma 1 · pp. 24–25

B. Proof of Lemma 2

  • Lemma 2’s proof analyzes mean-square error for local and cluster model parameters, using independence among the variables in its bound.The proof then plugs intermediate bounds into the inequality and averages over time.Source: B. Proof of Lemma 2 · pp. 25–26

C. Proof of Lemma 3

The supplied appendix passages describe the proof strategies for Lemma 3 and Theorem 2. They use virtual cluster models for analysis and account for edge aggregation participation in MACFL.

C. Proof of Lemma 3

  • Lemma 3’s proof uses virtual averaged cluster parameters and analyzes cluster and global model error at synchronization intervals.The connected edge AP is indexed by the AP serving each mobile user.Source: C. Proof of Lemma 3 · p. 26

D. Proof of Theorem 2

  • Theorem 2’s proof accounts for all mobile users participating in edge-model aggregation at each iteration.It derives an upper bound on the mean-square error of local and cluster model parameters, under the stated setup of common model initialization.Source: D. Proof of Theorem 2 · pp. 28–29

Related papers

Loading 2108.09103v1…