PaperlayerView exact revision on arXiv

Source-linked AI summary

Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge

Takayuki Nishio, Ryo Yonetani

arXiv:1804.08333v2cs.NIcs.LG

TL;DR

Heterogeneous computation and wireless resources can make privacy-preserving federated learning inefficient in practical cellular networks. The paper introduces FedCS, which selects and schedules clients using resource constraints so more updates can be aggregated within deadlines. Experiments on simulated MEC environments show that FedCS completes training significantly faster than the original FL protocol.

  • Problem

    Federated learning needs an efficient implementation for cellular networks with heterogeneous client computation, communication, and data resources.

  • Method

    FedCS uses resource-aware client selection and scheduling under deadlines to maximize the number of client updates aggregated in each round.

  • Results

    FedCS completed training significantly faster than the original FL protocol in simulated MEC experiments using large-scale image datasets.

  • Takeaways & Limitations

    FedCS provided high-performance models in shorter training times by incorporating more clients, across IID and Non-IID data settings and varying throughput uncertainty.

Abstract

from arXiv · show

We envision a mobile edge computing (MEC) framework for machine learning (ML) technologies, which leverages distributed client data and computation resources for training high-performance ML models while preserving client privacy. Toward this future goal, this work aims to extend Federated Learning (FL), a decentralized learning framework that enables privacy-preserving training of models, to work with heterogeneous clients in a practical cellular network. The FL protocol iteratively asks random clients to download a trainable model from a server, update it with own data, and upload the updated model to the server, while asking the server to aggregate multiple client updates to further improve the model. While clients in this protocol are free from disclosing own private data, the overall training process can become inefficient when some clients are with limited computational resources (i.e. requiring longer update time) or under poor wireless channel conditions (longer upload time). Our new FL protocol, which we refer to as FedCS, mitigates this problem and performs FL efficiently while actively managing clients based on their resource conditions. Specifically, FedCS solves a client selection problem with resource constraints, which allows the server to aggregate as many client updates as possible and to accelerate performance improvement in ML models. We conducted an experimental evaluation using publicly-available large-scale image datasets to train deep neural networks on MEC environment simulations. The experimental results show that FedCS is able to complete its training process in a significantly shorter time compared to the original FL protocol.

I. INTRODUCTION

Federated learning preserves client privacy by keeping data local, but heterogeneous resources and wireless conditions make practical cellular-network training inefficient. FedCS addresses this with resource-aware client selection and achieves significantly shorter training time than the original protocol.

  • I. INTRODUCTION: Federated learning trains models through iterative parameter exchanges while keeping private client data local.Clients download a global model, update it using local data, upload parameters, and the server aggregates updates.
  • I. INTRODUCTION: Limited computation or poor wireless channels lengthen client updates and delay the server’s aggregation step.These delays reduce the efficiency of the iterative training process in practical cellular networks.
  • I. INTRODUCTION: FedCS sets deadlines and selects resource-compatible clients so the server can aggregate as many updates as possible within limited time frames.The client-selection problem considers computation and communication constraints and is solved greedily.
  • I. INTRODUCTION: FedCS completed deep-neural-network training significantly faster than the original FL protocol in simulated MEC experiments.The evaluation used publicly available large-scale image datasets for object classification.
  • I. INTRODUCTION: Existing work has not sufficiently addressed efficient FL under heterogeneous client computation, communication, and data resources.Prior approaches focused on security, model compression, or hyperparameter optimization without specifically considering these heterogeneous resources together.

II. FEDERATED LEARNING

Federated learning trains a shared model from distributed private data without requiring clients to disclose that data. The protocol coordinates local updates and server aggregation across repeated rounds.

  • II. FEDERATED LEARNING: Federated learning addresses privacy concerns by allowing clients to keep their data in local storage during decentralized model training.Clients communicate model parameters rather than uploading private data.
  • II. FEDERATED LEARNING: Each round asks ⌈K × C⌉ random clients to receive, update, and upload global-model parameters for server aggregation.K denotes the total number of clients, while C is the fraction considered in each round.
  • II. FEDERATED LEARNING: The protocol requires participating clients to have sufficient computational resources for repeated forward- and backpropagation during model updates.The described setting focuses on supervised deep-neural-network training.

B. Heterogeneous Client Problem in FL

Standard FL does not account for heterogeneous client data sizes, computation, or channel conditions, creating bottlenecks in cellular-network training. FedCS is proposed to operate efficiently with such heterogeneous resources.

  • B. Heterogeneous Client Problem in FL: Larger client datasets can lengthen model updates, while poor channel conditions lengthen parameter uploads.Either condition can delay subsequent communication and aggregation.
  • B. Heterogeneous Client Problem in FL: Because aggregation waits for client updates, heterogeneous resources can become bottlenecks in the FL training process.A deadline that discards late updates may instead waste bandwidth and delayed-client resources.
  • B. Heterogeneous Client Problem in FL: FedCS proposes a new FL protocol designed to work efficiently with clients having heterogeneous resources.The proposal is introduced specifically for the heterogeneous-client problem.
  • B. Heterogeneous Client Problem in FL: The MEC platform is assumed to operate over stable, uncongested wireless networks while processes still face limited bandwidth.The setting focuses on periods such as midnight or early morning because models and communications are typically large.

B. FedCS Protocol

FedCS replaces random participation with resource-aware selection and scheduled uploads. It estimates client completion times, selects clients that meet a deadline, and iterates aggregation until a target performance or final deadline.

  • B. FedCS Protocol: FedCS requests resource information from random clients before selecting participants for the current training task.Reported information includes channel state, available computation, and task-relevant data size.
  • B. FedCS Protocol: The MEC operator estimates distribution and update-upload times, then selects clients that can complete those steps within a deadline.Selected clients receive the global model and update it in parallel before uploading parameters.
  • B. FedCS Protocol: FedCS distributes the global model by multicast and allocates radio resources for scheduled client uploads.The server aggregates the resulting updates and measures model performance after each round.
  • B. FedCS Protocol: FedCS repeats all post-initialization steps until the model reaches desired performance or the final deadline arrives.The protocol can use a target such as 90% classification accuracy as a stopping condition.
  • C. Algorithm for Client Selection Step: Client selection aims to maximize the number of updates aggregated before the deadline while scheduling upload resources to prevent congestion.The protocol assumes sequential uploads for simplicity, with the same total transmission time when uploads share resource blocks in parallel.

1. Initialization

FedCS initializes client selection by requesting resource information and choosing clients whose updates can be completed within a round deadline. Its optimization maximizes accepted client updates under timing constraints, using a greedy heuristic while balancing round size against the number of aggregations.

  • Initialization: The protocol requests client resource information before selecting clients for model update and upload.Relevant information includes wireless conditions, computational resources, and data resources.
  • Client Selection: Client Selection maximizes the number of selected client updates, |S|, subject to an estimated elapsed-time constraint.The elapsed time is evaluated through client update and upload completion.
  • Timing Model: Upload times accumulate sequentially, while model updates can proceed during the preceding clients’ upload periods.This overlap allows computation and wireless communication processes to share elapsed time.
  • Optimization Strategy: FedCS uses a greedy heuristic that repeatedly adds the client consuming the least update-and-upload time until the round deadline is reached.The algorithm returns the selected client set after iteratively processing the candidate set.
  • Round Deadline: Increasing T_round can involve more clients per round but reduces the number of possible aggregations before T_final.The evaluation examines how T_round choices affect final trained-model performance.

IV. PERFORMANCE EVALUATION

The performance evaluation simulates a MEC environment and tests the protocol on realistic machine-learning tasks using publicly available large-scale datasets.

  • IV. PERFORMANCE EVALUATION: The evaluation simulates a MEC environment and conducts realistic ML experiments using publicly available large-scale datasets.The experiments assess how the protocol works in a simulated edge-computing setting.

A. Simulated Environment

The simulated environment models an urban cellular MEC network with an edge server, base station, and heterogeneous clients. LTE wireless conditions and throughput-based timing estimates provide the communication setting for client selection.

  • Network Topology: The simulation contains an edge server, base station, and K = 1000 clients distributed across a 2 km-radius urban microcell.The base station and server are co-located at the cell center.
  • Wireless Model: Wireless communication follows an LTE-based urban Micro NLOS channel model with a 2.5 GHz carrier frequency.The model uses specified antenna heights and transmission settings.
  • Wireless Model: Each client receives 10 resource blocks, corresponding to 1.8 MHz, in 0.5 ms time slots.This models a practical bandwidth limitation for wireless transfers.
  • Timing Assumptions: Average client throughput is used to estimate upload time, with stable network conditions and sampled short-term variation during uploads.The stable-condition assumption reflects clients being likely unused and stationary during nighttime or early morning.
  • Timing Model: Upload time is calculated as t_UL_k = D_m/θ_k, while distribution time is based on D_m divided by the minimum throughput among selected clients.Client selection and aggregation time are assumed negligible because server computation is sufficiently high.

B. Experimental Setup of ML Tasks

The experiments train convolutional neural networks on CIFAR-10 and Fashion-MNIST under IID and Non-IID client-data distributions. Up to 100 of 1000 clients are selected per round, and testing data is reserved for classification evaluation.

  • Datasets: The study evaluates CIFAR-10 and Fashion-MNIST object-classification tasks using publicly available image datasets.CIFAR-10 has 10 object classes, while Fashion-MNIST contains 10 fashion-product classes.
  • Data Distribution: Training data is distributed among K = 1000 clients using both IID and Non-IID sampling procedures.Client data ownership is randomly assigned within a range of 100 to 1,000 images.
  • FL Configuration: Each FL round selects at most K × C = 100 clients with C = 0.1.The testing dataset is used only to measure classification performance.

C. Global Models and Their Updates

The evaluation uses a standard convolutional neural network and specified training hyperparameters to model global updates, with results reported for IID CIFAR-10 and Fashion-MNIST.

  • The global model is a convolutional neural network with approximately 4.6 million parameters.It uses six convolutional layers followed by three fully connected layers.
  • Each round uses mini-batches of 50, five epochs, an initial learning rate of 0.25, and learning-rate decay of 0.99.
  • Client update times varied from 5 to 500 seconds on average because computation capability differed across clients.The modeled range was intended to represent workstation and weaker mobile-device capabilities.
  • Table I reports time-to-accuracy and final accuracy for CIFAR-10 and Fashion-MNIST under the IID setting.ToA measures time to reach a target testing accuracy, while Accuracy is measured after the final deadline.

D. Evaluation Details

FedCS was evaluated against deadline-limited standard FL using time-to-accuracy and final accuracy, across IID and Non-IID image-classification settings and different round deadlines.

  • IID setting: FedCS outperformed FedLim in time-to-accuracy on both CIFAR-10 and Fashion-MNIST under the IID setting.At Tround = 3 and r = 0, FedCS reached 75% accuracy 76.5 minutes earlier on CIFAR-10 and 85% accuracy 33.3 minutes earlier on Fashion-MNIST.
  • IID setting: FedCS achieved higher final classification accuracy than FedLim, especially on CIFAR-10.Final accuracy was measured immediately after Tfinal.
  • IID setting: FedCS incorporated an average of 7.7 clients per round, compared with 3.3 clients for FedLim when Tround = 3.The larger number of participating clients was identified as one reason for improved training efficiency.
  • Effect of Tround: Round deadlines had a trade-off: long deadlines increased clients per round but reduced aggregation steps, while short deadlines limited accessible clients and degraded accuracy.The evaluation varied Tround while keeping Tfinal fixed.
  • Effect of Tround: Dynamic selection of Tround was left for future work because fixed deadlines could either limit clients per round or reduce the number of aggregation steps.
  • Non-IID setting: Under Non-IID data, FedCS still worked well, whereas FedLim failed to reach 50% and 70% accuracy on CIFAR-10 and Fashion-MNIST, respectively.Overall Non-IID performance was lower and more variable than under IID data.

V. CONCLUSION

FedCS is presented as an efficient FL protocol for heterogeneous MEC clients, with experiments showing shorter training time across datasets and data-partition settings. The study also identifies larger models and dynamically varying resources as future extensions.

  • FedCS provided high-performance models in significantly shorter time than the compared protocol by incorporating more clients into training.
  • This result held across datasets, IID and Non-IID data splits, and uncertainty in throughput and computation capability.
  • The study used sufficiently simple deep neural networks, leaving training of models with dozens of millions of parameters on very large-scale data as an extension.
  • Future work also includes more dynamic scenarios in which average resources and model update or upload times fluctuate.
Loading 1804.08333v2…