Source-linked AI summary
Split Federated Learning: Speed up Model Training in Resource-Limited Wireless Networks
Songge Zhang, Wen Wu, Penghui Hu, Shaofeng Li, Ning Zhang
TL;DR
Resource-limited wireless networks make FL communication-heavy and SL latency-prone as client participation grows. GSFL combines split learning with group-based federated training, and simulations report lower latency with satisfactory accuracy than vanilla SL and FL.
Problem
FL suffers communication overhead from uploading large models, while SL can incur long training latency with multiple clients in resource-limited wireless networks.
Method
GSFL partitions clients into groups and trains split client-side and server-side models through distribution, group training, and model aggregation.
Results
GSFL achieves accuracy comparable to SL and centralized learning, nearly 500% faster convergence than FL, and shorter delay than SL.
Takeaways & Limitations
GSFL reduces training latency while preserving high model accuracy in the reported simulations.
Abstract
from arXiv · showhide
In this paper, we propose a novel distributed learning scheme, named group-based split federated learning (GSFL), to speed up artificial intelligence (AI) model training. Specifically, the GSFL operates in a split-then-federated manner, which consists of three steps: 1) Model distribution, in which the access point (AP) splits the AI models and distributes the client-side models to clients; 2) Model training, in which each client executes forward propagation and transmit the smashed data to the edge server. The edge server executes forward and backward propagation and then returns the gradient to the clients for updating local client-side models; and 3) Model aggregation, in which edge servers aggregate the server-side and client-side models. Simulation results show that the GSFL outperforms vanilla split learning and federated learning schemes in terms of overall training latency while achieving satisfactory accuracy.
I. INTRODUCTION
FL incurs communication overhead from large model uploads, while SL faces long latency with multiple clients; GSFL addresses these constraints through group-based parallel training.
- I. INTRODUCTION: FL suffers from communication overhead because clients upload large AI models, whereas SL incurs long training latency when many clients participate.
- I. INTRODUCTION: A prior hybrid federated split-learning scheme reduces communication and client computation overhead but requires many server-side models, creating prohibitive storage demand.
- I. INTRODUCTION: GSFL uses group-based training to circumvent sequential client processing and reduce overall training latency.The scheme operates in a split-then-federated manner and supports parallel training across clients in different groups.
- I. INTRODUCTION: GSFL is designed to speed training relative to conventional SL and FL while maintaining satisfactory accuracy.
II. GROUP-BASED SPLIT FEDERATED LEARNING SCHEME
GSFL partitions clients into groups that collaboratively train local models while an edge server hosts identical server-side models, then distributes, trains, and aggregates model components.
- II. GROUP-BASED SPLIT FEDERATED LEARNING SCHEME: GSFL partitions N clients into M groups, where clients within each group collaboratively train a local AI model without sharing local data.The access point connects the clients to an edge server with abundant computation and storage resources.
- II. GROUP-BASED SPLIT FEDERATED LEARNING SCHEME: The scheme has three stages: model distribution, model training, and model aggregation.Distribution sends client-side models, training completes split learning, and aggregation combines the trained models into the whole model.
A. Step 1: Model Distribution
During model distribution, the access point partitions the original AI model into segments and sends client-side models to the first-trained client in each group.
- A. Step 1: Model Distribution: The AP partitions the original AI training model into segments to obtain initial server-side and client-side models.
- A. Step 1: Model Distribution: The AP distributes client-side models to the first-trained clients in each group over wireless links.
B. Step 2: Model Training in Each Group
Within each group, clients perform split learning by sending smashed data to the AP, receiving gradients for local updates, and passing trained client-side models onward.
- B. Step 2: Model Training in Each Group: Clients sample local mini-batches, perform client-side forward propagation, and transmit smashed data to the AP for server-side forward propagation.
- B. Step 2: Model Training in Each Group: The AP computes loss and gradients, updates the server-side model, and returns smashed-data gradients so clients can update their client-side models.
- B. Step 2: Model Training in Each Group: After local training, each group passes the trained client-side model through the AP to the next client, and the final client sends it back to the AP.
C. Step 3: Model Aggregation among Groups
After all groups finish training, the access point aggregates server-side and client-side models into new models, completing one training round.
- The AP aggregates all server-side models and all client-side models after group training completes.
- Model aggregation can use FedAVG to combine the trained models.
- The aggregated model enters the next training round until satisfactory accuracy is reached.
III. INITIAL SIMULATION RESULTS
Simulations on GTSRB compare GSFL with centralized learning, vanilla split learning, and federated learning. GSFL preserves accuracy comparable to SL and CL while converging faster and reducing delay relative to the baselines.
- The simulation uses 30 clients divided into 6 groups and evaluates accuracy across training rounds and delay.
- About 31.45% lower delay makes GSFL faster than vanilla SL while achieving faster convergence.
- GSFL reaches accuracy comparable to vanilla SL and centralized learning on GTSRB.
- GSFL improves convergence speed over FL by nearly 500%.
IV. CONCLUSION AND FUTURE WORK
The paper concludes that GSFL reduces training latency while preserving high model accuracy. Future work will examine cut-layer selection, client grouping, and resource allocation.
- GSFL greatly reduces training latency while preserving high model accuracy.
- Future work will study how cut-layer selection and client grouping affect system performance.
- Future work will design resource allocation for communication bandwidth and computing resources.