Source-linked AI summary
AdaptSFL: Adaptive Split Federated Learning in Resource-constrained Edge Networks
Zheng Lin, Guanqiao Qu, Wei Wei, Xianhao Chen, Kin K. Leung
TL;DR
Large models make on-device training difficult, and SFL’s system optimization under resource constraints remains insufficiently explored. AdaptSFL analyzes convergence and jointly adapts model splitting and client-side aggregation; simulations report that it reaches target accuracy in significantly less time than benchmarks.
Problem
Large models can make on-device FL training extremely difficult, while the effects of model splitting and client-side aggregation on SFL convergence have lacked unified analysis.
Source: §I · pp. 1–2Method
AdaptSFL derives a convergence bound for model splitting and client-side aggregation, then jointly optimizes these controls to reduce SFL training latency under loss and resource constraints.
Source: §I · p. 2Results
AdaptSFL achieves target accuracy with significantly less time than benchmarks, and its adaptive aggregation strategy improves convergence time by factors of 1.8 and 1.9 versus SFL with I = 1 under IID and non-IID settings.
Source: C. Ablation Study of The AdaptSFL… · p. 12, §VIII · p. 13Takeaways & Limitations
The analysis and simulations support controlling client-side aggregation and model splitting together to reduce SFL training latency under resource and data heterogeneity.
Source: §I · p. 2, §VIII · p. 13
Abstract
from arXiv · showhide
The increasing complexity of deep neural networks poses significant barriers to democratizing them to resource-limited edge devices. To address this challenge, split federated learning (SFL) has emerged as a promising solution by of floading the primary training workload to a server via model partitioning while enabling parallel training among edge devices. However, although system optimization substantially influences the performance of SFL under resource-constrained systems, the problem remains largely uncharted. In this paper, we provide a convergence analysis of SFL which quantifies the impact of model splitting (MS) and client-side model aggregation (MA) on the learning performance, serving as a theoretical foundation. Then, we propose AdaptSFL, a novel resource-adaptive SFL framework, to expedite SFL under resource-constrained edge computing systems. Specifically, AdaptSFL adaptively controls client-side MA and MS to balance communication-computing latency and training convergence. Extensive simulations across various datasets validate that our proposed AdaptSFL framework takes considerably less time to achieve a target accuracy than benchmarks, demonstrating the effectiveness of the proposed strategies.
I. INTRODUCTION
The introduction motivates adaptive SFL optimization as a way to balance training convergence and resource use, then presents AdaptSFL’s convergence analysis and joint control of client-side aggregation and model splitting. It also situates the framework against prior work and outlines its edge-network model.
I. INTRODUCTION
- Large models make on-device FL training difficult, while SFL shifts much of the workload to a server but still requires careful control of communication and computation.More frequent client-side aggregation and shallower cuts can improve convergence while increasing communication overhead.Source: §I · pp. 1–2
- AdaptSFL jointly adapts client-side model aggregation and model splitting to minimize SFL training latency while meeting target learning performance under resource constraints.The framework formulates the joint optimization using a convergence bound and training-loss and resource constraints.Source: §I · p. 2
- The convergence analysis characterizes how model splitting and client-side aggregation affect SFL convergence, addressing a gap the authors describe as uninvestigated.The paper presents this as the first SFL convergence analysis to characterize both effects.Source: §I · p. 2, §II · p. 3
II. RELATED WORK
- Prior FL aggregation-frequency work and SL split-point work do not resolve SFL’s joint optimization gap because SFL has distinct server- and client-side aggregation behavior.The paper identifies the missing unified convergence analysis and joint optimization under heterogeneous edge bandwidth and computing resources.Source: §II · p. 3
A. System Model
- The edge server trains server-side sub-models, while participating edge devices execute client-side forward and back-propagation.The server-side common sub-model is shared across clients and synchronized every round; the non-common sub-model is aggregated every I rounds with client-side sub-models.Source: A. System Model · p. 3
- The model assumes unbiased stochastic-gradient estimates and uniform local-dataset sizes across edge devices for concise expression.The stochasticity is attributed to the sampling order of mini-batches.Source: A. System Model · p. 4
III. THE RESOURCE-ADAPTIVE SFL FRAMEWORK
- AdaptSFL replaces per-round client-side aggregation with resource-adaptive client-side aggregation and model splitting to address communication inefficiency and workload tradeoffs.The system model includes an edge server and a fed server; the edge server hosts common and non-common server-side sub-models.Source: A. System Model · pp. 3–4
B. The AdaptSFL Framework
AdaptSFL combines split training in every round with client-side model aggregation every I rounds. Its resource-adaptive client-side aggregation and model splitting are designed to reduce training latency while achieving target learning performance.
B. The AdaptSFL Framework
- Split training runs in every training round, while client-side model aggregation occurs every I rounds until the model converges.The edge server initializes and partitions the model before training, then determines the client-side aggregation interval and split points.Source: B. The AdaptSFL Framework · p. 4
- Each device runs client-side forward propagation and sends its activations and labels to the edge server for server-side training.The server returns activation gradients so devices can back-propagate through their client-side sub-models.Source: B. The AdaptSFL Framework · pp. 4–5
- The edge server updates its common sub-model each round, equivalent to centralized SGD, while devices update their respective server-side non-common sub-models individually.Server-side non-common sub-models arise when devices have different numbers of layers on the server side.Source: B. The AdaptSFL Framework · p. 5
- Every I rounds, the fed server combines client-side sub-models with server-side non-common sub-models to forge and aggregate client-specific models.It then sends updated client-side and server-side non-common sub-models back to the relevant devices and edge server.Source: B. The AdaptSFL Framework · p. 5
IV. CONVERGENCE ANALYSIS OF ADAPTSFL
The analysis derives a convergence bound for SFL that accounts for client-side model aggregation and model splitting. It shows how these choices affect convergence and the communication–latency trade-off.
IV. CONVERGENCE ANALYSIS OF ADAPTSFL
- The analysis provides a convergence bound for SFL that characterizes the effects of client-side model aggregation and model splitting on training convergence.The paper presents this analysis as a basis for developing an efficient iterative method.Source: §IV · p. 5
- Compared with conventional FL, delayed client-side aggregation adds an error term proportional to I^2, while the maximum client-side depth Lc governs synchronization efficiency.A smaller Lc improves convergence speed; a larger Lc degrades convergence performance.Source: §IV · p. 6
- Fewer aggregation rounds and a shallower cut layer reduce the rounds needed to reach target convergence accuracy and can improve converged accuracy.These gains come with increased communication overhead and longer latency per training round, motivating joint optimization of aggregation and splitting.Source: §IV · p. 6
V. PROBLEM FORMULATION
The paper formulates joint optimization of client-side model aggregation and model splitting to minimize AdaptSFL training latency over heterogeneous edge networks.
V. PROBLEM FORMULATION
- The optimization objective is to minimize AdaptSFL training latency over edge networks with heterogeneous participating devices.The formulation is guided by convergence bounds that quantify the effects of aggregation and splitting on convergence and communication efficiency.Source: §V · p. 6
- The client-side aggregation decision I specifies how many training rounds occur between aggregations of forged client-side sub-models.I is a positive integer.Source: §V · p. 7
- The binary split variable μ_i,j selects whether layer j is the cut layer for edge device i.The collection μ represents the model-splitting decisions across devices and layers.Source: §V · p. 7
A. Training Latency Analysis
The latency analysis models the computation and communication costs of AdaptSFL’s split-training and client-side aggregation stages. It expresses these costs using device workloads, transmission rates, and the selected cut layers.
A. Training Latency Analysis
- The analysis estimates latency for one training round by modeling the split-training steps and the client-side aggregation stage that occurs every I rounds.It omits network and computing-capacity probing costs as negligible relative to training costs.Source: A. Training Latency Analysis · p. 7
- Client-side forward-propagation latency depends on each device’s per-sample workload, mini-batch size, and computing capability.Computing capability is measured in FLOPS, and the cut layer affects the workload.Source: A. Training Latency Analysis · p. 7
- Activation and activation-gradient transmission latency is modeled using the data sizes at the cut layer and the corresponding uplink and downlink rates.The cut-layer choice therefore determines the activation data sizes used in the transmission estimates.Source: A. Training Latency Analysis · p. 7
- Server-side forward- and back-propagation latency is estimated from per-sample workloads and the edge server’s computing capability.Client-side back-propagation latency is also analyzed for each device.Source: A. Training Latency Analysis · p. 7
- Client-side aggregation latency includes uplink and downlink transfers of client-side and server-side non-common sub-models, while the aggregation delay itself is ignored.The paper states that aggregation delay is typically much smaller than the other steps.Source: A. Training Latency Analysis · pp. 7–8
B. Joint Client-side Model Aggregation and Model Splitting
The section formulates joint client-side model aggregation and model splitting to minimize the time needed for convergence. The design must balance aggregation intervals and split points against both communication-computing overhead and convergence.
B. Joint Client-side Model Aggregation and Model Splitting
- The objective is to minimize the time consumed for model convergence by jointly choosing the client-side aggregation interval and model-splitting decisions.Source: B. Joint Client-side Model Aggregation… · p. 8
- The aggregation interval trades communication overhead against convergence, while model splitting affects communication-computing overhead and convergence.Source: B. Joint Client-side Model Aggregation… · p. 8
- Constraints require a positive-integer aggregation interval and ensure each edge device has a single cut layer; C1 guarantees global convergence accuracy.Source: B. Joint Client-side Model Aggregation… · p. 8
- The resulting combinatorial, non-convex mixed-integer nonlinear problem is NP-hard in general, making polynomial-time optimal solution infeasible.Source: B. Joint Client-side Model Aggregation… · p. 8
VI. SOLUTION APPROACH
The solution approach decomposes the joint optimization into tractable subproblems and solves them iteratively. It combines an aggregation-interval update with optimization of the model-splitting variables.
VI. SOLUTION APPROACH
- The joint problem is decomposed into subproblems over client-side aggregation and model-splitting variables, with auxiliary variables introduced to linearize the objective.Source: §VI · pp. 8–9
- For fixed µ and T, the optimal aggregation interval is selected from the floor or ceiling of I′, where I′ solves Ξ(I′) = 0 by Newton–Raphson.Source: §VI · p. 9
- For fixed I, the µ,T subproblem is formulated as a mixed-integer linear fractional program and reformulated as a mixed-integer linear program using Dinkelbach’s algorithm.Source: §VI · p. 9
- The block-coordinate descent algorithm alternates updates to I and to µ,T until the objective change is within εb; it may be rerun every I rounds or longer under dynamic conditions.Source: §VI · p. 10
VII. PERFORMANCE EVALUATION
The evaluation section describes simulations of AdaptSFL and its tailored aggregation and splitting strategies. The setup spans two CIFAR datasets, two data-distribution settings, and two neural networks.
VII. PERFORMANCE EVALUATION
- The evaluation is designed to assess AdaptSFL’s learning performance and the effectiveness of its client-side aggregation and model-splitting strategies.Source: §VII · p. 10
A. Simulation Setup
- The default setup uses 20 edge devices with varying compute and uplink rates across rounds, while the server has 20 TFLOPS and the inter-server link is fixed at 400 Mbps.Source: A. Simulation Setup · p. 10
- Experiments use CIFAR-10 and CIFAR-100 under IID and non-IID data distributions, with samples evenly shuffled for IID and split into 40 label-sorted shards for non-IID.Source: A. Simulation Setup · p. 10
- The models are VGG-16, with 13 convolutional and 3 fully connected layers, and ResNet-18, with 17 convolutional and 1 fully connected layer.Source: A. Simulation Setup · p. 10
B. Performance Evaluation of AdaptSFL Framework
AdaptSFL adapts client-side model aggregation (MA) and model splitting (MS) to reduce training latency while maintaining target performance. Evaluation reports faster convergence, resource robustness, and benefits from both adaptive strategies; the conclusion notes that real-world validation remains future work.
B. Performance Evaluation of AdaptSFL Framework
- At least 7.7 times convergence speed improvement over RMA+RMS is reported while guaranteeing training accuracy.Fig. 6 presents converged accuracy and time across CIFAR-10 and CIFAR-100, IID and non-IID settings, and VGG-16 and ResNet-18.Source: Figure 6 · p. 11, B. Performance Evaluation of AdaptSFL… · p. 11
- As network resources decline, AdaptSFL’s converged time increases only slightly, and Gaussian measurement noise causes only minor variations.The resource comparison uses CIFAR-10 under IID settings; the measurement-error experiment models fluctuations in data rates and computing capabilities.Source: Figure 7 · p. 11, Figure 8 · p. 11, B. Performance Evaluation of AdaptSFL… · p. 12
- AdaptSFL’s latency advantage over MA+RMS, RMA+MS, and RMA+RMS grows as the number of edge devices increases.The comparison covers CIFAR-10 under IID and non-IID settings.Source: Figure 10 · p. 12, B. Performance Evaluation of AdaptSFL… · p. 12
C. Ablation Study of The AdaptSFL Framework
- The proposed adaptive MA strategy achieves 1.5% and 5.1% higher convergence accuracy than PSL and improves convergence time by factors of 1.8 and 1.9 versus SFL with I = 1, under IID and non-IID settings, respectively.Source: Figure 11 · p. 12, C. Ablation Study of The AdaptSFL… · p. 12
- Deeper model split points reduce convergence accuracy and speed, with stronger effects under non-IID settings; the proposed MS strategy accelerates convergence while guaranteeing training performance.The paper attributes the effect to less of the global model being aggregated per round and greater sensitivity to local-data differences under non-IID settings.Source: C. Ablation Study of The AdaptSFL… · pp. 12–13, Figure 12 · p. 13
VIII. CONCLUSIONS
- AdaptSFL achieves target accuracy with significantly less time than benchmarks using resource-adaptive MA and MS under resource and data heterogeneity.Source: §VIII · p. 13
- Future work includes optimizing batch size and learning rate, accounting for probing costs in dynamic environments, and prototyping SFL on real-world edge systems.Source: §VIII · p. 13
APPENDIX
The proof fixes a training round and uses the latest earlier multiple of the client-side aggregation interval as a reference point.
A. PROOF OF LEMMA 1
- For round t, the proof chooses the largest t0 ≤ t satisfying t0 mod I = 0.Source: A. PROOF OF LEMMA 1 · p. 13
- The selected reference round satisfies t − t0 ≤ I.Source: A. PROOF OF LEMMA 1 · p. 13
- The proof invokes a vector-norm inequality for n = 2, n = t − t0, and n = N; its final step follows from Assumption 2.Source: A. PROOF OF LEMMA 1 · p. 13
B. PROOF OF THEOREM 1
The proof applies smoothness and expectation identities to bound the training-round terms, using the stated aggregation condition and Assumption 2.
B. PROOF OF THEOREM 1
- The proof begins by applying the loss function’s smoothness at training round t ≥ 1.Source: B. PROOF OF THEOREM 1 · p. 13
- The derivation uses zero-mean, independent client gradients and Assumption 2 to justify steps in bounding the gradient terms.Source: B. PROOF OF THEOREM 1 · p. 14
- For the proof’s I = 1 case, server-side sub-models are aggregated in every training round.Source: B. PROOF OF THEOREM 1 · p. 15
C. PROOF OF THEOREM 2
The proof analyzes the objective as a function of the client-side aggregation interval and locates its minimum using the function’s shape.
C. PROOF OF THEOREM 2
- The root I′ of Ξ(I) = 0 can be obtained using the Newton-Raphson method.Source: C. PROOF OF THEOREM 2 · p. 15
- The objective decreases and then increases with I, reaching a minimum at I = I′.Source: C. PROOF OF THEOREM 2 · p. 15
- Under constraint C4, when I′ > 1, the optimal I* is selected by minimizing Θ′(I) over floor(I′) and ceiling(I′).Source: C. PROOF OF THEOREM 2 · p. 15